Cross-Layer Co-Design of Heterogeneous Multi-Core Scheduling and Prompt Optimization for Edge-AI in Smart Cities and Telemedicine

Abstract

Edge Artificial Intelligence has emerged as a tentative paradigm to address low-latency and privacy constraints in smart cities and telemedicine, yet deployment remains bottlenecked by computational scarcities when hosting Large Language Models. This paper presents a cross-layer co-design framework that orchestrates heterogeneous multi-core task scheduling alongside semantic prompt optimization. Rather than optimizing hardware and software in isolation, we propose a dynamic feedback loop where upper-level prompt compression actively mitigates Key-Value cache overheads, while lower-level multi-core schedulers dynamically reallocate CPU-GPU-NPU clusters based on real-time task urgency. Empirical implementation on heterogeneous edge platforms encountered non-linear latency spikes and memory fragmentation during multi-task concurrency, requiring iterative heuristic calibrations. Our findings suggest that while semantic pruning preserves text utility to some extent, potential biases in clinical inference could emerge under extreme compression, implying that a completely linear optimization flow remains elusive. Ultimately, the co-design approach offers a possible path toward reconciling the resource-constrained nature of edge hardware with the heavy computational demands of specialized LLM tasks, though further research is needed to quantify long-term system stability across more erratic workloads.