General-Purpose Cloud Compute For Agentic AI: Architectural Evolution For Scalable Inference, Reasoning, and Orchestration

Abstract

The rapid evolution of generative artificial intelligence is transforming the requirements placed on cloud infrastructure. While graphics processing units (GPUs) remain central to accelerating large language model (LLM) inference, the emergence of Agentic AI systems introduces demands that extend well beyond accelerator throughput. Agentic AI performs iterative reasoning, long-horizon planning, memory retrieval, tool invocation, and multi-agent collaboration, requiring continuous coordination across compute, networking, storage, and virtualization layers that traditional inference-only workloads never exercised. This paper examines how general-purpose cloud compute has architecturally evolved to support these coordination demands. It synthesizes recent literature on agentic reasoning frameworks, LLM inference-serving systems, memory-augmented agent architectures, and composable infrastructure technologies such as Compute Express Link (CXL) to develop the Agentic AI Compute Stack, a five-layer conceptual framework describing how general-purpose compute operates as the orchestration backbone connecting enterprise applications to AI accelerators. The paper further analyzes workload-aware scheduling, heterogeneous CPU-GPU coordination, and memory-management techniques as the specific architectural mechanisms that determine whether agentic workloads scale efficiently. It argues that the future scalability of Agentic AI depends on optimizing the complete infrastructure stack rather than accelerator performance in isolation, and that this reframing carries direct implications for how cloud providers architect, schedule, and provision the infrastructure underneath autonomous AI systems. The discussion concludes by identifying open architectural questions for infrastructure research as agentic workloads continue to grow in scale and autonomy.