AWS and Cloudflare Redesign Infrastructure for the Era of AI Agents

The Arrival of a Machine-Dominated Internet

For years, the foundation of the internet has been designed around predictable human behavior—typing keywords into search boxes, clicking on links, scrolling through pages, and streaming videos. However, this assumption is now being fundamentally challenged. Autonomous software known as AI agents is transitioning into full-scale operation, causing machine-generated internet traffic to grow rapidly to the point where traditional infrastructure struggles to keep up. Tech giants like Amazon Web Services (AWS) and Cloudflare are taking the lead in redesigning foundational technologies. The goal? To rebuild the internet not for humans, but for machines—a paradigm shift now spreading across the tech industry.

The “Unusual”

Traffic Generated by AI Agents AI agents differ significantly from traditional web usage due to their traffic patterns. Unlike human users who slowly browse pages, AI agents activate multiple sub-agents within seconds, execute hundreds of database queries, search documents, and call APIs. Then, just as quickly as they appear, they vanish. These “bursting and massive” traffic patterns create unforeseen loads on traditional infrastructure. Tia White, General Manager of Amazon OpenSearch Service, told TechCrunch, “Agents are moving from experimental stages to full-scale operations, generating traffic patterns that existing infrastructure was never designed for.” White further explained, “Agents spike unpredictably and idle without notice. Companies need search capabilities that can adapt to these fluctuations without incurring costs for idle computing resources,” highlighting the limitations of existing infrastructure.

AWS’s New OpenSearch Serverless To address

these challenges, AWS unveiled its next-generation OpenSearch Serverless on May 28, 2026. This fully managed search and vector database system is optimized for the workloads of AI agents. Vector databases, capable of efficiently storing and searching high-dimensional data vectors generated by AI, are indispensable for processing vast amounts of information at high speeds. The most significant technological shift lies in the separation of computing and storage. Previously, storage and computing were integrated, requiring at least one instance to run continuously, incurring costs even during idle times. The new version scales computing up within seconds when agents execute tasks and down to zero during idle periods, eliminating costs during downtime. According to AWS, there will be zero costs when idle.

A Future Dominated by Machine Traffic

Although AI agents currently account for a relatively small proportion of overall internet activity, machine-generated traffic has already reached significant levels. Data from Cloudflare reveals that bots accounted for 31% of total HTTP traffic over the last six months. Of that, AI crawlers, search engines, and AI assistants made up about a quarter of all bot requests, equating to approximately 8% of total internet traffic. Cloudflare’s Senior Product Manager Lai Yi Ohlsen told TechCrunch, “By early 2027, non-human traffic will surpass human traffic.” If this prediction holds true, it means machines will become the primary users of the internet.

Google Declares the “Agent Era”

This movement is not limited to AWS and Cloudflare. At its developer-focused conference “Google I/O” in May 2026, Google announced that users would be able to delegate tasks to AI systems. These tasks include everyday activities like shopping research, travel bookings, web browsing, and app operations—all handled by AI agents. The shift isn’t restricted to consumer-facing AI agents. Businesses are increasingly adopting agents for internal operations and customer services, leading to a surge in new machine-to-machine communication traffic that IT departments must adapt to.

The Inevitable Redesign of Cloud

Infrastructure Underlying all these changes is a shared challenge: adapting systems originally built for a human-centered internet to an era dominated by autonomous agents that continuously retrieve information, call tools, and generate machine-to-machine traffic. AWS’s refreshed OpenSearch Serverless is part of this broader effort. Its architecture, which separates computing and storage and scales up or down instantly based on demand, is a direct response to the unpredictable traffic generated by AI agents. Traditional cloud services were optimized for human access patterns—daytime traffic increases and nighttime decreases. In contrast, AI agents generate traffic independent of human activity cycles, with thousands of agents potentially activating simultaneously before ceasing activity moments later. Addressing such extreme fluctuations requires a fundamental overhaul of infrastructure architecture.

A Paradigm Shift in Infrastructure Design

This transformation in internet infrastructure design is more than a mere technical optimization. It signifies a fundamental shift in the very nature of the internet—from a tool designed for human use to a foundation for machines to process and exchange information. As AI agents transition from experimentation to real-world applications, cloud infrastructure must evolve from “human-centric” to “machine-centric.” The coordinated efforts of tech giants like AWS, Cloudflare, and Google underscore the speed and scale of this change. As the adoption of AI agents accelerates, cloud infrastructure could be rewritten entirely. The internet may evolve from a tool for human convenience to a platform where machines interact with and process information—and this transformation is already underway.