What AI Changes About Enterprise Network Design

When organizations talk about preparing enterprise networks for AI, bandwidth is often one of the first questions.

That makes sense. More AI use means more traffic, and if agents become common across the enterprise, there could eventually be a lot more of it.

But AI agents change more than traffic volume. They also change how work moves across the network.

Imagine asking an agent to review last month's cloud spending and explain what changed. An employee sees one request and waits for one answer. Behind that request, the agent may need to talk to a model, sign in to a cloud service, pull billing data, check other systems, gather more information, and return to the model several times before the task is finished.

For the employee, all of that is one piece of work.

For the network, it may be dozens of separate interactions.

That difference is worth thinking about as enterprises plan for more AI use.

For many business applications, people still help set the pace. We open an application, wait for it to respond, read what is on the screen, and decide what to do next. Automated systems have been communicating with each other for decades, so machine-to-machine traffic is certainly not new. What AI changes is how easily a normal employee request can now create a much larger chain of machine activity behind it.

Once the task starts, the software can keep moving between models, applications, data sources, and services without waiting for a person between each step.

That makes AI network planning about more than how much bandwidth will be needed.

It also raises questions about what the traffic looks like, where it needs to go, how many systems a task depends on, and how consistently those systems can communicate with each other.

AI traffic is not simply more of the same

We are starting to get useful data on how AI traffic behaves outside the data center.

Cisco measured live AI inference traffic moving across service provider networks and found some important differences from normal web traffic. AI inference connections lasted about twice as long, while regular web traffic had a median flow rate about ten times higher. AI traffic was also much more likely to send significant amounts of information upstream.

That does not look like a simple story of AI flooding networks with very large transfers.

A prompt and a text response are small compared with video, software downloads, backups, and many of the other things enterprise networks already carry. An AI connection may move data at a fairly modest rate while staying open longer.

Agents add another dimension because they can create many more interactions while completing a task.

In one Cisco test, an AI agent performing a research task generated up to 450% more network traffic than a person doing the same work manually. While that number shouldn't apply to every agent or business process, it's interesting to see why the traffic increased.

The agent kept communicating with other systems as it worked.

That distinction matters when planning a network. The question is not only how large an individual AI transaction might be. It is also how many transactions may sit behind an important business task and how those transactions depend on one another.

Start with the task

I think this may be the most useful place to begin when planning for AI workloads.

Take an important task the business expects an agent to perform and work backward from it.

What does the agent actually need in order to finish?

It may need a model running in a public cloud. It may need data from an internal application. It might call a software service hosted somewhere else. It could retrieve information from a database, use another tool, and return to the model several times before the employee gets a result.

Suddenly, the task is spread across infrastructure the organization owns, infrastructure someone else owns, and several network paths connecting all of it.

That is a different planning conversation from estimating how much additional internet bandwidth the company might need next year.

It also helps explain why network quality becomes so important.

Suppose that task requires 30 separate interactions. None of them has to use much bandwidth, and the network may never come close to full capacity.

The task still depends on those interactions working consistently.

Some may need to happen in a specific order because the next step depends on the result of the previous one. Others may happen at the same time. Some may cross the public internet. Others may stay inside a cloud environment or return to a private application.

A small delay in one interaction may not matter much. Repeated across a long sequence of dependent requests, it can matter more.

Amazon Web Services makes a similar point in its guidance for AI agents. Calls to outside tools happen as part of the agent's work, so connection setup, retries, timeouts, and tool performance all contribute to the time required to complete the task.

OpenAI has seen another side of the same issue. As its models became faster, some of the overhead around the model became easier to see. The company reduced unnecessary network trips and introduced persistent connections for longer-running agent workflows. In some tool-heavy workflows, those changes improved end-to-end execution time by as much as 40 percent.

Neither example is really about buying a larger circuit.

They are about how efficiently and consistently the different parts of an AI workload communicate.

That is why I would want to understand the task before deciding what the network needs.

Where does the work actually go?

This becomes one of the first questions I would bring into an AI network planning discussion.

Where is the model running?

Where does the agent get the data it needs?

Which applications will it call?

Which services are outside the organization?

Does the task stay mostly inside one cloud environment, or does it move between several?

How often does it need to return to a private data center or another part of the enterprise?

The answers matter because the path becomes part of the application.

Two agents may use the same model and create roughly the same amount of traffic, yet place very different demands on the network because one stays close to its data while the other moves repeatedly between several environments.

This is not a new networking principle. Location and path quality have always mattered. AI just gives us another reason to understand them.

Instead of looking only at how an employee reaches an application, we also need to understand how the applications, models, tools, and data involved in that employee's task reach one another.

How much of the work has to happen in sequence?

This is another question I would want answered early.

If ten operations can happen at the same time, the network faces a different problem than if all ten must happen one after another.

Sequential work makes each interaction's quality more important because the next step may not begin until the current one finishes.

That does not automatically mean the network will become the bottleneck. The model, application, database, or outside service may still account for most of the delay.

But as model response times improve, the other parts of the workflow become easier to notice.

This is why I would be careful about using average bandwidth or even average network performance as the main measure of AI readiness.

The network can look healthy overall while one important path or dependency creates a poor result for the task.

Planning around the task helps expose those dependencies before AI use reaches a much larger scale.

Where does security sit in the path?

Network planning for AI has to include security from the beginning.

Agents may work with financial systems, customer data, source code, internal applications, cloud platforms, and other information the business needs to protect.

That means there will be authentication, access decisions, inspection, data protection, and policies controlling what information can move between systems.

Those controls become part of the same path the agent is using to complete the task.

An agent that works across several applications may also cross several security boundaries. That makes it important to understand where controls are being applied and how they affect the full workflow. The answer is not to remove inspection or weaken controls so AI can move faster.

The planning question is whether those controls are placed and designed to support the work without adding unnecessary friction every time the agent moves between systems.

This is where network and security planning become difficult to separate. The route traffic takes, the controls applied to it, and the application experience are all connected.

What happens when something does not respond as expected?

If an application is slow, a person may wait a few seconds before trying again. Software can retry almost immediately.

At small scale, that does not matter much.

At larger scale, thousands of agents responding to the same struggling service could create additional traffic at exactly the wrong time, creating a denial-of-service-type situation. This is one reason retry behavior, timeouts, alternate paths, and dependency failures belong in the planning discussion.

The question is not simply whether a connection stays up. It is whether an important AI task can continue when one of the services it depends on becomes slow or unavailable, and whether the agent's response to that failure creates another problem.

That is a resilience question, but it is also a network design question.

Can we see the task as a whole?

The last question I would ask is whether the organization can see enough of the workflow to understand how it is behaving.

That does not mean every networking team needs to become an AI development team.

It means that if an important AI workload depends on a model, three applications, two cloud services, a database, and several security controls, looking at any one connection in isolation may not tell us very much about the health of the task.

Microsoft recommends looking at the broader request path for AI applications, including retrieval, outside tools, orchestration, network round trips, retries, and slower requests that may disappear inside averages.

That type of visibility is useful for troubleshooting, but I think it is just as important for planning.

You cannot design around a workload well if you don't understand where it goes or which parts of the path matter most. Over time, that visibility can also show whether the assumptions used during planning were right.

Maybe bandwidth does become the problem.

Maybe one cloud path becomes more important than expected.

Maybe an outside service creates most of the delay.

Maybe security processing needs to move closer to the applications and data.

The point is to see the workload clearly enough to make those decisions based on how it actually behaves.

Planning for a different pace of work

Enterprise networks have adapted to major changes before.

Applications moved from corporate data centers to the cloud. Employees became more distributed. The public internet became part of the path to applications that businesses depend on every day.

AI looks like another step in that evolution.

This time, one change is the pace at which applications themselves can work.

An employee can give an agent a task and then step out of the process while software moves between models, applications, data, and services on that person's behalf.

That is why I would not begin an AI network planning discussion with only one question about additional bandwidth.

I would begin with the work.

What does an important AI task need to communicate with? Where do those systems live? Which interactions depend on one another? Where will security controls be applied? What happens when a dependency fails? And can we see enough of the full path to understand how the workload behaves?

Capacity will still matter. In many environments, it may matter a great deal.

But as more work moves at machine pace, the quality, consistency, and design of the communication between systems may matter just as much as the amount of traffic they carry.

That feels like the more useful place to start planning.