Research · Latency · Network

Where the milliseconds go: the network cost of robot policies in the cloud

Edge computers now cost more than many consumer robots, so the model moves to the cloud and the network becomes part of the robot. A robot that calls its policy over the network has one hard number: the time its current chunk of actions takes to play. We measured how much the network takes, how distance changes it, and which fixes actually moved it.

Why the network is part of a consumer robot

Consumer robots are priced like appliances. The parts for a Hugging Face SO-101 arm come to about $1301. Innate's MARS, a home robot with an arm, lidar and cameras, sells for under $2,000 and carries a Jetson Orin Nano 8GB as its computer2. Even 1X's NEO humanoid is $20,000, or $499 a month3.

Edge compute has moved the other way. In July 2026 NVIDIA raised Jetson prices by up to 101%: the Orin Nano Super developer kit went from $249 to $399, the AGX Thor developer kit from $3,499 to $5,499, and the Thor class T5000 module to $4,999 in quantities of a thousand4. Running a multi billion parameter policy on the robot puts you at the top of that line, so the computer that could hold a large model costs more than many of the robots that would carry it. The alternative is to keep the robot's own computer small and run the model on a shared GPU in the cloud. Then the network stops being a detail and becomes part of the robot.

How large a part depends on how fast the robot has to react. A robot that only has to fold a towel can ask for its next chunk with plenty of actions still in hand. One that has to catch a slipping object, follow a moving person or correct a grasp mid motion runs a faster loop, asks more often and has less left when it does. The faster the reflex a robot needs, the larger the share of its budget the network takes.

0 ms50 ms100 ms150 ms200 ms250 ms15 Hz, ask 4 steps early11% of 267 ms20 Hz, ask 2 steps early30% of 100 ms30 Hz, ask 2 steps early45% of 67 ms50 Hz, ask 2 steps early75% of 40 msa 30 ms round triptime left in the chunk when the robot asks
Figure 1. The same 30 ms round trip against the slack a robot has when it asks for its next chunk, (L − r) / hz. At 15 Hz asking four steps early it is a tenth of the budget. A robot built to react faster asks more often with less left, and at 50 Hz the network alone eats most of the time a model would need. The control rates are illustrative; the arithmetic is the point.

The budget

A policy like π0.5 answers with a chunk of actions, and the robot asks for the next chunk while it plays the current one5,6. For a DROID setup at 15 Hz with 8 actions per chunk, the next answer has to arrive within about 533 ms or the arm runs out of actions and stops. Every millisecond the network takes is a millisecond the model cannot have. The numbers below come from more than 170 of our own measurement records between September and November 2026, some by a tester using the platform as an ordinary user and some by our own engineers. Each is a median unless it says otherwise.

0 ms400 ms800 ms1200 msπ0.5 on an L4, client in the same region221.7 msπ0.5 on an L4, real xArm6 arm592 ms7B model on an A100, real SO-1011276 msπ0.5 chunk: 533 msits budget: 700 msinference on the GPUnetwork and transport
Figure 2. Median time from a robot sending its observation to holding its next chunk, split into the model's inference and everything around it: the network, sending frames, reading them. On a nearby client the network is a sliver. On real arms and longer paths it grows into a real share of the budget.

On a nearby client, π0.5 on an L4 answered in 222 ms, and the network round trip was under 10 ms of that. A real xArm6 spent 29 ms on the round trip plus another 40 ms or so reading its camera and sending frames. A 7B model driving an SO-101 from the East Coast spent 98 ms on the round trip and 158 ms outside its worker in all. None of these is the biggest cost on its own, but on a tight budget each one decides whether the arm keeps moving.

Distance

The fastest way to lose the budget is to put the GPU far from the robot. Cloud robotics systems have long offloaded computation to whatever server is available7, but a robot's control loop pays for every millisecond of the round trip, every time it asks.

0 ms200 ms400 ms600 ms800 ms283.7p50355.1p95worker nearby537p50687.7p95worker far awaychunk budget 533 ms
Figure 3. The same robot client and the same simulated π0.5 (a fixed 230 ms of compute), with its worker near or far away. Time spent outside the worker went from 94 ms to 411 ms, and the far worker missed the chunk budget at the median.

Every millisecond of that difference comes from distance alone: the client, the model and the code were the same in both runs.

Regional routing

Since distance is the cost that grows fastest, the platform decides where a robot's worker runs from where the robot is. When a robot connects, we locate it by its public address and rank our regions by their distance to it. Each region is one metro area, roughly 200 km across, grouping the data centers we can start a GPU in there. The robot's worker starts in the nearest region with a card free, and in the next nearest when that one is full. Nobody picks a region by hand, and a robot never needs to know which data center it is talking to.

before: a worker across the country, about 290 ms awayrobot, West Coast24 to 28 msrobot, East Coast56 msregion, one metro arearobot and its round trip to the nearest region
Figure 4. Our seven regions across the United States, each one metro area, and two test robots routed to the nearest. The round trips are probes with no model behind them, from a cloud machine on the West Coast and a machine on the East Coast. The dashed line is the kind of placement robots got before routing, a worker on the far side of the country.

The effect is large. In the experiment behind Figure 3, the same robot client with its worker moved from a far region to a near one spent 87 ms outside the worker instead of 331 ms, a fourfold drop, and its median end to end time fell from 537 ms to 284 ms, from over the chunk budget to comfortably inside it. Robots that used to land on a worker about 290 ms away now reach one in a few tens of milliseconds: 24 to 28 ms on the West Coast and 56 ms from the East Coast in our probes. No other change in this post moved the median as much except replacing the relay.

Distance can erase a faster GPU

A faster card only helps if the network does not give the time back. On an A100 nearby, π0.5 answered in 121 ms end to end. An A100 on the other coast answered in 310 ms, with 191 ms of it on the network, slower than an L4 next door.

0 ms100 ms200 ms300 msL4, nearby221.7 msA100 80GB, nearby121.4 msA100 40GB, other coast310.3 msinferenceeverything else
Figure 5. π0.5 for one robot, median end to end. A faster card nearly halves inference, and a worker on the other side of the country gives all of it back to the network.

The plumbing

The rest of the wins came from ordinary networking work, none of which touches the model. Each one below was measured before and after on its own.

One write per reply. Our server used to send each reply's header and body as two separate writes. Sending them in a single write halved a ping from a laptop to its worker, from 201.9 ms to 99.9 ms. A later fix in a layer we had missed took a robot's median end to end time from 334.8 ms to 279.3 ms, with the network part falling from 118.7 ms to 67.8 ms.

A tunnel instead of a relay. Robots first reached workers through a relay that forwarded raw connections. Replacing it with an encrypted tunnel brought the median from 275.3 ms to 60.0 ms, the p99 from 3,646.9 ms to 139.6 ms, and the first act on a connection from 2,146 ms to 132 ms. This was the largest single change we made.

Connections opened before they are needed. A brand new connection starts slowly, so the first large request on it pays for it. Opening the next connection in the background and warming it with a megabyte of traffic took the first 256 KB act on it from 1,046 ms to at most 304 ms.

Fewer bytes from the robot. Resizing camera images to the 224 by 224 the model actually reads, on the robot before sending, shrank each request from 105,168 bytes to 22,835 and the median from 104.1 ms to 86.9 ms.

One stream for observe and act. Moving observations and actions onto one long lived stream barely changed the median: in a 30 minute run it went from 194.6 ms to 208.2 ms, while the p99 fell from 418.4 ms to 339.4 ms. Against plain requests on the same lines, the medians matched within a few milliseconds, but the first act on a fresh stream arrived in 336 to 340 ms instead of 685 to 797 ms.

Benchmarks next to the policy. When a simulation benchmark scores a policy, the two used to sit on different machines with a 249 ms ping between them. Placing them on the same machine brought that to 13 to 15 ms.

10 ms30 ms100 ms300 ms1000 ms3000 msReplies written in one piece (ping)Encrypted tunnel instead of a relay (p99)Connection opened and warmed aheadImages resized on the robot (upload)One stream for observe and act (p99)Benchmark placed next to the policy (ping)beforeafter
Figure 6. Before and after for the network fixes we measured both ways, on a log scale. Each row is its own measurement with its own percentile and conditions, so the rows do not add up.

A quiet link was a quieter problem than it looked. With an act every half second on a 21 ms line, 52 acts in a run paid for an extra round trip after the connection had gone idle; once the link was kept from restarting its slow start, one did.

Not every fix was a latency fix. Sending binary frames instead of base64 text cut upload size by about a quarter, but on a 27 ms link the end to end time moved from 254.7 ms to 253.6 ms. Fewer bytes matter on slow links and barely at all on fast ones.

References

  1. Hackster.io. Hugging Face Launches the SO-101, an Upgraded Low-Cost 3D-Printable Autonomous Robot Arm. 2025.
  2. Innate. Show HN: MARS, Personal AI robot for builders (< $2k). 2025.
  3. Fast Company. 1X Technologies home robot NEO. 2025.
  4. CNX Software. NVIDIA increases the price of Jetson modules and devkits by up to 101%. July 2026.
  5. K. Black, M. Y. Galliker, S. Levine. Real-Time Execution of Action Chunking Flow Policies. 2025.
  6. F. Capuano, S. Palma, M. Aractingi, et al. Asynchronous Robot Inference: Decoupling Action Prediction and Execution. Hugging Face, 2025.
  7. J. Ichnowski, K. Chen, K. Dharmarajan, et al. FogROS2: An Adaptive Platform for Cloud and Fog Robotics Using ROS 2. ICRA 2023.