GPU Routing

The nearest GPU. For every robot.

On-demand GPUs in seven US regions. Each robot runs in the one nearest it; a GPU you add, in the one with the shortest round trip we measure from the place you name. Billed by the second.

How it works

Nearest the robot, or your place

A robot connected to a policy runs in the region nearest the public address it connects from. A GPU you add starts near the place you name: the places with the shortest round trip we measure from near it are asked up to three at once, those that answer on the spot one at a time, and the first to have the card wins.

robot_user.py
# everything ending in _user is yours
policy_user = sequence_ai.connect(
    "my_policy_user", api_key=ROBOT_KEY_user)
while True:
    policy_user.observe(read_obs_user())
    robot_user.apply(policy_user.act())
    # straight to its worker, over TLS
shell
# a GPU of your own, near a place
$seq gpu add --place "Denver, CO" --gpu L4
$seq gpu ssh gpu_id_user

Routing

Seven regions, all in the US

Seven regions across the US, each a metro. A card costs the same in every one.

On its worker

Placed once, then direct

The gateway places a robot as it connects. From then on every observation and every action goes between the robot and its worker directly, over TLS, never through the gateway.

  • The first card wins. The places nearest a robot are asked at once, and the first to give a card wins, unless a nearer one gives one within 10 seconds of it.
  • Kept for its episode. A robot that moves plays the episode it is in where it is, and moves at its next one to the region nearest it then.
  • Billed by the second. From when its machine has its GPU; the wait for a card is not billed.

Measured

The round trip to the nearest region

From most of the US, the network’s round trip to the nearest region is about 30 ms or less, and within 50 ms from almost everywhere.

Measured from 76 places in the US, the network’s round trip alone: a policy’s own time on its GPU comes on top.

See every place on a map

Questions

Before you route a robot

What teams ask first. The docs answer the rest.

Read how a robot is routed

How is a region picked?

A robot’s, by the public address its connect comes from, read again at every connect; behind a VPN or a proxy, that is where the network meets the internet. A GPU’s, by the place you name with --place, a US city or a latitude and longitude in the US: the regions with the shortest round trip we measure from near it come first, and by distance where nothing near it was measured.

Does my robot’s code change?

No. It connects to its policy as before, and the routing happens as it connects.

Can I have a GPU of my own?

Yes. seq gpu add --place starts one near a US city or a latitude and longitude in the US, the places with the shortest round trip from there asked up to three at once, those that answer on the spot one at a time. It is yours as root in a container, reached with seq gpu ssh, and billed by the second, for at least 60 seconds each time one starts.

Do the figures include inference?

No. They are the network’s round trip from each place to the region nearest it. What a policy takes on its GPU comes on top.

Put your robots on their nearest GPU

One connect from the robot, or one command for a GPU of your own.