sequence-ai, the package these docs are about, is open source under the Apache-2.0 license: the
client your robot connects with, the seq command, and the components that make a robot policy fast to
reach from its robot over the internet. The components run on a GPU of your own, for free, and the same package
rents a GPU of ours near your robot or deploys your policy to ours. Its source, its own docs and its issues are on
GitHub: github.com/General-Sequences/sequence-ai.
# the robot's client, the seq command and every component, # and uvicorn, numpy and zstandard for a server of your own pip install sequence-ai
The client and the components run on Python 3.9 or later, on Linux and macOS, and the seq command on
3.10 or later. cuda_graphs uses the torch your model already has, 2.4 or later. Each component is one
file, and most import only the standard library.
Three ways to run it
| Whose GPU | What you run on it | On the robot | |
|---|---|---|---|
| Your own server | yours, or any GPU your robot can reach | your policy server, with the components you add | connect("http://your-gpu:8000") |
| A GPU near your robot | one of ours, by city (Add GPUs) | the same server | connect("http://<its address>:8000") |
| A hosted deployment | ours | nothing: we build your policy, serve it and score it (Deploy) | connect("my_policy_user") |
The first needs no account and costs nothing. The other two are billed while a GPU runs
(Pricing), and their commands, seq gpu add and seq deploy, are in
the same package.
On the robot
Given an address, connect() goes straight to a policy server of yours, with the components of the
robot’s side on its connection — BBR, TCP keepalive, a ping while it is idle, and msgpack — and
nothing of ours in between: no key, no robot, nothing billed. It answers with the same two calls as a policy of
ours, observe() and act() (observe() and act()).
import sequence_ai # everything ending in _user is yours with sequence_ai.connect("http://your-gpu:8000") as policy_user: policy_user.observe({"image": frame_user, "state": state_user}) actions_user = policy_user.act()["actions"]
Why BBR, and a ping while the connection is idle where the server answers it, shorten an upload after a pause: Round trips. The connection’s other components, each one file, on the robot’s side, on the server’s and on both, with adapters for openpi and LeRobot, are listed in the repository: What does each network component do?
A robot that already runs openpi’s own client against openpi’s own server on a GPU of yours keeps
both, with one changed line on each side: the server, started with python -m sequence_ai.network.openpi
scripts/serve_policy.py, pings each robot every 100 ms and decodes its JPEG pictures, and the robot sends
them as JPEG with image_tools imported from sequence_ai.network.openpi in place of
openpi_client’s (How do I run π0 (openpi) on a remote GPU?).
With openpi’s own server, which answers no ping while it runs inference, a Linux robot without BBR can
still have its window halved once before each upload; BBR avoids that
(sequence-ai’s measurements).
On the GPU
The components of the GPU’s side wrap the model your server already has:
from sequence_ai.engine import cuda_graphs from sequence_ai.engine.batcher import Batcher # replayed from CUDA graphs, kept only if its outputs match cuda_graphs.install(model_user, ["action_head.denoiser"]) # several robots' requests in one forward pass batcher_user = Batcher(infer_a_batch_user, sizes=[1, 2, 4, 8]) # in each request's handler action_user = await batcher_user(observation_user)
cuda_graphsreplays a module your model calls many times a run from CUDA graphs captured from it, and checks each shape against eager mode byte for byte: a shape that differs runs in eager mode.batcherruns the calls waiting for a model as one call, in the batch sizes the model lists.replicasruns replicas of a model, each in its own process on the GPU given to it, and replaces one that dies.slacksays how many robots a replica can serve, from the time an action chunk leaves each robot and each robot’s own round trip, and how long an open batch may wait.
With openpi, LeRobot or GR00T: How do I use the inference components with openpi, LeRobot or GR00T?
Its source, docs and issues
- The code, its tests and examples, and a page for each component: the repository and its docs.
- A pull request can change the components,
sequence_ai.engineandsequence_ai.network, with their tests, examples and docs. The rest of the package, the client of our GPUs and deployments, is published there as it ships, at each release: a change to it begins as an issue (Contributing). - To cite it: CITATION.cff.
What runs our GPUs and deployments, from sending a robot to a GPU to billing it, is not in the package.