Quickstart

Two endpoints share one key, one request shape and one response envelope. /v1/chat serves the vision-language models: messages in, a message out. /v1/act serves the action policies: an observation in, an action chunk out. Both answer with the same id, created, model, usage, perf and warnings — only the payload differs.

pip install requests
export SEQUENCES_API_KEY="seq_..."

Authentication

A bearer token on every request. Same header on both endpoints — the point of one key is that nothing about the second brain requires a second credential.

Authorization: Bearer $SEQUENCES_API_KEY
Content-Type: application/json

Model names

Every model is addressed as accounts/<namespace>/models/<id>. First-party models live under accounts/sequences; a dedicated deployment of your own checkpoint lives under your account, and nothing else about the request changes.

The catalogue gives each model an endpoint field with two possible values. That is the only classification you need — architecture is there to explain the invoice, and you can ignore it.

Chat

Messages in, one message out. Nothing is injected: no system prompt of ours is prepended, no message is rewritten, no reply is parsed. What you send is what the model sees. Unknown fields are rejected rather than ignored — a request that silently drops half of what you sent is worse than one that fails.

import requests

r = requests.post(
    "https://api.generalsequences.com/v1/chat",
    headers={"Authorization": f"Bearer {KEY}"},
    json={
        "model": "accounts/sequences/models/claude-fable-5",
        "messages": [
            {"role": "system", "content": "Reply with one short instruction."},
            {"role": "user", "content": [
                {"type": "text", "text": "What should the arm do?"},
                {"type": "image_url",
                 "image_url": {"url": f"data:image/jpeg;base64,{frame}"}},
            ]},
        ],
    },
)
r.json()["message"]["content"]
r.json()["usage"]["prompt_tokens"]   # prompt/completion here; observation/action on /v1/act

Act

One request shape for every action policy, feed-forward and world-model alike. Send the frames you have, the joint state, and a language instruction; receive a chunk of actions with the semantics needed to execute them.

{
  "model": "accounts/sequences/models/pi05-droid",
  "observation": {
    "images": [
      {"view": "exterior_1", "data": "<base64>", "timestamp_ms": 1000},
      {"view": "wrist_left",  "data": "<base64>", "timestamp_ms": 1000}
    ],
    "proprioception": {
      "joint_positions": [0.1, -0.2, 0.3, 0.4, -0.1, 0.0, 0.5],
      "gripper": [0.8]
    },
    "instruction": "put the mug on the plate"
  },
  "action_horizon": null,            // optional override
  "return_world_prediction": false   // world models only
}

{
  "object": "act.completion",
  "action_chunk": {
    "steps": [{"index": 0, "values": [0.0072, ...]}, ...],
    "action_space": "joint_absolute",
    "action_dim": 32,
    "control_frequency_hz": 15.0,
    "covers_seconds": 1.0          // send the next request within this
  },
  "usage": {"frames_received": 2, "frames_used": 2, ...},
  "perf_metrics": {"ttfa_ms": 17.6, "inference_ms": 17.6},
  "world_prediction": null,
  "warnings": []
}

Models

Both brains in one list. Each entry carries endpoint (which URL to call), context_frames (how many frames this model reads), action_dim, action_horizon, control_hz, and the licence terms under which we are permitted to serve it.

Add ?include_blocked=true to also see models we hold in the catalogue but refuse to serve. Well-known checkpoints forbid paid hosting, and some share a name with a sibling that permits it — keeping them listed with their reason is how that knowledge survives the next reader.

Frames and windows

images is a flat list, and the same view may appear more than once. Repeat a view with increasing timestamps and it becomes that camera's video. That is the whole difference between the two families:

When you send too few

The earliest frame is repeated until the window is full, which matches how these models handle their own cold start — and the response says so:

["view 'exterior_1': padded 32 frame(s) to reach context of 33"]

Padding is never silent. A model quietly seeing something other than what you believe you sent is the hardest class of bug to find on a robot, so it is reported every time.

When you send too many

The most recent are kept and the rest dropped. You are still billed for what arrived — bandwidth and storage are real — so usage reports frames_received next to frames_used. A large gap means the catalogue's context_frames is worth a look.

Action chunks

A call returns a chunk, not a single action: action_horizon steps meant to be executed at control_frequency_hz. Their product is covers_seconds, and it is your scheduling deadline — issue the next request before it elapses or the controller starves.

A longer chunk survives a worse network and costs less per second of motion; a shorter one reacts sooner. That trade is yours to make with action_horizon, and the default is whatever the checkpoint was trained with, because departing from it costs success rate.

action_space is returned on every chunk and must be honoured. The same floats mean opposite things under joint_absolute and joint_delta, and executing one as the other will drive the arm into its limits.

Errors

A 403 is a statement about the model, not about you — retrying will never change it. The reason is always included, because a refusal without one reads like a bug and gets retried in a loop.