Teleoperation

Take over any robot. Hand it back.

A person drives a robot from a console of your own, then gives it back to its policy, which never left its card.

How it works

The robot’s loop stays as it is

Put teleop on a robot’s list and connect with that name, and a person can watch it, take it over and give it back. Its act() returns its policy’s chunk while the policy drives, the newest chunk a person sent while they drive, and its hold while it is stopped.

robot_user.py
# everything ending in _user is yours
policy_user, teleop_user = sequence_ai.connect(
    "my_policy_user", "teleop",
    api_key=ROBOT_KEY_user)
while True:
    policy_user.observe(read_obs_user())
    robot_user.apply(policy_user.act())
    # its policy's chunk, or a person's
console_user.py
op_user = sequence_ai.teleop(
    key=ROBOT_KEY_user,
    operator="alice_user")
# watch it, until you take it over
for frame_user in op_user.stream():
    if take_over_user(frame_user):
        break
op_user.take()      # drive it
op_user.send(chunk_user)
op_user.release()   # back to its policy
op_user.stop()      # stop it, any time

Driving

A person drives. The policy waits on its card.

One person drives a robot at a time, from a console that holds that robot’s own key. Handed back, the robot’s next act goes to its policy at once, and every takeover is on the record.

Hand back

Back to its policy, with no cold start

While a person drives, the robot keeps its seat and its worker stays up, its card held and billed as when its policy drives. Released, the robot’s next act goes to that worker at once, with no start and no wait in line.

  • One driver at a time. While a person drives a robot, another console’s take() is refused and told who drives it.
  • Stop comes first. stop() goes ahead of everything else on the robot’s connection, from a console that drives it or only watches.
  • A quiet console stops it. A console that drives the robot and goes quiet for 3 seconds stops it.
shell
$seq policy status my_policy_user
# each robot: driven by its model,
# driven by alice_user, or stopped

Records

Every takeover, kept

Every take, release, stop and timeout goes into your account’s audit trail: which robot, when, and who — the name the console gave and the robot key it came with.

  • Demonstrations as data. With a robot’s Record on, seq robot record writes its steps to a file of yours, every chunk a person sent among them, and each step says who drove it.
  • Yours alone. Your recorder reads them from the robot’s worker, with every take, release, stop and timeout. The platform stores none of them.
shell
$seq robot set arm_1_user \
    record=on
$seq robot record arm_1_user
$seq audit

Questions

Before you take a robot over

What teams ask first. The docs answer the rest.

Read the teleoperation guide

Does my robot’s code change?

One more name: connect it with "teleop" among its names. Its loop stays as it is, and policy.driver says who drives it now: "model", "human" or "stopped".

Which key does the console use?

The robot’s own. The key names its robot, so the console needs no robot name, and replacing it turns away the robot and its operator together.

What does the robot do between a person’s chunks?

It plays the newest chunk a person sent, once: one a newer chunk replaced before the robot’s next act() is never played. With none newer, the robot holds where it is and stays connected, so a console sends a chunk at least as often as the robot’s loop calls act().

Does a console need a machine of its own?

No. A console holds no machine: nothing is started or held for it. While a person drives, the robot’s worker is billed as when its policy drives.

Take over your first robot today

One argument on the robot, and a console of your own.