Template updates

MolmoAct2 DROID

molmoact2-droid · policy template by Ai2 · Franka Panda · 15 Hz · L4. Each version below is fixed once published, newest first.

seq init molmoact2-droid gives the stable version, seq init molmoact2-droid@beta the beta, and seq init molmoact2-droid@v<N> any version below that is not withdrawn. A deployment made from this template names the version it runs in seq policy versions and on its page in the console; seq policy upgrade moves it to the newer stable one.

Performance

No runs yet.

Versions

  1. v4stablebeta

    MolmoAct2 for DROID as v3; policy.py says, as the README does, how a worker's CPU and memory are billed.

    Needs sequence-ai 0.56.0 or later to deploy it (pip install -U sequence-ai); its client runs on 0.56.0 or later with its video encoder (pip install -U "sequence-ai[observe]").

    • v3's files; only policy.py's comment on CPU and memory changed: the machine its card sets (an L4's 8 cores and 32 GiB), or what it declares where that is more.
    Use v4
    molmoact2-droid v4shell
    # start a policy from this version, then deploy it
    # my_policy_user: the name you choose
    seq init molmoact2-droid@v4 my_policy_user
    seq deploy my_policy_user
    
    # or move a deployment made from this template to it
    seq policy upgrade my_policy_user --to molmoact2-droid@v4
    
    # on the robot's machine: this version's client.py and its README
    pip install -U sequence-ai
    seq init molmoact2-droid@v4 my_policy_user

    In the console, Deploy deploys this version: the console deploys the stable one.

  2. v3

    MolmoAct2 for DROID as v2; the README says a worker's CPU and memory are billed as the machine its card sets.

    Needs sequence-ai 0.56.0 or later to deploy it (pip install -U sequence-ai); its client runs on 0.56.0 or later with its video encoder (pip install -U "sequence-ai[observe]").

    • v2's files; only the README's billing of CPU and memory changed: the machine its card sets (an L4's 8 cores and 32 GiB), or what it declares where that is more.
    Use v3
    molmoact2-droid v3shell
    # start a policy from this version, then deploy it
    # my_policy_user: the name you choose
    seq init molmoact2-droid@v3 my_policy_user
    seq deploy my_policy_user
    
    # or move a deployment made from this template to it
    seq policy upgrade my_policy_user --to molmoact2-droid@v3
    
    # on the robot's machine: this version's client.py and its README
    pip install -U sequence-ai
    seq init molmoact2-droid@v3 my_policy_user
  3. v2

    MolmoAct2 for DROID as before; preprocess takes the cameras, joints and gripper by name, as other DROID templates do.

    Needs sequence-ai 0.56.0 or later to deploy it (pip install -U sequence-ai); its client runs on 0.56.0 or later with its video encoder (pip install -U "sequence-ai[observe]").

    • preprocess(wrist=..., exterior_1=..., exterior_2=..., joint_positions=..., gripper=..., instruction=...) replaces cameras= and state=; what reaches the policy is the same.
    • Pass each DROID camera as RGB, img[..., :3][..., ::-1]: its frames are BGRA.
    • Run client.py from Python 3.10 or later on the robot's laptop: DROID's own environment is Python 3.7.
    Use v2
    molmoact2-droid v2shell
    # start a policy from this version, then deploy it
    # my_policy_user: the name you choose
    seq init molmoact2-droid@v2 my_policy_user
    seq deploy my_policy_user
    
    # or move a deployment made from this template to it
    seq policy upgrade my_policy_user --to molmoact2-droid@v2
    
    # on the robot's machine: this version's client.py and its README
    pip install -U sequence-ai
    seq init molmoact2-droid@v2 my_policy_user
  4. v1

    Ai2's MolmoAct2 for a Franka Panda with DROID's cameras, on an L4.

    Needs sequence-ai 0.56.0 or later to deploy it (pip install -U sequence-ai); its client runs on 0.56.0 or later with its video encoder (pip install -U "sequence-ai[observe]").

    Use v1
    molmoact2-droid v1shell
    # start a policy from this version, then deploy it
    # my_policy_user: the name you choose
    seq init molmoact2-droid@v1 my_policy_user
    seq deploy my_policy_user
    
    # or move a deployment made from this template to it
    seq policy upgrade my_policy_user --to molmoact2-droid@v1
    
    # on the robot's machine: this version's client.py and its README
    pip install -U sequence-ai
    seq init molmoact2-droid@v1 my_policy_user