Template updates
MolmoAct2 DROID
molmoact2-droid · policy template by Ai2 · Franka Panda · 15 Hz · L4. Each version below is fixed once published, newest first.
seq init molmoact2-droid gives the stable version, seq init molmoact2-droid@beta the beta, and seq init molmoact2-droid@v<N> any version below that is not withdrawn. A deployment made from this template names the version it runs in seq policy versions and on its page in the console; seq policy upgrade moves it to the newer stable one.
Performance
No runs yet.
Versions
-
MolmoAct2 for DROID as v3;
policy.pysays, as the README does, how a worker's CPU and memory are billed.Needs sequence-ai 0.56.0 or later to deploy it (
pip install -U sequence-ai); its client runs on 0.56.0 or later with its video encoder (pip install -U "sequence-ai[observe]").- v3's files; only
policy.py's comment on CPU and memory changed: the machine its card sets (an L4's 8 cores and 32 GiB), or what it declares where that is more.
Use v4
# start a policy from this version, then deploy it # my_policy_user: the name you choose seq init molmoact2-droid@v4 my_policy_user seq deploy my_policy_user # or move a deployment made from this template to it seq policy upgrade my_policy_user --to molmoact2-droid@v4 # on the robot's machine: this version's client.py and its README pip install -U sequence-ai seq init molmoact2-droid@v4 my_policy_user
In the console, Deploy deploys this version: the console deploys the stable one.
- v3's files; only
-
MolmoAct2 for DROID as v2; the README says a worker's CPU and memory are billed as the machine its card sets.
Needs sequence-ai 0.56.0 or later to deploy it (
pip install -U sequence-ai); its client runs on 0.56.0 or later with its video encoder (pip install -U "sequence-ai[observe]").- v2's files; only the README's billing of CPU and memory changed: the machine its card sets (an L4's 8 cores and 32 GiB), or what it declares where that is more.
Use v3
# start a policy from this version, then deploy it # my_policy_user: the name you choose seq init molmoact2-droid@v3 my_policy_user seq deploy my_policy_user # or move a deployment made from this template to it seq policy upgrade my_policy_user --to molmoact2-droid@v3 # on the robot's machine: this version's client.py and its README pip install -U sequence-ai seq init molmoact2-droid@v3 my_policy_user
-
MolmoAct2 for DROID as before;
preprocesstakes the cameras, joints and gripper by name, as other DROID templates do.Needs sequence-ai 0.56.0 or later to deploy it (
pip install -U sequence-ai); its client runs on 0.56.0 or later with its video encoder (pip install -U "sequence-ai[observe]").preprocess(wrist=..., exterior_1=..., exterior_2=..., joint_positions=..., gripper=..., instruction=...)replacescameras=andstate=; what reaches the policy is the same.- Pass each DROID camera as RGB,
img[..., :3][..., ::-1]: its frames are BGRA. - Run
client.pyfrom Python 3.10 or later on the robot's laptop: DROID's own environment is Python 3.7.
Use v2
# start a policy from this version, then deploy it # my_policy_user: the name you choose seq init molmoact2-droid@v2 my_policy_user seq deploy my_policy_user # or move a deployment made from this template to it seq policy upgrade my_policy_user --to molmoact2-droid@v2 # on the robot's machine: this version's client.py and its README pip install -U sequence-ai seq init molmoact2-droid@v2 my_policy_user
-
Ai2's MolmoAct2 for a Franka Panda with DROID's cameras, on an L4.
Needs sequence-ai 0.56.0 or later to deploy it (
pip install -U sequence-ai); its client runs on 0.56.0 or later with its video encoder (pip install -U "sequence-ai[observe]").Use v1
# start a policy from this version, then deploy it # my_policy_user: the name you choose seq init molmoact2-droid@v1 my_policy_user seq deploy my_policy_user # or move a deployment made from this template to it seq policy upgrade my_policy_user --to molmoact2-droid@v1 # on the robot's machine: this version's client.py and its README pip install -U sequence-ai seq init molmoact2-droid@v1 my_policy_user