Train a robot in simulation

Saved run results

Dependency cache ready: /mnt/workspace/hands-on-modern-rl-notebooks/hands-on-modern-rl-experiment08-maniskill-f222f245edea.ready
Repository: /runs/maniskill-en-release/workspaces/maniskill
Experiment runtime: /runs/maniskill-en-release/workspaces/maniskill/modelscope-space/hands-on-modern-rl-experiment08-maniskill

Available tasks:
  push-cube            PushCube · Panda · PushCube-v1
  pick-cube            PickCube · Panda · PickCube-v1
  stack-cube           StackCube · Panda · StackCube-v1
  peg-insertion        PegInsertionSide · Panda · PegInsertionSide-v1

Selected: PushCube · Panda
Algorithm: PPO
Epoch schedule: 2 epochs × 5,000 steps = 10,000 total steps
Every epoch is evaluated and saved as a separately selectable model.

Device: CPU
Starting the same training generator used by the live Studio...

Creating 1 parallel PushCube-v1 simulations on cpu
PPO update · step=5,000
parallel_envs=1  rollout=50  policy_device=cpu  simulation=CPU PhysX fallback
policy_loss=-0.00138733  value_loss=0.00185259
EVAL mean_dense_return=21.164 std=0.000
SAVE epoch=1/2 model=policy.zip
PPO update · step=10,000
parallel_envs=1  rollout=50  policy_device=cpu  simulation=CPU PhysX fallback
policy_loss=-0.00639765  value_loss=0.00385106
EVAL mean_dense_return=1.616 std=0.000
SAVE epoch=2/2 model=policy.zip
REPLAY epoch=1/2 model=policy.zip · CPU Vulkan camera replay unavailable (RuntimeError); generated a camera-free GIF from the learned policy's real TCP, object, and goal poses
REPLAY epoch=2/2 model=policy.zip · CPU Vulkan camera replay unavailable (RuntimeError); generated a camera-free GIF from the learned policy's real TCP, object, and goal poses
Saved 2 independently selectable ManiSkill policies and replays

Final phase: complete
Final score: 1.6157939434051514
Saved 2 selectable epoch models in this run.
Model index: /runs/maniskill-en-release/workspaces/maniskill/modelscope-space/hands-on-modern-rl-experiment08-maniskill/artifacts/notebook-models.json

/opt/maniskill/lib/python3.10/site-packages/stable_baselines3/common/vec_env/base_vec_env.py:77: UserWarning: The `render_mode` attribute is not defined in your environment. It will be set to None.
  warnings.warn("The `render_mode` attribute is not defined in your environment. It will be set to None.")
/opt/maniskill/lib/python3.10/site-packages/stable_baselines3/common/vec_env/base_vec_env.py:77: UserWarning: The `render_mode` attribute is not defined in your environment. It will be set to None.
  warnings.warn("The `render_mode` attribute is not defined in your environment. It will be set to None.")
/opt/maniskill/lib/python3.10/site-packages/stable_baselines3/common/vec_env/base_vec_env.py:77: UserWarning: The `render_mode` attribute is not defined in your environment. It will be set to None.
  warnings.warn("The `render_mode` attribute is not defined in your environment. It will be set to None.")

Run the inference cell below to choose an epoch model and visualize its exact learned policy.

The newest epoch is selected by default. Choose another saved model, then click Visualize selected policy.

VBox(children=(Dropdown(description='Epoch model:', layout=Layout(width='95%'), options=(('Epoch 2/2 · 10,000 …

{
  "score": 1.6157939434051514,
  "steps": 10000,
  "epochs": 2,
  "checkpoints": 2,
  "preview": "learned-policy.gif"
}

{
  "score": 1.6157939434051514,
  "steps": 10000,
  "epochs": 2,
  "checkpoints": 2,
  "preview": "learned-policy.gif"
}