在仿真中训练机器人

本次运行结果

Dependency cache ready: /mnt/workspace/hands-on-modern-rl-notebooks/hands-on-modern-rl-experiment08-maniskill-f222f245edea.ready
代码仓库： /runs/maniskill-zh-release/workspaces/maniskill
实验运行目录： /runs/maniskill-zh-release/workspaces/maniskill/modelscope-space/hands-on-modern-rl-experiment08-maniskill

可选任务：
  push-cube            PushCube · 熊猫机械臂 · PushCube-v1
  pick-cube            PickCube · 熊猫机械臂 · PickCube-v1
  stack-cube           StackCube · 熊猫机械臂 · StackCube-v1
  peg-insertion        PegInsertionSide · 熊猫机械臂 · PegInsertionSide-v1

所选任务： PushCube · 熊猫机械臂
算法： PPO
轮次安排：2 轮 × 5,000 步 = 10,000 总步数
每轮都会评估并保存一个可单独选择的模型。

Device: CPU
启动与在线 Studio 相同的训练流程…

Creating 1 parallel PushCube-v1 simulations on cpu
PPO update · step=5,000
parallel_envs=1  rollout=50  policy_device=cpu  simulation=CPU PhysX fallback
policy_loss=-0.00138733  value_loss=0.00185259
EVAL mean_dense_return=21.164 std=0.000
SAVE epoch=1/2 model=policy.zip
PPO update · step=10,000
parallel_envs=1  rollout=50  policy_device=cpu  simulation=CPU PhysX fallback
policy_loss=-0.00639765  value_loss=0.00385106
EVAL mean_dense_return=1.616 std=0.000
SAVE epoch=2/2 model=policy.zip
REPLAY epoch=1/2 model=policy.zip · CPU Vulkan camera replay unavailable (RuntimeError); generated a camera-free GIF from the learned policy's real TCP, object, and goal poses
REPLAY epoch=2/2 model=policy.zip · CPU Vulkan camera replay unavailable (RuntimeError); generated a camera-free GIF from the learned policy's real TCP, object, and goal poses
Saved 2 independently selectable ManiSkill policies and replays

最终阶段： complete
最终得分： 1.6157939434051514
已保存 2 个本次运行可选择的轮次模型。
模型索引： /runs/maniskill-zh-release/workspaces/maniskill/modelscope-space/hands-on-modern-rl-experiment08-maniskill/artifacts/notebook-models.json

/opt/maniskill/lib/python3.10/site-packages/stable_baselines3/common/vec_env/base_vec_env.py:77: UserWarning: The `render_mode` attribute is not defined in your environment. It will be set to None.
  warnings.warn("The `render_mode` attribute is not defined in your environment. It will be set to None.")
/opt/maniskill/lib/python3.10/site-packages/stable_baselines3/common/vec_env/base_vec_env.py:77: UserWarning: The `render_mode` attribute is not defined in your environment. It will be set to None.
  warnings.warn("The `render_mode` attribute is not defined in your environment. It will be set to None.")
/opt/maniskill/lib/python3.10/site-packages/stable_baselines3/common/vec_env/base_vec_env.py:77: UserWarning: The `render_mode` attribute is not defined in your environment. It will be set to None.
  warnings.warn("The `render_mode` attribute is not defined in your environment. It will be set to None.")

运行下面的推理单元格，选择轮次模型并查看该策略的回放。

默认选择最新轮次；也可以选择其他已保存模型，然后点击 回放所选策略.

VBox(children=(Dropdown(description='训练轮次模型：', layout=Layout(width='95%'), options=(('轮次 2/2 · 10,000 步 · 得分 1…

{
  "score": 1.6157939434051514,
  "steps": 10000,
  "epochs": 2,
  "checkpoints": 2,
  "preview": "learned-policy.gif"
}

{
  "score": 1.6157939434051514,
  "steps": 10000,
  "epochs": 2,
  "checkpoints": 2,
  "preview": "learned-policy.gif"
}