在迷宫里发现奖励

本次运行结果

Dependency cache ready: /Users/sanbu/Documents/Codex/2026-10-20/revie/work/chapter-lab-runs/minigrid-zh-final/workspaces/minigrid/.cache/online-experiments/hands-on-modern-rl-experiment06-minigrid-adventure-849307d84044.ready
代码仓库： /Users/sanbu/Documents/Codex/2026-10-20/revie/work/chapter-lab-runs/minigrid-zh-final/workspaces/minigrid
实验运行目录： /Users/sanbu/Documents/Codex/2026-10-20/revie/work/chapter-lab-runs/minigrid-zh-final/workspaces/minigrid/modelscope-space/hands-on-modern-rl-experiment06-minigrid-adventure

可选任务：
  empty                Reach the Goal · MiniGrid-Empty-6x6-v0
  doorkey              Door & Key · MiniGrid-DoorKey-6x6-v0
  multiroom            Multi-Room · MiniGrid-MultiRoom-N2-S4-v0
  unlock               Unlock the Door · MiniGrid-Unlock-v0
  obstructed           Obstructed Maze · MiniGrid-ObstructedMaze-1Q-v1
  dynamic              Dynamic Obstacles · MiniGrid-Dynamic-Obstacles-6x6-v0

所选任务： Reach the Goal
算法： PPO
轮次安排：2 轮 × 1,000 步 = 2,000 总步数
每轮都会评估并保存一个可单独选择的模型。

Device: CPU

启动与在线 Studio 相同的训练流程…

Initialized PPO with CnnPolicy on CPU

update · step 1,000
------------------------------------------------
| train/loss                 |     -0.00101993 |
| train/value_loss           |       0.0474573 |
| train/policy_gradient_loss |    -0.000102817 |
| train/entropy_loss         |        -1.94193 |
| train/learning_rate        |          0.0003 |
| train/n_updates            |               4 |
------------------------------------------------
EVAL step=1,000 mean_reward=0.00 std=0.00 mean_length=144.0
SAVE epoch=1/2 model=policy.zip replay=empty-learned-policy.gif

update · step 2,000
------------------------------------------------
| train/loss                 |     -0.00374641 |
| train/value_loss           |      0.00900346 |
| train/policy_gradient_loss |     -0.00745966 |
| train/entropy_loss         |        -1.91692 |
| train/learning_rate        |          0.0003 |
| train/n_updates            |              12 |
------------------------------------------------
EVAL step=2,000 mean_reward=0.00 std=0.00 mean_length=144.0
SAVE epoch=2/2 model=policy.zip replay=empty-learned-policy.gif

Saved 2 independently selectable epoch policies and learned-policy replays

最终阶段： complete
最终得分： 0.0
已保存 2 个本次运行可选择的轮次模型。
模型索引： /Users/sanbu/Documents/Codex/2026-10-20/revie/work/chapter-lab-runs/minigrid-zh-final/workspaces/minigrid/modelscope-space/hands-on-modern-rl-experiment06-minigrid-adventure/artifacts/notebook-models.json

运行下面的推理单元格，选择轮次模型并查看该策略的回放。

VBox(children=(Dropdown(description='训练轮次模型：', layout=Layout(width='95%'), options=(('轮次 2/2 · 2,000 步 · 得分 0.…

默认选择最新轮次；也可以选择其他已保存模型，然后点击 回放所选策略.

{
  "score": 0.0,
  "steps": 2000,
  "epochs": 2,
  "checkpoints": 2,
  "preview": "empty-learned-policy.gif"
}

{
  "score": 0.0,
  "steps": 2000,
  "epochs": 2,
  "checkpoints": 2,
  "preview": "empty-learned-policy.gif"
}