让像素变成动作

本次运行结果

Dependency cache ready: /Users/sanbu/Documents/Codex/2026-10-20/revie/work/chapter-lab-runs/atari-zh-final/workspaces/atari/.cache/online-experiments/hands-on-modern-rl-experiment03-atari-a4edb25287ec.ready
代码仓库： /Users/sanbu/Documents/Codex/2026-10-20/revie/work/chapter-lab-runs/atari-zh-final/workspaces/atari
实验运行目录： /Users/sanbu/Documents/Codex/2026-10-20/revie/work/chapter-lab-runs/atari-zh-final/workspaces/atari/modelscope-space/hands-on-modern-rl-experiment03-atari

可选任务：
  pong                 Pong · ALE/Pong-v5
  breakout             Breakout · ALE/Breakout-v5
  space-invaders       Space Invaders · ALE/SpaceInvaders-v5
  freeway              Freeway · ALE/Freeway-v5
  seaquest             Seaquest · ALE/Seaquest-v5
  qbert                Q*bert · ALE/Qbert-v5
  beam-rider           Beam Rider · ALE/BeamRider-v5
  enduro               Enduro · ALE/Enduro-v5

所选任务： Freeway
算法： DQN
轮次安排：2 轮 × 1,000 步 = 2,000 总步数
每轮都会评估并保存一个可单独选择的模型。

Device: CPU
启动与在线 Studio 相同的训练流程…

A.L.E: Arcade Learning Environment (version 0.12.1+8a8fafb)
[Powered by Stella]

Initialized DQN with CnnPolicy on CPU
BASELINE replay_buffer=10,000 warmup=200 batch=32 target_update=250 exploration=1.00→0.01 train_reward=clipped eval_reward=raw schedule_horizon=2,000
SMOKE_TEST evaluation=2 epochs × 1 episode
TRAINING_PLAN epochs=2 steps_per_epoch≈1,000 save_policy=every_epoch

PROGRESS step=1,000/2,000 phase=gradient updates fps=111.3 eta=0.1min
EVAL starting epoch 1/2 at step=1,000

update · step 1,000
----------------------------------------------
| rollout/exploration_rate |            0.01 |
| train/loss               |     1.42251e-08 |
| train/learning_rate      |          0.0001 |
| train/n_updates          |             199 |
----------------------------------------------
EVAL step=1,000 mean_reward=0.00 std=0.00 mean_length=2047.0
EPOCH 1/2 complete
CHECKPOINT saved epoch model: freeway-model-r1792678519106672000-step-000001000.zip
CHECKPOINT this is the best policy so far and will be used for the final replay

PROGRESS step=2,000/2,000 phase=gradient updates fps=89.3 eta=0.0min
EVAL starting epoch 2/2 at step=2,000

update · step 2,000
----------------------------------------------
| rollout/exploration_rate |            0.01 |
| train/loss               |     3.70372e-08 |
| train/learning_rate      |          0.0001 |
| train/n_updates          |             448 |
----------------------------------------------
EVAL step=2,000 mean_reward=0.00 std=0.00 mean_length=2045.0
EPOCH 2/2 complete
CHECKPOINT saved epoch model: freeway-model-r1792678519106672000-step-000002000.zip

Restored best epoch model from step 1,000 with mean reward 0.00
Saved 2 selectable epoch policy models for run 1792678519106672000
Generated the best epoch replay: freeway-model-r1792678519106672000-step-000001000-preview.gif

最终阶段： complete
最终得分： 0.0
已保存 2 个本次运行可选择的轮次模型。
模型索引： /Users/sanbu/Documents/Codex/2026-10-20/revie/work/chapter-lab-runs/atari-zh-final/workspaces/atari/modelscope-space/hands-on-modern-rl-experiment03-atari/artifacts/notebook-models.json

运行下面的推理单元格，选择轮次模型并查看该策略的回放。

VBox(children=(Dropdown(description='训练轮次模型：', layout=Layout(width='95%'), options=(('轮次 2/2 · 2,000 步 · 得分 0.…

默认选择最新轮次；也可以选择其他已保存模型，然后点击 回放所选策略.

{
  "score": 0.0,
  "steps": 2000,
  "epochs": 2,
  "checkpoints": 2,
  "preview": "freeway-model-r1792678519106672000-step-000002000-preview.gif"
}

{
  "score": 0.0,
  "steps": 2000,
  "epochs": 2,
  "checkpoints": 2,
  "preview": "freeway-model-r1792678519106672000-step-000002000-preview.gif"
}