Turn pixels into actions

Saved run results

Dependency cache ready: /Users/sanbu/Documents/Codex/2026-10-20/revie/work/chapter-lab-runs/atari-en-final/workspaces/atari/.cache/online-experiments/hands-on-modern-rl-experiment03-atari-a4edb25287ec.ready
Repository: /Users/sanbu/Documents/Codex/2026-10-20/revie/work/chapter-lab-runs/atari-en-final/workspaces/atari
Experiment runtime: /Users/sanbu/Documents/Codex/2026-10-20/revie/work/chapter-lab-runs/atari-en-final/workspaces/atari/modelscope-space/hands-on-modern-rl-experiment03-atari

Available tasks:
  pong                 Pong · ALE/Pong-v5
  breakout             Breakout · ALE/Breakout-v5
  space-invaders       Space Invaders · ALE/SpaceInvaders-v5
  freeway              Freeway · ALE/Freeway-v5
  seaquest             Seaquest · ALE/Seaquest-v5
  qbert                Q*bert · ALE/Qbert-v5
  beam-rider           Beam Rider · ALE/BeamRider-v5
  enduro               Enduro · ALE/Enduro-v5

Selected: Freeway
Algorithm: DQN
Epoch schedule: 2 epochs × 1,000 steps = 2,000 total steps
Every epoch is evaluated and saved as a separately selectable model.

Device: CPU
Starting the same training generator used by the live Studio...

A.L.E: Arcade Learning Environment (version 0.12.1+8a8fafb)
[Powered by Stella]

Initialized DQN with CnnPolicy on CPU
BASELINE replay_buffer=10,000 warmup=200 batch=32 target_update=250 exploration=1.00→0.01 train_reward=clipped eval_reward=raw schedule_horizon=2,000
SMOKE_TEST evaluation=2 epochs × 1 episode
TRAINING_PLAN epochs=2 steps_per_epoch≈1,000 save_policy=every_epoch

PROGRESS step=1,000/2,000 phase=gradient updates fps=111.3 eta=0.1min
EVAL starting epoch 1/2 at step=1,000

update · step 1,000
----------------------------------------------
| rollout/exploration_rate |            0.01 |
| train/loss               |     1.42251e-08 |
| train/learning_rate      |          0.0001 |
| train/n_updates          |             199 |
----------------------------------------------
EVAL step=1,000 mean_reward=0.00 std=0.00 mean_length=2047.0
EPOCH 1/2 complete
CHECKPOINT saved epoch model: freeway-model-r1792678519106672000-step-000001000.zip
CHECKPOINT this is the best policy so far and will be used for the final replay

PROGRESS step=2,000/2,000 phase=gradient updates fps=89.3 eta=0.0min
EVAL starting epoch 2/2 at step=2,000

update · step 2,000
----------------------------------------------
| rollout/exploration_rate |            0.01 |
| train/loss               |     3.70372e-08 |
| train/learning_rate      |          0.0001 |
| train/n_updates          |             448 |
----------------------------------------------
EVAL step=2,000 mean_reward=0.00 std=0.00 mean_length=2045.0
EPOCH 2/2 complete
CHECKPOINT saved epoch model: freeway-model-r1792678519106672000-step-000002000.zip

Restored best epoch model from step 1,000 with mean reward 0.00
Saved 2 selectable epoch policy models for run 1792678519106672000
Generated the best epoch replay: freeway-model-r1792678519106672000-step-000001000-preview.gif

Final phase: complete
Final score: 0.0
Saved 2 selectable epoch models in this run.
Model index: /Users/sanbu/Documents/Codex/2026-10-20/revie/work/chapter-lab-runs/atari-en-final/workspaces/atari/modelscope-space/hands-on-modern-rl-experiment03-atari/artifacts/notebook-models.json

Run the inference cell below to choose an epoch model and visualize its exact learned policy.

VBox(children=(Dropdown(description='Epoch model:', layout=Layout(width='95%'), options=(('Epoch 2/2 · 2,000 s…

The newest epoch is selected by default. Choose another saved model, then click Visualize selected policy.

{
  "score": 0.0,
  "steps": 2000,
  "epochs": 2,
  "checkpoints": 2,
  "preview": "freeway-model-r1792678519106672000-step-000002000-preview.gif"
}

{
  "score": 0.0,
  "steps": 2000,
  "epochs": 2,
  "checkpoints": 2,
  "preview": "freeway-model-r1792678519106672000-step-000002000-preview.gif"
}