Discover rewards in a maze

Saved run results

Dependency cache ready: /Users/sanbu/Documents/Codex/2026-10-20/revie/work/chapter-lab-runs/minigrid-en-final/workspaces/minigrid/.cache/online-experiments/hands-on-modern-rl-experiment06-minigrid-adventure-849307d84044.ready
Repository: /Users/sanbu/Documents/Codex/2026-10-20/revie/work/chapter-lab-runs/minigrid-en-final/workspaces/minigrid
Experiment runtime: /Users/sanbu/Documents/Codex/2026-10-20/revie/work/chapter-lab-runs/minigrid-en-final/workspaces/minigrid/modelscope-space/hands-on-modern-rl-experiment06-minigrid-adventure

Available tasks:
  empty                Reach the Goal · MiniGrid-Empty-6x6-v0
  doorkey              Door & Key · MiniGrid-DoorKey-6x6-v0
  multiroom            Multi-Room · MiniGrid-MultiRoom-N2-S4-v0
  unlock               Unlock the Door · MiniGrid-Unlock-v0
  obstructed           Obstructed Maze · MiniGrid-ObstructedMaze-1Q-v1
  dynamic              Dynamic Obstacles · MiniGrid-Dynamic-Obstacles-6x6-v0

Selected: Reach the Goal
Algorithm: PPO
Epoch schedule: 2 epochs × 1,000 steps = 2,000 total steps
Every epoch is evaluated and saved as a separately selectable model.

Device: CPU
Starting the same training generator used by the live Studio...

Initialized PPO with CnnPolicy on CPU

update · step 1,000
------------------------------------------------
| train/loss                 |     -0.00101993 |
| train/value_loss           |       0.0474573 |
| train/policy_gradient_loss |    -0.000102817 |
| train/entropy_loss         |        -1.94193 |
| train/learning_rate        |          0.0003 |
| train/n_updates            |               4 |
------------------------------------------------
EVAL step=1,000 mean_reward=0.00 std=0.00 mean_length=144.0
SAVE epoch=1/2 model=policy.zip replay=empty-learned-policy.gif

update · step 2,000
------------------------------------------------
| train/loss                 |     -0.00374641 |
| train/value_loss           |      0.00900346 |
| train/policy_gradient_loss |     -0.00745966 |
| train/entropy_loss         |        -1.91692 |
| train/learning_rate        |          0.0003 |
| train/n_updates            |              12 |
------------------------------------------------
EVAL step=2,000 mean_reward=0.00 std=0.00 mean_length=144.0
SAVE epoch=2/2 model=policy.zip replay=empty-learned-policy.gif

Saved 2 independently selectable epoch policies and learned-policy replays

Final phase: complete
Final score: 0.0
Saved 2 selectable epoch models in this run.
Model index: /Users/sanbu/Documents/Codex/2026-10-20/revie/work/chapter-lab-runs/minigrid-en-final/workspaces/minigrid/modelscope-space/hands-on-modern-rl-experiment06-minigrid-adventure/artifacts/notebook-models.json

Run the inference cell below to choose an epoch model and visualize its exact learned policy.

VBox(children=(Dropdown(description='Epoch model:', layout=Layout(width='95%'), options=(('Epoch 2/2 · 2,000 s…

The newest epoch is selected by default. Choose another saved model, then click Visualize selected policy.

{
  "score": 0.0,
  "steps": 2000,
  "epochs": 2,
  "checkpoints": 2,
  "preview": "empty-learned-policy.gif"
}

{
  "score": 0.0,
  "steps": 2000,
  "epochs": 2,
  "checkpoints": 2,
  "preview": "empty-learned-policy.gif"
}