Try different control environments

Saved run results

Dependency cache ready: /Users/sanbu/Documents/Codex/2026-10-20/revie/work/chapter-lab-runs/studio-en/workspaces/gymnasium/.cache/online-experiments/hands-on-modern-rl-experiment-gymnasium-75bfe84ee278.ready
Repository: /Users/sanbu/Documents/Codex/2026-10-20/revie/work/chapter-lab-runs/studio-en/workspaces/gymnasium
Experiment runtime: /Users/sanbu/Documents/Codex/2026-10-20/revie/work/chapter-lab-runs/studio-en/workspaces/gymnasium/modelscope-space/hands-on-modern-rl-experiment-gymnasium

Curated experiments:
  Bandit · ε-greedy
  Blackjack · Monte Carlo
  GridWorld · Q-Learning
  FrozenLake · Q-Learning
  CliffWalking · SARSA
  Taxi · Q-Learning
  CartPole · DQN
  CartPole · PPO
  MountainCar · Tabular Q
  Acrobot · PPO
  Pendulum · PPO
  MountainCarContinuous · SAC
Epoch schedule: 2 epochs × 1,000 steps

Matplotlib is building the font cache; this may take a moment.

      Live training log
      Multi-Armed Bandit training console
========================================================================
    0.0s  CONFIG  steps=2000 epochs=2 alpha=0.1 epsilon=0.1 seed=42
   11.5s  EPOCH   1/2 step=1000/2000 average_reward=0.665 best_arm=3
   11.5s  SAVE    model_id=1792630667-472000-epoch-01

      Live training log
      Multi-Armed Bandit training console
========================================================================
    0.0s  CONFIG  steps=2000 epochs=2 alpha=0.1 epsilon=0.1 seed=42
   11.5s  EPOCH   1/2 step=1000/2000 average_reward=0.665 best_arm=3
   11.5s  SAVE    model_id=1792630667-472000-epoch-01
   11.6s  EPOCH   2/2 step=2000/2000 average_reward=0.723 best_arm=3
   11.6s  SAVE    model_id=1792630667-472000-epoch-02

      Live training log
      Multi-Armed Bandit training console
========================================================================
    0.0s  CONFIG  steps=2000 epochs=2 alpha=0.1 epsilon=0.1 seed=42
   11.5s  EPOCH   1/2 step=1000/2000 average_reward=0.665 best_arm=3
   11.5s  SAVE    model_id=1792630667-472000-epoch-01
   11.6s  EPOCH   2/2 step=2000/2000 average_reward=0.723 best_arm=3
   11.6s  SAVE    model_id=1792630667-472000-epoch-02

<Figure size 820x400 with 1 Axes>

Result artifact: /Users/sanbu/Documents/Codex/2026-10-20/revie/work/chapter-lab-runs/studio-en/workspaces/gymnasium/modelscope-space/hands-on-modern-rl-experiment-gymnasium/artifacts/bandit-run-summary.json
Saved {len(playground.load_models(EXPERIMENT))} selectable epoch models for this task.

VBox(children=(Dropdown(description='Epoch model:', layout=Layout(width='95%'), options=(('Epoch 2/2 · 2,000/2…

The newest epoch is selected by default. Choose another saved model, then click Visualize selected policy.

{
  "score": 0.723,
  "steps": 2000,
  "epochs": 2,
  "checkpoints": 2,
  "preview": "learned-policy.png"
}

{
  "score": 0.723,
  "steps": 2000,
  "epochs": 2,
  "checkpoints": 2,
  "preview": "learned-policy.png"
}