Keep the cart balanced

Saved run results

Dependency cache ready: /Users/sanbu/Documents/Codex/2026-10-20/revie/work/chapter-lab-runs/studio-en/workspaces/cartpole/.cache/online-experiments/hands-on-modern-rl-experiment01-cartpole-1e241ccbaa8e.ready
Repository: /Users/sanbu/Documents/Codex/2026-10-20/revie/work/chapter-lab-runs/studio-en/workspaces/cartpole
Experiment runtime: /Users/sanbu/Documents/Codex/2026-10-20/revie/work/chapter-lab-runs/studio-en/workspaces/cartpole/modelscope-space/hands-on-modern-rl-experiment01-cartpole

objc[25450]: Class SDL_RumbleMotor is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11d8ccd40) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x11f27c9c8). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[25450]: Class SDL_RumbleContext is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11d8ccd90) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x11f27ca18). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[25450]: Class SDLApplication is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11d8cc890) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x11f27ca68). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[25450]: Class SDLAppDelegate is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11d8cc8e0) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x11f27cab8). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[25450]: Class SDLTranslatorResponder is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11d8cc958) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x11f27cb30). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[25450]: Class SDLMessageBoxPresenter is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11d8cc980) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x11f27cb58). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[25450]: Class SDL_cocoametalview is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11d8cc9d0) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x11f27cba8). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[25450]: Class SDLOpenGLContext is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11d8cca20) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x11f27cbf8). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[25450]: Class SDL_ShapeData is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11d8cca98) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x11f27cc70). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[25450]: Class SDL_CocoaClosure is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11d8ccae8) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x11f27ccc0). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[25450]: Class SDL_VideoData is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11d8ccb38) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x11f27cd10). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[25450]: Class SDL_WindowData is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11d8ccb88) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x11f27cd60). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[25450]: Class SDLWindow is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11d8ccbb0) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x11f27cd88). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[25450]: Class Cocoa_WindowListener is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11d8ccbd8) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x11f27cdb0). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[25450]: Class SDLView is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11d8ccc50) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x11f27ce28). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[25450]: Class METAL_RenderData is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11d8cccc8) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x11f27cea0). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[25450]: Class METAL_TextureData is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11d8ccd18) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x11f27cef0). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.

Available tasks:
  cartpole-ppo         CartPole-v1 · PPO · CartPole-v1

Selected: CartPole-v1 · PPO
Algorithm: PPO
Epoch schedule: 2 epochs × 15,000 steps = 30,000 total steps
Every epoch is evaluated and saved as a separately selectable model.

Device: CPU
Starting the same training generator used by the live Studio...

PPO initialized · n_steps=1000 batch_size=50 initial_reward=84.2

EPOCH 1/2 step=15,000 mean_reward=481.0 std=36.5
SAVE model=policy.zip replay_reward=369

EPOCH 2/2 step=30,000 mean_reward=500.0 std=0.0
SAVE model=policy.zip replay_reward=500

Saved 2 independently selectable PPO policies · final_mean=500.0

Final phase: complete
Final score: 500.0
Saved 2 selectable epoch models in this run.
Model index: /Users/sanbu/Documents/Codex/2026-10-20/revie/work/chapter-lab-runs/studio-en/workspaces/cartpole/modelscope-space/hands-on-modern-rl-experiment01-cartpole/artifacts/notebook-models.json

Run the inference cell below to choose an epoch model and visualize its exact learned policy.

VBox(children=(Dropdown(description='Epoch model:', layout=Layout(width='95%'), options=(('Epoch 2/2 · 30,000 …

The newest epoch is selected by default. Choose another saved model, then click Visualize selected policy.

{
  "score": 500.0,
  "steps": 30000,
  "epochs": 2,
  "checkpoints": 2,
  "preview": "learned-policy.gif"
}

{
  "score": 500.0,
  "steps": 30000,
  "epochs": 2,
  "checkpoints": 2,
  "preview": "learned-policy.gif"
}