让小车学会保持平衡

本次运行结果

Dependency cache ready: /Users/sanbu/Documents/Codex/2026-10-20/revie/work/chapter-lab-runs/studio-zh/workspaces/cartpole/.cache/online-experiments/hands-on-modern-rl-experiment01-cartpole-1e241ccbaa8e.ready
代码仓库： /Users/sanbu/Documents/Codex/2026-10-20/revie/work/chapter-lab-runs/studio-zh/workspaces/cartpole
实验运行目录： /Users/sanbu/Documents/Codex/2026-10-20/revie/work/chapter-lab-runs/studio-zh/workspaces/cartpole/modelscope-space/hands-on-modern-rl-experiment01-cartpole

objc[28243]: Class SDLApplication is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11dd94890) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x11f6012c8). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[28243]: Class SDLAppDelegate is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11dd948e0) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x11f601318). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[28243]: Class SDLTranslatorResponder is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11dd94958) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x11f601390). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[28243]: Class SDLMessageBoxPresenter is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11dd94980) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x11f6013b8). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[28243]: Class SDL_cocoametalview is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11dd949d0) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x11f601408). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[28243]: Class SDLOpenGLContext is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11dd94a20) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x11f601458). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[28243]: Class SDL_ShapeData is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11dd94a98) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x11f6014d0). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[28243]: Class SDL_CocoaClosure is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11dd94ae8) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x11f601520). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[28243]: Class SDL_VideoData is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11dd94b38) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x11f601570). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[28243]: Class SDL_WindowData is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11dd94b88) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x11f6015c0). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[28243]: Class SDLWindow is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11dd94bb0) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x11f6015e8). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[28243]: Class Cocoa_WindowListener is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11dd94bd8) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x11f601610). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[28243]: Class SDLView is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11dd94c50) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x11f601688). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[28243]: Class METAL_RenderData is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11dd94cc8) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x11f601700). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[28243]: Class METAL_TextureData is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11dd94d18) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x11f601750). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[28243]: Class SDL_RumbleMotor is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11dd94d40) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x11f601778). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[28243]: Class SDL_RumbleContext is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11dd94d90) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x11f6017c8). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.

可选任务：
  cartpole-ppo         CartPole-v1 · PPO · CartPole-v1

所选任务： CartPole-v1 · PPO
算法： PPO
轮次安排：2 轮 × 15,000 步 = 30,000 总步数
每轮都会评估并保存一个可单独选择的模型。

Device: CPU
启动与在线 Studio 相同的训练流程…

PPO initialized · n_steps=1000 batch_size=50 initial_reward=85.6

EPOCH 1/2 step=15,000 mean_reward=469.4 std=61.2
SAVE model=policy.zip replay_reward=369

EPOCH 2/2 step=30,000 mean_reward=500.0 std=0.0
SAVE model=policy.zip replay_reward=500

Saved 2 independently selectable PPO policies · final_mean=500.0

最终阶段： complete
最终得分： 500.0
已保存 2 个本次运行可选择的轮次模型。
模型索引： /Users/sanbu/Documents/Codex/2026-10-20/revie/work/chapter-lab-runs/studio-zh/workspaces/cartpole/modelscope-space/hands-on-modern-rl-experiment01-cartpole/artifacts/notebook-models.json

运行下面的推理单元格，选择轮次模型并查看该策略的回放。

VBox(children=(Dropdown(description='训练轮次模型：', layout=Layout(width='95%'), options=(('轮次 2/2 · 30,000 步 · 得分 5…

默认选择最新轮次；也可以选择其他已保存模型，然后点击 回放所选策略.

{
  "score": 500.0,
  "steps": 30000,
  "epochs": 2,
  "checkpoints": 2,
  "preview": "learned-policy.gif"
}

{
  "score": 500.0,
  "steps": 30000,
  "epochs": 2,
  "checkpoints": 2,
  "preview": "learned-policy.gif"
}