Teach several agents to cooperate

Saved run results

Dependency cache ready: /Users/sanbu/Documents/Codex/2026-10-20/revie/work/chapter-lab-runs/studio-en-rest/workspaces/multiagent/.cache/online-experiments/hands-on-modern-rl-experiment05-multiagent-games-4efbca48fa8c.ready
Repository: /Users/sanbu/Documents/Codex/2026-10-20/revie/work/chapter-lab-runs/studio-en-rest/workspaces/multiagent
Experiment runtime: /Users/sanbu/Documents/Codex/2026-10-20/revie/work/chapter-lab-runs/studio-en-rest/workspaces/multiagent/modelscope-space/hands-on-modern-rl-experiment05-multiagent-games

Available tasks:
  simple-spread        Cooperative Navigation · MPE/simple_spread_v3
  simple-tag           Predator & Prey · MPE/simple_tag_v3
  pistonball           Pistonball · Butterfly/pistonball_v6
  cooperative-pong     Cooperative Pong · Butterfly/cooperative_pong_v6

Selected: Cooperative Navigation
Algorithm: Parameter-sharing PPO
Epoch schedule: 2 epochs × 1,008 steps = 2,016 total steps
Every epoch is evaluated and saved as a separately selectable model.

Device: CPU
Starting the same training generator used by the live Studio...

objc[26936]: Class SDL_RumbleMotor is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11f7d8d40) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x1211889c8). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[26936]: Class SDL_RumbleContext is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11f7d8d90) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x121188a18). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[26936]: Class SDLApplication is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11f7d8890) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x121188a68). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[26936]: Class SDLAppDelegate is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11f7d88e0) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x121188ab8). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[26936]: Class SDLTranslatorResponder is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11f7d8958) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x121188b30). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[26936]: Class SDLMessageBoxPresenter is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11f7d8980) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x121188b58). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[26936]: Class SDL_cocoametalview is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11f7d89d0) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x121188ba8). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[26936]: Class SDLOpenGLContext is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11f7d8a20) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x121188bf8). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[26936]: Class SDL_ShapeData is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11f7d8a98) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x121188c70). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[26936]: Class SDL_CocoaClosure is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11f7d8ae8) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x121188cc0). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[26936]: Class SDL_VideoData is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11f7d8b38) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x121188d10). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[26936]: Class SDL_WindowData is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11f7d8b88) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x121188d60). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[26936]: Class SDLWindow is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11f7d8bb0) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x121188d88). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[26936]: Class Cocoa_WindowListener is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11f7d8bd8) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x121188db0). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[26936]: Class SDLView is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11f7d8c50) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x121188e28). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[26936]: Class METAL_RenderData is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11f7d8cc8) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x121188ea0). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
objc[26936]: Class METAL_TextureData is implemented in both /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/cv2/.dylibs/libSDL2-2.0.0.dylib (0x11f7d8d18) and /Users/sanbu/Documents/Codex/2026-10-20/revie/work/studio-venv/lib/python3.12/site-packages/pygame/.dylibs/libSDL2-2.0.0.dylib (0x121188ef0). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.

Initialized shared PPO policy for 3 vector-agent slots

PPO update step=1,008 mean_agent_reward=-514.2309
SAVE epoch=1/2 model=policy.zip replay_frames=155

PPO update step=2,016 mean_agent_reward=-10549.4402
SAVE epoch=2/2 model=policy.zip replay_frames=155

Saved 2 independently selectable multi-agent policies and replays

Final phase: complete
Final score: -10549.440152548465
Saved 2 selectable epoch models in this run.
Model index: /Users/sanbu/Documents/Codex/2026-10-20/revie/work/chapter-lab-runs/studio-en-rest/workspaces/multiagent/modelscope-space/hands-on-modern-rl-experiment05-multiagent-games/artifacts/notebook-models.json

Run the inference cell below to choose an epoch model and visualize its exact learned policy.

VBox(children=(Dropdown(description='Epoch model:', layout=Layout(width='95%'), options=(('Epoch 2/2 · 2,016 s…

The newest epoch is selected by default. Choose another saved model, then click Visualize selected policy.

{
  "score": -10549.440152548465,
  "steps": 2016,
  "epochs": 2,
  "checkpoints": 2,
  "preview": "learned-policy.gif"
}

{
  "score": -10549.440152548465,
  "steps": 2016,
  "epochs": 2,
  "checkpoints": 2,
  "preview": "learned-policy.gif"
}