Improve by playing against an opponent

Saved run results

Dependency cache ready: /Users/sanbu/Documents/Codex/2026-10-20/revie/work/chapter-lab-runs/studio-en/workspaces/selfplay/.cache/online-experiments/hands-on-modern-rl-experiment04-board-selfplay-9becbb2cc824.ready
Repository: /Users/sanbu/Documents/Codex/2026-10-20/revie/work/chapter-lab-runs/studio-en/workspaces/selfplay
Experiment runtime: /Users/sanbu/Documents/Codex/2026-10-20/revie/work/chapter-lab-runs/studio-en/workspaces/selfplay/modelscope-space/hands-on-modern-rl-experiment04-board-selfplay

Available tasks:
  kuhn-poker           Kuhn Poker · kuhn_poker
  leduc-poker          Leduc Poker · leduc_poker
  tic-tac-toe          Tic-Tac-Toe · tic_tac_toe
  connect-four         Connect Four · connect_four

Selected: Kuhn Poker
Algorithm: CFR+
Epoch schedule: 2 epochs × 250 steps = 500 total steps
Every epoch is evaluated and saved as a separately selectable model.

Device: CPU
Starting the same training generator used by the live Studio...

Optional module pokerkit_wrapper was not importable: No module named 'pokerkit'

CFR+ iteration=250 exploitability=0.00041426
SAVE epoch=1/2 policy=policy.npz

CFR+ iteration=500 exploitability=0.00017270
SAVE epoch=2/2 policy=policy.npz

Saved 2 CFR policies and sampled one terminal game per epoch

Final phase: complete
Final score: -0.00017269684974360877
Saved 2 selectable epoch models in this run.
Model index: /Users/sanbu/Documents/Codex/2026-10-20/revie/work/chapter-lab-runs/studio-en/workspaces/selfplay/modelscope-space/hands-on-modern-rl-experiment04-board-selfplay/artifacts/notebook-models.json

Run the inference cell below to choose an epoch model and visualize its exact learned policy.

VBox(children=(Dropdown(description='Epoch model:', layout=Layout(width='95%'), options=(('Epoch 2/2 · 500 ste…

The newest epoch is selected by default. Choose another saved model, then click Visualize selected policy.

{
  "score": -0.00017269684974360877,
  "steps": 500,
  "epochs": 2,
  "checkpoints": 2,
  "preview": "kuhn-poker-learned-policy.gif"
}

{
  "score": -0.00017269684974360877,
  "steps": 500,
  "epochs": 2,
  "checkpoints": 2,
  "preview": "kuhn-poker-learned-policy.gif"
}