Train a first-person visual policy

Saved run results

Dependency cache ready: /mnt/workspace/hands-on-modern-rl-notebooks/hands-on-modern-rl-experiment10-minestudio-b44597457f59.ready
Repository: /runs/minecraft-en-diskcache/workspaces/minecraft
Experiment runtime: /runs/minecraft-en-diskcache/workspaces/minecraft/modelscope-space/hands-on-modern-rl-experiment10-minestudio

Available tasks:
  mine-dirt            Mine Dirt · Visual PPO · MineStudio/mine-dirt
  collect-wood         Collect Wood · Visual PPO · MineStudio/collect-wood
  hunt-sheep           Hunt a Sheep · Visual PPO · MineStudio/hunt-sheep
  combat-zombie        Combat a Zombie · Visual PPO · MineStudio/combat-zombie

Selected: Mine Dirt · Visual PPO
Algorithm: CNN PPO
Epoch schedule: 2 epochs × 512 steps = 1,024 total steps
Every epoch is evaluated and saved as a separately selectable model.

Device: CPU
Starting the same training generator used by the live Studio...

Preparing persistent MineStudio engine cache at /mnt/workspace/hands-on-modern-rl/minestudio
Temurin Java 8 ready: /mnt/workspace/hands-on-modern-rl/temurin-jre8/jdk8u504-b01-jre/bin/java
MineStudio engine source: https://huggingface.co
MineStudio engine cache ready: /mnt/workspace/hands-on-modern-rl/minestudio/engine/build/libs/mcprec-6.13.jar
Starting the Minecraft renderer
Launching the MineStudio Java/Forge world. The first world connection normally takes several minutes; this run remains active while the controls stay locked.
INFO: Starting Minecraft process with device: cpu
CommandsCallback(commands=['/give @s minecraft:stone_shovel', '/execute as @p at @s run fill ~1 ~ ~1 ~4 ~2 ~4 minecraft:dirt'])
RewardsCallback()
CommandsCallback(commands=['/give @s minecraft:stone_shovel', '/execute as @p at @s run fill ~1 ~ ~1 ~4 ~2 ~4 minecraft:dirt'])
RewardsCallback()
CommandsCallback(commands=['/give @s minecraft:stone_shovel', '/execute as @p at @s run fill ~1 ~ ~1 ~4 ~2 ~4 minecraft:dirt'])
RewardsCallback()
Minecraft PPO update · step=512
device=cpu  fps=—
episode_reward_mean=0.0000  value_loss=0.03496
policy_gradient_loss=-0.01262
SAVE epoch=1/2 model=policy.zip
CommandsCallback(commands=['/give @s minecraft:stone_shovel', '/execute as @p at @s run fill ~1 ~ ~1 ~4 ~2 ~4 minecraft:dirt'])
RewardsCallback()
CommandsCallback(commands=['/give @s minecraft:stone_shovel', '/execute as @p at @s run fill ~1 ~ ~1 ~4 ~2 ~4 minecraft:dirt'])
RewardsCallback()
Minecraft PPO update · step=1,024
device=cpu  fps=—
episode_reward_mean=0.0000  value_loss=0.00154
policy_gradient_loss=-0.00418
SAVE epoch=2/2 model=policy.zip
Training is complete. Reusing one fresh Minecraft world to replay every saved policy.
INFO: Starting Minecraft process with device: cpu
CommandsCallback(commands=['/give @s minecraft:stone_shovel', '/execute as @p at @s run fill ~1 ~ ~1 ~4 ~2 ~4 minecraft:dirt'])
RewardsCallback()
REPLAY epoch=1/2 model=policy.zip return=0.238
CommandsCallback(commands=['/give @s minecraft:stone_shovel', '/execute as @p at @s run fill ~1 ~ ~1 ~4 ~2 ~4 minecraft:dirt'])
RewardsCallback()
REPLAY epoch=2/2 model=policy.zip return=0.590
Saved 2 independently selectable Minecraft policies and replays

Final phase: complete
Final score: 0.0
Saved 2 selectable epoch models in this run.
Model index: /runs/minecraft-en-diskcache/workspaces/minecraft/modelscope-space/hands-on-modern-rl-experiment10-minestudio/artifacts/notebook-models.json

Gym has been unmaintained since 2022 and does not support NumPy 2.0 amongst other critical functionality.
Please upgrade to Gymnasium, the maintained drop-in replacement of Gym, or contact the authors of your software and request that they upgrade.
Users of this version of Gym should be able to simply replace 'import gym' with 'import gymnasium as gym' in the vast majority of cases.
See the migration guide at https://gymnasium.farama.org/introduction/migration_guide/ for additional information.
ERROR:minestudio.simulator.minerl.env.malmo.instance.dcb778:[15:26:02] [Render thread/WARN]: ERROR : Couldn't load Narrator library : Unable to load library 'fliteWrapper': Can't obtain InputStream for linux-x86-64/libfliteWrapper.so
ERROR:minestudio.simulator.minerl.env.malmo.instance.dcb778:[15:26:06] [Realms Notification Availability checker #1/INFO]: Could not authorize you against Realms server: java.lang.RuntimeException: Failed to parse into SignedJWT: null
ERROR:minestudio.simulator.minerl.env.malmo.instance.dcb778:[15:26:12] [Worker-Main-12/ERROR]: Using missing texture, unable to load minecraft:optifine/ctm/default/empty.png : java.io.FileNotFoundException: minecraft:optifine/ctm/default/empty.png
ERROR:minestudio.simulator.minerl.env.malmo.instance.dcb778:[15:26:12] [Render thread/ERROR]: Error starting SoundSystem. Turning off sounds & music
ERROR:minestudio.simulator.minerl.env.malmo.instance.dcb778:java.lang.IllegalStateException: Failed to open OpenAL device
WARNING:minestudio.simulator.minerl.env.malmo.instance.dcb778:[15:26:12] [Render thread/WARN]: [OptiFine] Missing CTM sprite: minecraft:optifine/ctm/default/empty
WARNING:minestudio.simulator.minerl.env.malmo.instance.dcb778:[15:26:15] [Render thread/WARN]: Ambiguity between arguments [teleport, destination] and [teleport, targets] with inputs: [Player, 0123, @e, dd12be42-52a9-4a91-a8a1-11c01849e498]
WARNING:minestudio.simulator.minerl.env.malmo.instance.dcb778:[15:26:15] [Render thread/WARN]: Ambiguity between arguments [teleport, location] and [teleport, destination] with inputs: [0.1 -0.5 .9, 0 0 0]
WARNING:minestudio.simulator.minerl.env.malmo.instance.dcb778:[15:26:15] [Render thread/WARN]: Ambiguity between arguments [teleport, location] and [teleport, targets] with inputs: [0.1 -0.5 .9, 0 0 0]
WARNING:minestudio.simulator.minerl.env.malmo.instance.dcb778:[15:26:15] [Render thread/WARN]: Ambiguity between arguments [teleport, targets] and [teleport, destination] with inputs: [Player, 0123, dd12be42-52a9-4a91-a8a1-11c01849e498]
WARNING:minestudio.simulator.minerl.env.malmo.instance.dcb778:[15:26:15] [Render thread/WARN]: Ambiguity between arguments [teleport, targets, location] and [teleport, targets, destination] with inputs: [0.1 -0.5 .9, 0 0 0]
WARNING:minestudio.simulator.minerl.env.malmo.instance.dcb778:[15:26:35] [Render thread/WARN]: Failed to load texture: minecraft:optifine/ctm/default/empty.png
ERROR:minestudio.simulator.minerl.env.malmo.instance.dcb778:java.io.FileNotFoundException: minecraft:optifine/ctm/default/empty.png
WARNING:minestudio.simulator.minerl.env.malmo.instance.dcb778:[15:26:58] [Render thread/WARN]: Ambiguity between arguments [teleport, destination] and [teleport, targets] with inputs: [Player, 0123, @e, dd12be42-52a9-4a91-a8a1-11c01849e498]
WARNING:minestudio.simulator.minerl.env.malmo.instance.dcb778:[15:26:58] [Render thread/WARN]: Ambiguity between arguments [teleport, location] and [teleport, destination] with inputs: [0.1 -0.5 .9, 0 0 0]
WARNING:minestudio.simulator.minerl.env.malmo.instance.dcb778:[15:26:58] [Render thread/WARN]: Ambiguity between arguments [teleport, location] and [teleport, targets] with inputs: [0.1 -0.5 .9, 0 0 0]
WARNING:minestudio.simulator.minerl.env.malmo.instance.dcb778:[15:26:58] [Render thread/WARN]: Ambiguity between arguments [teleport, targets] and [teleport, destination] with inputs: [Player, 0123, dd12be42-52a9-4a91-a8a1-11c01849e498]
WARNING:minestudio.simulator.minerl.env.malmo.instance.dcb778:[15:26:58] [Render thread/WARN]: Ambiguity between arguments [teleport, targets, location] and [teleport, targets, destination] with inputs: [0.1 -0.5 .9, 0 0 0]
WARNING:minestudio.simulator.minerl.env.malmo.instance.dcb778:[15:27:10] [Render thread/WARN]: Failed to load texture: minecraft:optifine/ctm/default/empty.png
ERROR:minestudio.simulator.minerl.env.malmo.instance.dcb778:java.io.FileNotFoundException: minecraft:optifine/ctm/default/empty.png
WARNING:minestudio.simulator.minerl.env.malmo.instance.dcb778:[15:27:24] [Render thread/WARN]: Ambiguity between arguments [teleport, destination] and [teleport, targets] with inputs: [Player, 0123, @e, dd12be42-52a9-4a91-a8a1-11c01849e498]
WARNING:minestudio.simulator.minerl.env.malmo.instance.dcb778:[15:27:24] [Render thread/WARN]: Ambiguity between arguments [teleport, location] and [teleport, destination] with inputs: [0.1 -0.5 .9, 0 0 0]
WARNING:minestudio.simulator.minerl.env.malmo.instance.dcb778:[15:27:24] [Render thread/WARN]: Ambiguity between arguments [teleport, location] and [teleport, targets] with inputs: [0.1 -0.5 .9, 0 0 0]
WARNING:minestudio.simulator.minerl.env.malmo.instance.dcb778:[15:27:24] [Render thread/WARN]: Ambiguity between arguments [teleport, targets] and [teleport, destination] with inputs: [Player, 0123, dd12be42-52a9-4a91-a8a1-11c01849e498]
WARNING:minestudio.simulator.minerl.env.malmo.instance.dcb778:[15:27:24] [Render thread/WARN]: Ambiguity between arguments [teleport, targets, location] and [teleport, targets, destination] with inputs: [0.1 -0.5 .9, 0 0 0]
WARNING:minestudio.simulator.minerl.env.malmo.instance.dcb778:[15:27:35] [Render thread/WARN]: Failed to load texture: minecraft:optifine/ctm/default/empty.png
ERROR:minestudio.simulator.minerl.env.malmo.instance.dcb778:java.io.FileNotFoundException: minecraft:optifine/ctm/default/empty.png
WARNING:minestudio.simulator.minerl.env.malmo.instance.dcb778:[15:27:52] [Render thread/WARN]: Ambiguity between arguments [teleport, destination] and [teleport, targets] with inputs: [Player, 0123, @e, dd12be42-52a9-4a91-a8a1-11c01849e498]
WARNING:minestudio.simulator.minerl.env.malmo.instance.dcb778:[15:27:52] [Render thread/WARN]: Ambiguity between arguments [teleport, location] and [teleport, destination] with inputs: [0.1 -0.5 .9, 0 0 0]
WARNING:minestudio.simulator.minerl.env.malmo.instance.dcb778:[15:27:52] [Render thread/WARN]: Ambiguity between arguments [teleport, location] and [teleport, targets] with inputs: [0.1 -0.5 .9, 0 0 0]
WARNING:minestudio.simulator.minerl.env.malmo.instance.dcb778:[15:27:52] [Render thread/WARN]: Ambiguity between arguments [teleport, targets] and [teleport, destination] with inputs: [Player, 0123, dd12be42-52a9-4a91-a8a1-11c01849e498]
WARNING:minestudio.simulator.minerl.env.malmo.instance.dcb778:[15:27:52] [Render thread/WARN]: Ambiguity between arguments [teleport, targets, location] and [teleport, targets, destination] with inputs: [0.1 -0.5 .9, 0 0 0]
WARNING:minestudio.simulator.minerl.env.malmo.instance.dcb778:[15:28:04] [Render thread/WARN]: Failed to load texture: minecraft:optifine/ctm/default/empty.png
ERROR:minestudio.simulator.minerl.env.malmo.instance.dcb778:java.io.FileNotFoundException: minecraft:optifine/ctm/default/empty.png
WARNING:minestudio.simulator.minerl.env.malmo.instance.dcb778:[15:28:19] [Render thread/WARN]: Ambiguity between arguments [teleport, destination] and [teleport, targets] with inputs: [Player, 0123, @e, dd12be42-52a9-4a91-a8a1-11c01849e498]
WARNING:minestudio.simulator.minerl.env.malmo.instance.dcb778:[15:28:19] [Render thread/WARN]: Ambiguity between arguments [teleport, location] and [teleport, destination] with inputs: [0.1 -0.5 .9, 0 0 0]
WARNING:minestudio.simulator.minerl.env.malmo.instance.dcb778:[15:28:19] [Render thread/WARN]: Ambiguity between arguments [teleport, location] and [teleport, targets] with inputs: [0.1 -0.5 .9, 0 0 0]
WARNING:minestudio.simulator.minerl.env.malmo.instance.dcb778:[15:28:19] [Render thread/WARN]: Ambiguity between arguments [teleport, targets] and [teleport, destination] with inputs: [Player, 0123, dd12be42-52a9-4a91-a8a1-11c01849e498]
WARNING:minestudio.simulator.minerl.env.malmo.instance.dcb778:[15:28:19] [Render thread/WARN]: Ambiguity between arguments [teleport, targets, location] and [teleport, targets, destination] with inputs: [0.1 -0.5 .9, 0 0 0]
WARNING:minestudio.simulator.minerl.env.malmo.instance.dcb778:[15:28:30] [Render thread/WARN]: Failed to load texture: minecraft:optifine/ctm/default/empty.png
ERROR:minestudio.simulator.minerl.env.malmo.instance.dcb778:java.io.FileNotFoundException: minecraft:optifine/ctm/default/empty.png
ERROR:minestudio.simulator.minerl.env.malmo.instance.128321:[15:28:59] [Render thread/WARN]: ERROR : Couldn't load Narrator library : Unable to load library 'fliteWrapper': Can't obtain InputStream for linux-x86-64/libfliteWrapper.so
ERROR:minestudio.simulator.minerl.env.malmo.instance.128321:[15:29:02] [Realms Notification Availability checker #1/INFO]: Could not authorize you against Realms server: java.lang.RuntimeException: Failed to parse into SignedJWT: null
ERROR:minestudio.simulator.minerl.env.malmo.instance.128321:[15:29:08] [Worker-Main-9/ERROR]: Using missing texture, unable to load minecraft:optifine/ctm/default/empty.png : java.io.FileNotFoundException: minecraft:optifine/ctm/default/empty.png
ERROR:minestudio.simulator.minerl.env.malmo.instance.128321:[15:29:08] [Render thread/ERROR]: Error starting SoundSystem. Turning off sounds & music
ERROR:minestudio.simulator.minerl.env.malmo.instance.128321:java.lang.IllegalStateException: Failed to open OpenAL device
WARNING:minestudio.simulator.minerl.env.malmo.instance.128321:[15:29:08] [Render thread/WARN]: [OptiFine] Missing CTM sprite: minecraft:optifine/ctm/default/empty
WARNING:minestudio.simulator.minerl.env.malmo.instance.128321:[15:29:11] [Render thread/WARN]: Ambiguity between arguments [teleport, destination] and [teleport, targets] with inputs: [Player, 0123, @e, dd12be42-52a9-4a91-a8a1-11c01849e498]
WARNING:minestudio.simulator.minerl.env.malmo.instance.128321:[15:29:11] [Render thread/WARN]: Ambiguity between arguments [teleport, location] and [teleport, destination] with inputs: [0.1 -0.5 .9, 0 0 0]
WARNING:minestudio.simulator.minerl.env.malmo.instance.128321:[15:29:11] [Render thread/WARN]: Ambiguity between arguments [teleport, location] and [teleport, targets] with inputs: [0.1 -0.5 .9, 0 0 0]
WARNING:minestudio.simulator.minerl.env.malmo.instance.128321:[15:29:11] [Render thread/WARN]: Ambiguity between arguments [teleport, targets] and [teleport, destination] with inputs: [Player, 0123, dd12be42-52a9-4a91-a8a1-11c01849e498]
WARNING:minestudio.simulator.minerl.env.malmo.instance.128321:[15:29:11] [Render thread/WARN]: Ambiguity between arguments [teleport, targets, location] and [teleport, targets, destination] with inputs: [0.1 -0.5 .9, 0 0 0]
WARNING:minestudio.simulator.minerl.env.malmo.instance.128321:[15:29:29] [Render thread/WARN]: Failed to load texture: minecraft:optifine/ctm/default/empty.png
ERROR:minestudio.simulator.minerl.env.malmo.instance.128321:java.io.FileNotFoundException: minecraft:optifine/ctm/default/empty.png
WARNING:minestudio.simulator.minerl.env.malmo.instance.128321:[15:29:52] [Render thread/WARN]: Ambiguity between arguments [teleport, destination] and [teleport, targets] with inputs: [Player, 0123, @e, dd12be42-52a9-4a91-a8a1-11c01849e498]
WARNING:minestudio.simulator.minerl.env.malmo.instance.128321:[15:29:52] [Render thread/WARN]: Ambiguity between arguments [teleport, location] and [teleport, destination] with inputs: [0.1 -0.5 .9, 0 0 0]
WARNING:minestudio.simulator.minerl.env.malmo.instance.128321:[15:29:52] [Render thread/WARN]: Ambiguity between arguments [teleport, location] and [teleport, targets] with inputs: [0.1 -0.5 .9, 0 0 0]
WARNING:minestudio.simulator.minerl.env.malmo.instance.128321:[15:29:52] [Render thread/WARN]: Ambiguity between arguments [teleport, targets] and [teleport, destination] with inputs: [Player, 0123, dd12be42-52a9-4a91-a8a1-11c01849e498]
WARNING:minestudio.simulator.minerl.env.malmo.instance.128321:[15:29:52] [Render thread/WARN]: Ambiguity between arguments [teleport, targets, location] and [teleport, targets, destination] with inputs: [0.1 -0.5 .9, 0 0 0]
WARNING:minestudio.simulator.minerl.env.malmo.instance.128321:[15:30:03] [Render thread/WARN]: Failed to load texture: minecraft:optifine/ctm/default/empty.png
ERROR:minestudio.simulator.minerl.env.malmo.instance.128321:java.io.FileNotFoundException: minecraft:optifine/ctm/default/empty.png

Run the inference cell below to choose an epoch model and visualize its exact learned policy.

The newest epoch is selected by default. Choose another saved model, then click Visualize selected policy.

VBox(children=(Dropdown(description='Epoch model:', layout=Layout(width='95%'), options=(('Epoch 2/2 · 1,024 s…

{
  "score": 0.5903348049521453,
  "steps": 1024,
  "epochs": 2,
  "checkpoints": 2,
  "preview": "learned-policy.gif"
}

{
  "score": 0.5903348049521453,
  "steps": 1024,
  "epochs": 2,
  "checkpoints": 2,
  "preview": "learned-policy.gif"
}