把正确答案变成奖励

CPU · Python 3.12.13 · 1.66 s · 2026-10-22 08:53:43 UTC+8

本次运行结果

verifier_tests	6
token_surrogate	0.8500000238418579
sequence_surrogate	0.75

Rewards:
 tensor([[0., 0., 1., 1.],
        [0., 0., 0., 0.]])
Advantages:
 tensor([[-0.9998, -0.9998,  0.9998,  0.9998],
        [ 0.0000,  0.0000,  0.0000,  0.0000]])

'答案: 42' → 1.0
'答案: 35' → 0.0
'答案: 42 答案: 35' → 0.0
'答案: -3' → 1.0
'答案: 42.5' → 0.0
'42' → 0.0

{'token_surrogate': 0.8500000238418579, 'sequence_surrogate': 0.75}

Mean selected-token log-probability change: [-0.13198477029800415, -0.08652293682098389, 0.11663806438446045, 0.07964777946472168]

Optional exercise: fill answer and enable RUN_EXERCISE.

metrics.json

{
  "verifier_tests": 6,
  "token_surrogate": 0.8500000238418579,
  "sequence_surrogate": 0.75,
  "logprob_change": [
    -0.13198477029800415,
    -0.08652293682098389,
    0.11663806438446045,
    0.07964777946472168
  ]
}