Slime GRPO

ModelJudgedTierSelf-bestRoundsHoursAPI cost
Claude Sonnet 5.52.40×N12.45×1010.0 h$58.3trace →
Claude Opus 5.52.35×N12.48×710.0 h$87.7trace →
GPT-6.2 Astra2.15×N11.98×1210.0 h$59.1trace →
MiniMax M3.1 Flash1.91×N11.90×1710.0 h$5.73trace →
DeepSeek V4.1 Flash-10171.88×N11.94×1010.0 h$1.41trace →
Claude Haiku 5.51.85×N11.90×2310.1 h$16.8trace →
GPT-6.2 Sol1.85×N11.73×1210.0 h$13.9trace →
GLM-5.31.79×N11.72×1410.0 h$61.8trace →
Kimi k3.1-pro1.35×N11.25×910.0 h$14.7trace →
GPT-6 Astrasuspected nerf1.15×N11.13×2010.0 h$217trace →
ModelJudgedCandidate, three windows (s)Rollout sideTrain sideRollout host-side fixesSkip zero-coefficient reference forwardTrain-side main levers
Claude Sonnet 5.52.40×76.4 / 77.5 / 76.02.54×1.89×✓ sampler desync, batched top-k + compact + shared-memory transfer, fast JSON, round-robin routing✓✓ no recompute, NVLS off, 16 CPU threads, GDN split Ulysses-style (forward bitwise identical), comm groups kept, GC freeze, pinned master weights; train window 67.3 → 35.7 s (1.89×), second on train side
It also had a "skip backward based on advantage values" item that cut the train window further to ~18 s (train side 3.33×); banned by the task statement, not counted in the score or in this column; see top of page
Claude Opus 5.52.35×79.2 / 78.0 / 77.72.42×2.15×✓ sampler desync, only compact base64 top-k returned (post-rollout tail 28 → 2 s), top-k copied back once per step✓✓ no recompute, 16 CPU threads, free memory queried once before comm, pinned fp32 master weights, training process and rollout engine both resident (no more per-step sleep / wake), GDN backward split by token across 4 GPUs (forward unchanged); train window 66.2 → 30.9 s (2.15×), highest train side overall
Round 7 also measured a fused sampling kernel + round-robin routing (rollout 2.51×) and weight sync directly from on-GPU params (train 2.17×); its own full loop ~74 s/step; quick check didn't finish before time ran out, not committed
MiniMax M3.1 Flash1.91×99.0 / 95.8 / 94.32.07×1.60×✓ sampler desync, flat base64 top-k, single D2H, request intake polled every 8 steps (1.95 → 2.07×)✓partial: NCCL groups kept (train 1.22×), no recompute (→ 1.49×), params resident on GPU, micro-batch 8192 tokens (→ 1.60×)
DeepSeek V4.1 Flash-10171.88×101.8 / 97.0 / 95.62.10×1.60×✓ redundant nested top-k removed + orjson (rollout 111 → 82 s), host-side top-p check (→ 52 s)✓partial: no recompute, training model resident, logprob chunk 1024 → 4096
Claude Haiku 5.51.85×98.3 / 100.2 / 99.62.15×1.50×✓ sampler desync, engine sends binary top-k only, round-robin✓partial: no recompute, 24 training CPU threads, GC freeze + param buffers resident (1.40 → 1.50×)
GLM-5.31.79×105.3 / 100.7 / 102.02.19×~1.29×✓ host-side top-p flag, flat top-k✗ reference forward still there (~10 s/step); tried merging its micro-batches into larger ones, net slowerpartial: training CPU threads, NCCL kept across sleep/wake; recompute only changed to stride-1, not turned off
Kimi k3.1-pro1.35×138.2 / 133.1 / 137.11.32×1.35×partial: only sampler sync removed and top-k row copies merged; top-k JSON serialization untouched✗partial: training resident, no recompute; pairing micro-batches measured 1.44×, blocked by direction cosine 0.931 < 0.955
GPT-6 Astrasuspected nerf1.15×162.1 / 156.1 / 161.5≈0.98×1.48×✗ rollout untouched (measured 0.98–0.99× all three times)✓partial: training process group kept, selective recompute committed only in round 19 (1.20 → 1.48×)
↑ Leaderboard