Slime OPD

ModelJudgedTierSelf-bestRoundsHoursAPI cost
Claude Sonnet 5.51.76×N11.68×610.0 h$75.2trace →
Claude Opus 5.51.64×N11.63×1310.0 h$86.8trace →
Kimi k3.1-pro1.59×N11.57×2310.0 h$22.4trace →
GPT-6.2 Astra1.46×N11.46×1310.0 h$54.3trace →
GLM-5.31.46×N11.58×1310.0 h$80.6trace →
DeepSeek V4.1 Flash-10171.43×N11.43×1310.0 h$2.06trace →
MiniMax M3.1 Flash1.40×N11.50×2610.0 h$4.92trace →
GPT-6.2 Sol1.37×N11.31×1810.0 h$16.9trace →
GPT-6 Astrasuspected nerf1.30×N11.38×2210.0 h$245trace →
Claude Haiku 5.51.23×N11.25×410.0 h$12.8trace →
ModelJudgedCandidate, three windows (s)Rollout sideTeacher sideTrain sidePer-step sampler syncTrain-side main leversTeacher-side changes
Claude Sonnet 5.51.76×85.8 / 86.9 / 89.8~1.21×~1.09×~1.99×✓✓ skip zero-coefficient reference forward (−10.9 s), fused old-policy logprob, no recompute, 24 CPU threads, pinned master weights; GDN split by head with forward bitwise identical; only fp32 grad buffers released during sleep; pipelined weight sync; FLA warmupchunking only: teacher chunk 16384 + single-batch prefill 32768 + radix cache off
Claude Opus 5.51.64×92.3 / 96.8 / 92.11.34×1.20×2.23×✓ (round 2, rollout 1.27×)✓ skip zero-coefficient reference forward, advantages computed directly from training-forward logprobs (saves one actor forward pass), GDN split in Ulysses layout with forward bitwise identical, comm groups kept, 16 CPU threads, only grad buffers swapped out during sleep with weights restored on wake, no recompute but separate checkpointing for GDN modules; train side 2.23×, highest overall
Rounds 11–13 further removed the memory probe before each all_reduce and one sync in TE's CP backward, pinned fp32 master weights, train side measured 2.48×; but step-1 grads in the record run were anomalous again, still investigating when time ran out, not committed
✓ teacher input logprobs vocab-parallel, prefill chunk 8192, symmetric-memory all-reduce, teacher radix cache off (GDN prefill switched to CP-parallel kernel); teacher side 1.20×, highest overall
Kimi k3.1-pro1.59×96.9 / 96.8 / 96.91.28×1.06×2.11×✓ only working by round 23 (rollout 1.28×, its own estimate of overall up to 1.56×), changed only in a separate code tree, not merged into the commit; score recorded from the version merged on its behalf✓ skip reference forward, no full recompute, training resident, fused actor logprob, CPU optimizer 24 threads (train side 2.11×)chunking only: teacher chunk 16384 (teacher side 1.06×)
GLM-5.31.46×105.0 / 105.3 / 106.81.32×1.16×1.71×✓ (round 13)partial: skip reference forward, fused advantage computation, no recompute, CPU Adam 8 threads✓ teacher computes input logprobs per shard (−1.4 s), teacher chunk 16384
DeepSeek V4.1 Flash-10171.43×107.1 / 108.1 / 106.61.29×1.06×1.58×✓partial: skip reference forward, no recompute, training residentchunking only: teacher chunk 16384 + single-batch prefill 32768; student radix cache off
MiniMax M3.1 Flash1.40×108.6 / 109.3 / 111.81.27×—1.55×✓ (round 8)partial: skip reference forward, reuse logprob, two samples per micro-batch, process groups kept, NCCL_PROTO=Simple; full recompute kept (selective recompute peaks at 118 GiB, OOM)chunking only: teacher chunk 32768
GPT-6 Astrasuspected nerf1.30×117.6 / 119.4 / 116.8≈1.01×≈1.03×~1.4×✗partial: skip reference forward, fused OPD logprob, skip zero-coefficient entropy, finally turned on "no activation recompute" (validated only on the train side alone; in the full loop teacher and training share GPUs, not enough memory, OOM)✗ teacher side untouched (≈1.03×)
Claude Haiku 5.51.23×122.4 / 131.7 / 122.3≈1.0×≈1.0×1.71×✗ rollout untouchedpartial: reference forward removed and OPD logprob reused, no recompute, 24 training CPU threads, logprob chunking; only 3 rounds in 10 hours, 5+ hours of round 3 spent on FlashQLA (direction cosine 0.9574, right at the 0.9566 floor, reverted) and GDN split by head (grads didn't match)✗ teacher side untouched (≈1.0×)
↑ Leaderboard