YOCO

ModelJudgedTierSelf-bestRoundsHoursAPI cost
Claude Opus 5.52.68×N12.68×1810.0 h$57.6trace →
Claude Sonnet 5.52.26×N1—1810.0 h$43.8trace →
GPT-6.2 Astra2.01×N12.02×4010.0 h$81.4trace →
GPT-6.2 Sol1.86×N11.88×4510.0 h$20.8trace →
GPT-6 Astrasuspected nerf1.79×N11.79×2510.0 h$99.7trace →
Claude Haiku 5.51.70×N11.71×2310.0 h$27.4trace →
GLM-5.31.60×N11.60×1610.0 h$37.2trace →
MiniMax M3.1 Flash1.59×N11.59×2910.0 h$6.03trace →
DeepSeek V4.1 Flash-10171.55×N11.68×747.9 h$3.16trace →
Kimi k3.1-pro1.37×N11.38×3310.0 h$33.5trace →
ModelJudgedCross-GPU exchange via CUDA IPC + copy engine (no SMs)Forward / backward CUDA graphNewer cuBLASRetention / attention kernel swapOther main levers
Claude Opus 5.52.68×✓ Adam kernel writes bf16 weights straight to both GPUs, no more weight exchange; grads pushed via copy engine; this step 2.18× → 2.35×✓ YOCO_FAST_GRAPH, forward + loss + backward + grad reduction in one graph, 1.52× → 1.64×✓ 12.8.4.1, 2.11× → 2.18×✓ YOCO_FAST_RET custom Triton retention 1.985× → 2.08×; YOCO_FAST_ATTN switched to cuDNN SDPA 2.57× → 2.63×; retention kernel moved to Triton 3.8, 2.64× → 2.68×round 2 replaced the whole optimizer path (flat buffers, custom bitwise-equivalent CUDA Adam, custom grad reduction replacing DDP and OSS, straight to 1.52×); merged q/k/v/g and fc1/gate GEMMs; dW written directly into the flat grad buffer; fused cross-entropy; CUDA RMSNorm; per-shape GEMM algorithm table
Claude Sonnet 5.52.26×✓ YOCOFAST_PULL + Triton Adam TOPT, 1.84× → 1.96×; grads YOCOFAST_PULLAR, 1.96× → 2.05×✓ YOCOFAST_GRAPH, one graph per decoder layer (26), 1.52× → 1.60×✓ 12.6.4.1, 1.735× → 1.783×✓ YOCOFAST_GLA (ported simple_gla chunked kernel) 2.09× → 2.14×; YOCOFAST_CUDNN_ATTN 2.05× → 2.09×LAZY_GRAD (fp32 master grads materialized on demand) 2.14× → 2.25×; embedding grads reduce only used rows; fused cross-entropy; hand-written RMSNorm; per-step host sync removed
GPT-6 Astrasuspected nerf1.79×✓ YOCO_OSS_IPC_DMA, 1.52× → 1.65×; grads YOCO_DDP_IPC_REDUCE → 1.68× (both full check)✓ YOCO_GRAPH_DECODER, per layer, 1.23× → 1.31×✓ cuBLASLt from 12.8.4.1, loaded explicitly, 1.72× → 1.78×✗ retention untouched; started compiling FlashAttention-3 in round 26, log ends thereApex RMSNorm and its fused affine variant; fused cross-entropy; Adam overlapped with broadcast chunks; custom CUDA Adam + bf16 writeback
Claude Haiku 5.51.70×✗ still NCCL: broadcast in both directions at once, 8-stage pipeline (1.31× → 1.38×), grads switched to per-parameter all-reduce✗ not in the log; instead torch.compile fuses pointwise ops of FFN, projections, retention into blocks (1.585× → 1.637×)✗ no library swap; zero-bias addmm makes the bundled library take cuBLASLt, 1.577× → 1.585×partial: only retention pointwise ops and readout compiled; recurrence kernel and attention not swappedper-step logging sync removed (~11 ms); compiled RMSNorm 1.38× → 1.51×; sparse embedding grad reduction 1.637× → 1.662×; fused cross-entropy
GLM-5.31.60×✗ still NCCL (all_to_all + 8-stage pipeline); exchanged payload switched to bf16 (BF16X, 1.30× → 1.38×)✓ but forward only, and only in round 21: 1.51× → 1.60×; earlier judged "the judge's criterion hook doesn't allow it"✗✗fp32 master grads materialized only for this rank's shard; Triton kernels launched directly after caching (1.38× → 1.45×); bitwise-identical Triton RMSNorm; DDP reduction switched to AVG
MiniMax M3.1 Flash1.59×✗ no exchange: YOCO_OPT_REPLICATED_ADAM has both ranks run Adam on all params (1.00× → 1.11×); grad all-reduce still NCCL✗ judged infeasible, no measurement in the log✗✗ only rotary and recurrence kernels switched to direct launch (1.41× → 1.45×)Triton fused "cast + scale" 1.18× → 1.30×; Triton Adam also writing bf16 weights 1.34× → 1.40×; Triton RMSNorm with two-level reduction 1.48× → 1.53×
DeepSeek V4.1 Flash-10171.55×✗ no exchange: YOCO_NO_OSS replicates Adam state; grad all-reduce still NCCLtried, failed (whole-model capture in rounds 61–62 hit a PyTorch 2.3.1 allocator assertion; each component captures fine on its own)✗✗ compiling the whole mixer was slower (4.22–4.46 ms vs 3.95 ms per layer)compiled RMSNorm with inductor autotuning (→ 1.29×, after tuning 1.47× → 1.51×); Triton fused cast 1.29× → 1.35×; Triton Adam 1.35× → 1.47×; fused cross-entropy 1.53× → 1.56×
Kimi k3.1-pro1.37×✗ fp32 broadcast kept as-is, only chunked and pipelined with Adam (1.24× → 1.29×)✗ only listed as a next step at the end✗ tried cuBLASLt workspace, no gain✗ instead merged five projections into one GEMM (1.196× → 1.213×)Apex RMSNorm 1.09× → 1.16×; master-grad cast batched per DDP bucket 1.31× → 1.37×; fused cross-entropy; DDP bucket views (round 30)
↑ Leaderboard