YOCO
§ 02Models
| Model | Judged | Tier | Self-best | Rounds | Hours | API cost | |
|---|---|---|---|---|---|---|---|
| Claude Opus 5.5 | 2.68× | N1 | 2.68× | 18 | 10.0 h | $57.6 | trace → |
| Claude Sonnet 5.5 | 2.26× | N1 | — | 18 | 10.0 h | $43.8 | trace → |
| GPT-6.2 Astra | 2.01× | N1 | 2.02× | 40 | 10.0 h | $81.4 | trace → |
| GPT-6.2 Sol | 1.86× | N1 | 1.88× | 45 | 10.0 h | $20.8 | trace → |
| GPT-6 Astrasuspected nerf | 1.79× | N1 | 1.79× | 25 | 10.0 h | $99.7 | trace → |
| Claude Haiku 5.5 | 1.70× | N1 | 1.71× | 23 | 10.0 h | $27.4 | trace → |
| GLM-5.3 | 1.60× | N1 | 1.60× | 16 | 10.0 h | $37.2 | trace → |
| MiniMax M3.1 Flash | 1.59× | N1 | 1.59× | 29 | 10.0 h | $6.03 | trace → |
| DeepSeek V4.1 Flash-1017 | 1.55× | N1 | 1.68× | 74 | 7.9 h | $3.16 | trace → |
| Kimi k3.1-pro | 1.37× | N1 | 1.38× | 33 | 10.0 h | $33.5 | trace → |
§ 03Key levers
| Model | Judged | Cross-GPU exchange via CUDA IPC + copy engine (no SMs) | Forward / backward CUDA graph | Newer cuBLAS | Retention / attention kernel swap | Other main levers |
|---|---|---|---|---|---|---|
| Claude Opus 5.5 | 2.68× | ✓ Adam kernel writes bf16 weights straight to both GPUs, no more weight exchange; grads pushed via copy engine; this step 2.18× → 2.35× | ✓ YOCO_FAST_GRAPH, forward + loss + backward + grad reduction in one graph, 1.52× → 1.64× | ✓ 12.8.4.1, 2.11× → 2.18× | ✓ YOCO_FAST_RET custom Triton retention 1.985× → 2.08×; YOCO_FAST_ATTN switched to cuDNN SDPA 2.57× → 2.63×; retention kernel moved to Triton 3.8, 2.64× → 2.68× | round 2 replaced the whole optimizer path (flat buffers, custom bitwise-equivalent CUDA Adam, custom grad reduction replacing DDP and OSS, straight to 1.52×); merged q/k/v/g and fc1/gate GEMMs; dW written directly into the flat grad buffer; fused cross-entropy; CUDA RMSNorm; per-shape GEMM algorithm table |
| Claude Sonnet 5.5 | 2.26× | ✓ YOCOFAST_PULL + Triton Adam TOPT, 1.84× → 1.96×; grads YOCOFAST_PULLAR, 1.96× → 2.05× | ✓ YOCOFAST_GRAPH, one graph per decoder layer (26), 1.52× → 1.60× | ✓ 12.6.4.1, 1.735× → 1.783× | ✓ YOCOFAST_GLA (ported simple_gla chunked kernel) 2.09× → 2.14×; YOCOFAST_CUDNN_ATTN 2.05× → 2.09× | LAZY_GRAD (fp32 master grads materialized on demand) 2.14× → 2.25×; embedding grads reduce only used rows; fused cross-entropy; hand-written RMSNorm; per-step host sync removed |
| GPT-6 Astrasuspected nerf | 1.79× | ✓ YOCO_OSS_IPC_DMA, 1.52× → 1.65×; grads YOCO_DDP_IPC_REDUCE → 1.68× (both full check) | ✓ YOCO_GRAPH_DECODER, per layer, 1.23× → 1.31× | ✓ cuBLASLt from 12.8.4.1, loaded explicitly, 1.72× → 1.78× | ✗ retention untouched; started compiling FlashAttention-3 in round 26, log ends there | Apex RMSNorm and its fused affine variant; fused cross-entropy; Adam overlapped with broadcast chunks; custom CUDA Adam + bf16 writeback |
| Claude Haiku 5.5 | 1.70× | ✗ still NCCL: broadcast in both directions at once, 8-stage pipeline (1.31× → 1.38×), grads switched to per-parameter all-reduce | ✗ not in the log; instead torch.compile fuses pointwise ops of FFN, projections, retention into blocks (1.585× → 1.637×) | ✗ no library swap; zero-bias addmm makes the bundled library take cuBLASLt, 1.577× → 1.585× | partial: only retention pointwise ops and readout compiled; recurrence kernel and attention not swapped | per-step logging sync removed (~11 ms); compiled RMSNorm 1.38× → 1.51×; sparse embedding grad reduction 1.637× → 1.662×; fused cross-entropy |
| GLM-5.3 | 1.60× | ✗ still NCCL (all_to_all + 8-stage pipeline); exchanged payload switched to bf16 (BF16X, 1.30× → 1.38×) | ✓ but forward only, and only in round 21: 1.51× → 1.60×; earlier judged "the judge's criterion hook doesn't allow it" | ✗ | ✗ | fp32 master grads materialized only for this rank's shard; Triton kernels launched directly after caching (1.38× → 1.45×); bitwise-identical Triton RMSNorm; DDP reduction switched to AVG |
| MiniMax M3.1 Flash | 1.59× | ✗ no exchange: YOCO_OPT_REPLICATED_ADAM has both ranks run Adam on all params (1.00× → 1.11×); grad all-reduce still NCCL | ✗ judged infeasible, no measurement in the log | ✗ | ✗ only rotary and recurrence kernels switched to direct launch (1.41× → 1.45×) | Triton fused "cast + scale" 1.18× → 1.30×; Triton Adam also writing bf16 weights 1.34× → 1.40×; Triton RMSNorm with two-level reduction 1.48× → 1.53× |
| DeepSeek V4.1 Flash-1017 | 1.55× | ✗ no exchange: YOCO_NO_OSS replicates Adam state; grad all-reduce still NCCL | tried, failed (whole-model capture in rounds 61–62 hit a PyTorch 2.3.1 allocator assertion; each component captures fine on its own) | ✗ | ✗ compiling the whole mixer was slower (4.22–4.46 ms vs 3.95 ms per layer) | compiled RMSNorm with inductor autotuning (→ 1.29×, after tuning 1.47× → 1.51×); Triton fused cast 1.29× → 1.35×; Triton Adam 1.35× → 1.47×; fused cross-entropy 1.53× → 1.56× |
| Kimi k3.1-pro | 1.37× | ✗ fp32 broadcast kept as-is, only chunked and pipelined with Adam (1.24× → 1.29×) | ✗ only listed as a next step at the end | ✗ tried cuBLASLt workspace, no gain | ✗ instead merged five projections into one GEMM (1.196× → 1.213×) | Apex RMSNorm 1.09× → 1.16×; master-grad cast batched per DDP bucket 1.31× → 1.37×; fused cross-entropy; DDP bucket views (round 30) |