Claude Haiku 5.5 1.23×
Harness: Claude Code · switches: SLIME_OPD_NO_RECOMPUTE=1 SLIME_OPD_TRAIN_CPU_THREADS=24 SLIME_OPD_LOG_PROBS_CHUNK=8192
- Judged
- 1.23×
- Tier
- N1
- Self-best
- 1.25×
- Rounds
- 4
- Hours
- 10.0
- API cost
- $12.8
- Input tokens · 99.6% cached
- 212 M
- Output tokens
- 0.6 M
- Requests
- 434
§ 02Key levers
- Candidate, three windows (s)
- 122.4 / 131.7 / 122.3
- Rollout side
- ≈1.0×
- Teacher side
- ≈1.0×
- Train side
- 1.71×
- Per-step sampler sync
- ✗ rollout untouched
- Train-side main levers
- partial: reference forward removed and OPD logprob reused, no recompute, 24 training CPU threads, logprob chunking; only 3 rounds in 10 hours, 5+ hours of round 3 spent on FlashQLA (direction cosine 0.9574, right at the 0.9566 floor, reverted) and GDN split by head (grads didn't match)
- Teacher-side changes
- ✗ teacher side untouched (≈1.0×)
§ ARaw trace
Raw work log12 sections
0.1 hRound 1 (started 2026-11-15 00:51 UTC)
### Baseline facts (from bootstrap logs, unchanged reference) - Frozen reference timing (task/timing_ref.json): rollout 75.9 s (of which teacher tail 16.5 s), train 73.3 s, teacher replay 18.2 s, full loop 154.3 s. Fractions: rollout 0.4919, train 0.4751, teacher 0.1072. - Full-loop step breakdown from bootstrap full_ref_a train.log (per-step timer means over 5 steps, rank 0): - actor_train 55.8 s (fwd+recompute+bwd+optimizer over 64 microbatches) - ref_log_probs 12.5 s (weight switch to ref + full no-grad forward with reference weights; its result only feeds the KL-loss term whose coefficient is 0.00) - log_probs 10.0 s (weight switch back + full no-grad forward of the actor, needed for OPD advantages) - sleep 3.6 s, update_weights 2.2 s, wake_up 1.2 s - Fixture rollouts: ~90% of responses hit the 8192 cap, ~515k response tokens per rollout. - GPU bf16 matmul peak measured here: ~787 TFLOPS (Hopper, 132 SMs). Train step is far from GEMM-bound: ~9.5e15 FLOP per GPU per step vs 73 s. - Judge run durations (bootstrap prepare.log): time --side train ~16 min, --side rollout ~9 min, --side teacher ~3 min; quick check ~37 min; full check ~1.5 h.
0.1 hPlan for round 1
- Profile the unchanged reference train side (judge profile --side train, running detached, output /tmp/r1/profile_train).
- Candidate cuts from code reading: (a) skip the reference forward + weight switches when kl-loss coefficient and kl coefficient are both 0 (dead computation: the KL term is multiplied by 0); (b) reuse training-forward log-probs instead of a separate no-grad forward for OPD; (c) entropy computed although entropy_coef is 0.0.1 hRound 1 changes (in progress, 01:03 UTC)
- Switch file:/submission/switches.json={"environment": {"SLIME_OPD_NO_RECOMPUTE": "1"}}. - (A) Skip the reference-model forward and its two weight switches when both KL coefficients are zero (actor.py::_needs_ref_log_probs). The reference log-probs only fed theuse_kl_lossterm withkl_loss_coef=0.00(and thekl_coefadvantage term, which is 0). The KL term and itskl_lossmetric are dropped inloss.pyunder the same condition (kl_lossis not read by the judge). No switch: unconditional dead-work removal. - (B) Fused OPD log-probs (actor.pyfuse_opd_logprobs,loss.pyfused block): for the sglang teacher, the student's log-probs come from the policy loss's own training forward instead of a separate no-grad forward; the reverse-KL penalty is applied per microbatch from detached log-probs (ratio = exp(0) = 1 exactly). Same design as upstream's existing reuse path for GRPO. No switch. - (C)SLIME_OPD_NO_RECOMPUTE=1(megatron_utils/arguments.py): full activation recompute off (memory strategy; the backward reads saved activations; memory measured headroom: ~29.5 GB Megatron resident per GPU, teacher 35 GB reserved). - Note: an earlier edit toslime/utils/arguments.pywas reverted: files outside all judge sides would make every check judge all three sides.
0.1 hRound 1 results (01:26–01:40 UTC)
- Train profile of the unchanged reference (bench.py profile --side train, 00:51): stopped at 01:26 — each rank's torch trace grew to ~870 MB gz and was still exporting after 35 min. Lesson: the judge's train profile is too heavy for a round; use the slime timers (Timer ... end (elapsed)) and the judge'stimeinstead. Partial traces deleted. Queued rollout/teacher profiles were cancelled for the same reason (replan from timers andtimeruns). - Reference step breakdown (bootstrap full_ref_a timers, 5 steps): actor_train 55.8 s, ref_log_probs 12.5 s, log_probs 10.0 s, sleep 3.6 s, update_weights 2.2 s, wake_up 1.2 s. -time --side trainon changes A+B+C (switches:SLIME_OPD_NO_RECOMPUTE=1), /tmp/r1/time_train_a/time.json: train side speedup 1.5745 (min window), projected step speedup 1.2097. Train-mode actor_train ~38.5–39 s per step; ref/log_probs timers gone; no tracebacks. GPU cost 0.827 GPU-hours. - Reference record is not bitwise repeatable (train_ref.jsonrepeatable=False): the judge decides the training side statistically (N1), so ulp-level reorderings are admissible if the statistical gates pass. - Next:check --quick(train side only) at /tmp/r1/quick_a, then commit. -check --quick(train side, /tmp/r1/quick_a/check.json, 01:50): PASS, tier N1. Gates: update norm d=-1.2e-4 (tol 1.2e-3), Adam second-moment norm d=1.3e-4 (tol 4.4e-4), mean loss d=-1.7e-5 (tol 3.75e-5), first grad norm d=-2.2e-4 (tol 1.7e-3), update-direction cosine 0.9935 (floor 0.9566), train-inference mismatch ratio 1.0. - Commits inslime/:ad765082(A+B: dead reference forward removed; training-forward log-probs reused for OPD) anda212f8ff(C: SLIME_OPD_NO_RECOMPUTE switch).switches.json={"environment": {"SLIME_OPD_NO_RECOMPUTE": "1"}}. - Round 1 result: train side 1.5745 (min window), projected step 1.2097. Full-loop speedup not yet measured (final eval). - Round 1 overran its 30-min pace (the train profile cost ~35 min); the round's conclusion is logged here.
1.0 hRound 2 plan (started 01:55 UTC)
- Microbatch packing (N1, grouping-only):SLIME_OPD_MAX_TOKENS_PER_GPU=9216so two ~8.3k-token samples share one microbatch (bin cap = max_tokens_per_gpu x cp_size = 18432). Currently every sample is alone in its microbatch (64 microbatches). Memory check: diagnostic peak-allocated line added to the train log (train-mode time run has no teacher; full mode adds ~35 GB teacher, so the train-mode peak must stay below ~105 GB). - Next candidates: teacherOPD_TEACHER_SERVER_ARGS="--chunked-prefill-size 8192"(teacher side, the judge's own rebatching envelope); optimizer offload fraction (N1); sleep/wake tuning (small).
1.0 hFinding (round 2): the Qwen3.5 linear-attention layers run replicated
-slime_plugins/models/qwen3_5.pybuilds the attention of every layer onHuggingfaceAttention; the default--qwen-gdn-implementationisreplicated(the recipe does not set it). The replicated GDN module keeps full weights on every rank: the sequence hidden states are all-gathered over TP (SP) and CP, so the 24 linear-attention layers run the whole 8k-token sample on each of the 4 GPUs (~4x the ideal GDN work per GPU). That explains the low train MFU (~26%: ~72 GFLOP/token x 8.3k tokens per microbatch / 0.7 s). - The alternative--qwen-gdn-implementation megatron(qwen3_5_native.py, Megatron's TP/CP GatedDeltaNet) has different parameter names and sharded shapes (in_projcombined, partition_sections). The judge's record gate compares the names and shapes of every bf16 parameter, fp32 master weight and Adam moment exactly, so this path cannot pass the structure gate without a new checkpoint layout. Not pursued. - Possible in-layout lever (not done): shard the replicated GDN computation over TP/CP while keeping the full parameter shapes (needs gradient all-reduce for replicated weights). Large change; keep as the expensive option.
1.0 hRound 2: microbatch packing (SLIME_OPD_MAX_TOKENS_PER_GPU=9216) — rejected on memory
- Time run /tmp/r2/time_mb (train side, NO_RECOMPUTE + 2 samples per microbatch): steady-state actor_train 34.8–35.3 s (vs ~38.5 s single-sample), but the train-mode peak was allocated 100.5 GB, reserved ~134.5–135 GB per GPU. The full loop adds the teacher's resident 35 GB, so this configuration would exceed the 140 GB card. Not usable. Not committed; the switch code stays in the working tree, default off, and is not in switches.json. - Next: time the validated configuration (NO_RECOMPUTE only) with the peak-memory diagnostic, to bound the full-loop memory (reserved must stay below ~105 GB with the teacher resident). - Expensive lever, designed but NOT implemented (risk: gradient semantics under TP-duplicated losses and CP chunks): shard the replicated GDN over the 4 TP x CP ranks by heads with the full parameter shapes kept: each rank slices its rows of in_proj_qkv / in_proj_z / in_proj_a / in_proj_b, its conv1d channels, dt_bias and A_log, and its columns of out_proj; the output is an autograd all-reduce (4-way); the input gather backward must sum over TP (reduce-scatter, tensor_parallel_output_grad=True); parameter gradients are partial per rank and need one all-reduce per step before the optimizer (6.4 GB per step, ~60 ms). Estimated saving ~10 s of the 38.5 s actor step (replicated GDN projections are ~80 TFLOP per GPU per microbatch with recompute off). - Repeat of the validated config (NO_RECOMPUTE only), /tmp/r2/time_base2/time.json: train side 1.592, windows [45.69, 45.74, 45.23] s, projected step 1.2146. Consistent with the first run (1.5745). The earlier 1.09 run (/tmp/r2/time_base_diag) had one 60 s actor step (window range 47%, invalid by the 10% rule) with no code difference; treat it as host noise. Peak memory of the validated config: allocated 64.0 GB, reserved 67.6 GB (train mode) => about 103 GB with the 35 GB teacher resident, inside the 139.8 GB card.
1.0 hRound 2: head-sharded linear attention (switch SLIME_OPD_GDN_HEAD_SHARD=1) — implemented, under validation
- Implementation:slime_plugins/models/qwen3_5.py_forward_head_sharded: each TP x CP rank computes its key/value heads (4 key + 8 value heads of 16/32) over the whole sequence, with row/column slices of the full parameters (shapes unchanged); output projection partial sums all-reduced over TP x CP (_AllReduceSumCpBackward: backward sums over CP).hf_attention.py: the SP gather backward becomes a reduce-scatter.model.pyfinalize:finalize_head_shard_gradssums the partial parameter grads over TP only, before Megatron's DP x CP reduction (over TP x CP it would double count). - Time (train side, timing run 520750 with the earlier TP x CP finalize; same ops, slightly more comm): actor_train steps 37.6/37.0/34.4/35.0 s (validated config ~38.5 s); peak memory allocated 54.1 GB, reserved 57.8 GB (was 64.0/67.6). Judged train side 1.584 (windows 45.1/46.0/42.3 s), projected step 1.212 (validated: 1.592 / 1.215). The judged metric did not move (per-step overhead outside actor_train is noisy, 7–9 s); gain on actor_train ~3 s. - First quick check of the head-sharded path (/tmp/r2/quick_gdn): INCORRECT — loss −11.1 vs reference 0.052 from step 0, first grad norm 3.7e5, update cosine 0.001. Cause: the flaShortConvolutionweight is[channels, 1, kernel]and its forward rearranges it to[channels, kernel]; the sharded path passed the 3-D slice straight tocausal_conv1d. Fixed withconv1d.weight.squeeze(1). Re-runningcheck --quick(/tmp/r2/quick_gdn2). - Sleep/wake: the first sleep took 1.5 s and wake 0.8 s; later sleeps 4–5 s (after the NCCL world destroy), suspected cost isclear_memory(clear_host_memory=True). Added switchSLIME_OPD_SLEEP_KEEP_HOST_CACHE=1(keeps pinned host cache), to time.
1.0 hRound 2 outcomes (03:30 UTC)
- Rejected: microbatch packing (memory 135 GB reserved). - Rejected: teacher--chunked-prefill-size 8192(teacher side 1.0227, projected step 1.0024: not worth a teacher check). - Rejected (no gain): sleep host-cache flush skip (SLIME_OPD_SLEEP_KEEP_HOST_CACHE): sleep still 4.3–5.6 s. Reverted. - Parked on branchwip-gdn-head-shard(commit 566f537f, not validated): head-sharded linear attention. The forward is now correct (loss matches) but the update direction fails the gate (cosine 0.949 < 0.9566, grad norm just outside tolerance). Gain was ~3 s of the actor step. Stopped. - Submission state:optata212f8ff,switches.json={"environment": {"SLIME_OPD_NO_RECOMPUTE": "1"}}. - Full-loop projection: the reference full loop is 154.3 s (rollout 76 s + train-side ~78 s). The full-loop actor step is slower than in train mode (reference 55.8 s full vs ~44 s train mode), so the projected full-loop speedup is ~1.2 (rollout 76 + actor ~46 + sleep/wake ~3 + weight sync ~2 + misc). The final evaluation measures the real value.
4.7 hRound 3 (started 05:31 UTC)
- Lever: linear-attention kernel library. The replicated Qwen3.5 GDN layers can run FlashQLA's TileLang kernels (--qwen-gdn-backend flashqla, slime_plugins/models/qwen_gdn_backend.py). FlashQLA 0.1.3 (PyPI, pure-Python wheel, TileLang 0.1.12 already in the image) installed into /opt/candidate-env with --no-deps. Its default scale (K^-0.5) and arguments match fla's call. Switch:SLIME_OPD_GDN_BACKEND=flashqla(megatron_utils/arguments.py), with NO_RECOMPUTE. - Time runtime --side train(switches /tmp/r3/switches_fqla.json) started; result below when complete. -time --side trainwith FlashQLA (/tmp/r3/time_fqla/time.json): train side 1.629, windows 44.18/44.70/43.40 s, projected step 1.2247 (validated fla config: 1.592 / 1.2146 on repeat). Actor steps 36.2–37.7 s (fla 38–39 s). First-use compile happened in the warm-up step (220 s), first_use 0 in windows. Gain ~1 s per step, numerics not yet checked. - Quick check of the FlashQLA config started (/tmp/r3/quick_fqla). - Quick check of the FlashQLA config (/tmp/r3/quick_fqla/check.json): PASS at N1 but at the edge: update-direction cosine 0.9574 (floor 0.9566, the fla config gives 0.993–0.995); first-gradient-norm d = +0.0016 (tolerance 0.00165). The fla config's runs vary by ~2.5e-4 on that gate. FlashQLA changes the update direction systematically. No-go: a final-evaluation fail would score 0 for a ~1 s (about 1%) gain. Backend switch removed from the tree. - Diagnostic kept:SLIME_OPD_DIAG_PROFILE_ROLLOUT=<step>(off by default) prints a per-kernel CUDA table for one train step (localization only; not in the submission switches). - Round 3 profile (diagnostic record run, one train step, rank 0;/tmp/r3/diag_profile.txt): CUDA time per step 33.5 s: GEMMs ~7.8 s (+ LayerNormLinear/linear backward), NCCL ~4.4 s (SP all-gathers 3.0 s, reduce-scatter 1.3 s, CP send/recv 0.5 s), device copies ~4.3 s (~64k copy_ calls, consistent with the replicated linear attention's full-sequence copies), pageable host-to-device copies ~2.1 s (3007 calls), GDN kernels ~1.6 s, causal conv ~1.0 s, attention ~0.5 s. The host optimizer step is 4.8 s of CPU time with the GPU idle. The GDN kernels are not the bottleneck. - Head-sharded GDN debugging (/tmp/r3/compare_grads.py, per-parameter gradients of step 0): shared and per-head parameters disagree (input_layernorm cosine down to −0.88, dt_bias/A_log cosine ~0–0.3); the CP all-reduce of the output gradient works (norm 6.6e-5 → 7.5e-5), so the defect is elsewhere. Stopped: not worth more time (~3 s). - Hypothesis being tested: Ray sets OMP_NUM_THREADS=1 for the Megatron actors (it does so when the variable is unset), which would make the host AdamW step single-threaded. Time run with OMP_NUM_THREADS=24 (/tmp/r3/time_omp). - Thread count of the Megatron trainer: Ray sets OMP_NUM_THREADS=1 for actors when unset, which made the host AdamW step slow. Setting torch's intra-op threads to 24 in the training actors only (SLIME_OPD_TRAIN_CPU_THREADS=24, actor.py init; rollout and teacher processes are untouched). Time runs:/tmp/r3/time_omp(global OMP_NUM_THREADS=24: windows 43.2/45.0/41.3 s),/tmp/r3/time_thr2(trainer only): train side 1.6663, windows 42.12/42.61/44.03 s, actor steps ~35.5–37.3 s, projected step 1.2345 (validated fla baseline 1.2146). Quick check of this configuration started (/tmp/r3/quick_thr). - Warning (round 3): the trainer-thread configuration failedcheck --quick(/tmp/r3/quick_thr: INCORRECT, grad_norm_first d = 0.00174 vs tolerance 0.00165; update cosine 0.9592). Every check since 05:43 sits near cosine 0.958 (FlashQLA 0.9574, diagnostic run 0.9582, debug copy 0.9584, thread run 0.9592), while the committed code gave 0.9935–0.9955 at 01:50–05:26. Checking whether the committed code (/tmp/r3/tree_head, clean HEAD) still gives 0.994 under the current machine state. - Root cause of the 0.957 cosine (round 3): the training environment'sfla(flash-linear-attention 0.5.2) has a backend registry;FlashQLABackend(fla/ops/gated_delta_rule/backends/flash_qla.py) isdefault_enable = Trueand dispatcheschunk_gated_delta_ruleto FlashQLA's TileLang kernels whenever theflash_qlapackage is importable. Installing flash-qla into /opt/candidate-env (05:31) therefore changed the GDN kernels of every training process. Uninstalled at 07:42; the clean HEAD (/tmp/r3/tree_head) passes again:/tmp/r3/quick_head_noflashqlaPASS N1, cosine 0.9946, grad_norm_first d = −2.6e-4. Consequences: the FlashQLA speed result (1.2247) and all timings after 05:31 (the OMP and trainer-thread runs) were measured with FlashQLA active and are invalid; they must be repeated in the clean environment. - Rule: do not install packages into /opt/candidate-env that the training stack might auto-detect (flash_qla, etc.). - Clean-environment re-test of the trainer-thread setting (working tree = HEAD + thread setting only, diagnostic block removed): chaintime --side trainthencheck --quickwith{NO_RECOMPUTE=1, SLIME_OPD_TRAIN_CPU_THREADS=24}(/tmp/r3/time_clean_thr,/tmp/r3/quick_clean_thr). - Adopted (round 3): commite79fe452(slime/, branch opt):SLIME_OPD_TRAIN_CPU_THREADS(torch intra-op threads of the training actors). Clean-environment results:time --side train/tmp/r3/time_clean_thr: train side 1.6571, windows 44.47/43.24/43.90 s, projected step 1.2321 (previous committed config 1.2146 on repeat).check --quick/tmp/r3/quick_clean_thr: PASS N1 (update norm d −1.1e-4 / tol 1.2e-3; first grad norm d −2.2e-4 / tol 1.65e-3; cosine 0.9936 / floor 0.9566; mismatch 1.0)./submission/switches.json={"environment": {"SLIME_OPD_NO_RECOMPUTE": "1", "SLIME_OPD_TRAIN_CPU_THREADS": "24"}}. Full train-side check started (/tmp/r3/full_thr). - Full train-side check of the committed configuration (/tmp/r3/full_thr/check.json, 08:38): PASS, tier N1, projected step speedup 1.2287, train side 1.644 (windows 43.53/44.29/43.05 s vs reference 73.7/72.8/73.4 s). - Rollout-side re-check in the clean environment with the final switch file (/tmp/r3/rollout_final, started 08:39). - Final check on the committed configuration, rollout side (/tmp/r3/rollout_final/check.json, 09:20): PASS, tier N1; judged rollout + train; rollout exact ok, token forcing mean |d| 0.00769 (reference 0.00779), mismatch ratio 1.0083; train 1.6359; projected 1.2185 (rollout side 0.989 is run-to-run noise: no switch touches the rollout processes). Teacher side last passed at 04:20 with unchanged code. - Shape-grouped step profile (diagnostic copy/tmp/r3/tree_diag,/tmp/r3/diag_shapes.txt): the large copies are the CPU-offload transfers (chunks of 16–50 M elements, ~2.2 s of device time per step) and the replicated linear attention's full-sequence copies ([1, 8704, 4096],[1, 8704, 16, 2, 128], ~1–1.5 s). SP all-gathers of the TE layers take ~3.1 s (12481 calls). No cheap exact fix found. - Teacher side re-check on the committed configuration started (/tmp/r3/teacher_final). - Teacher side on the committed configuration (/tmp/r3/teacher_final/check.json, 10:00): PASS, tier N1; judged teacher + train; teacher side 0.9986 (unchanged code); train 1.693 in this run; projected 1.2412. All three sides now have PASS verdicts one79fe452. - Log-probability chunk size (SLIME_OPD_LOG_PROBS_CHUNK=8192, one chunk per microbatch; train-side switch in megatron_utils/arguments.py):time --side train/tmp/r3/time_chunk: train side 1.6942 (windows 42.98/42.98/43.03 s, actor steps 35.5–36.1 s), projected 1.2417. Quick check started (/tmp/r3/quick_chunk).
4.7 hCurrent summary (updated 10:45 UTC, round 3)
- Submission: branchoptofslime/ate283a80a(commits:ad765082dead reference forward removed + OPD log-probs reused;a212f8ffNO_RECOMPUTE switch;e79fe452trainer CPU threads switch;e283a80alog-probs chunk switch).sglang/andMegatron-LM/unchanged. Environment:/opt/candidate-envclean (flash-qla removed). - switches.json:{"environment": {"SLIME_OPD_NO_RECOMPUTE": "1", "SLIME_OPD_TRAIN_CPU_THREADS": "24", "SLIME_OPD_LOG_PROBS_CHUNK": "8192"}}- Latest full check (train side):/tmp/r3/full_chunk/check.json→ PASS, tier N1, projected step speedup 1.2453 (train 1.7081, windows 43.14/42.48/42.09 s vs reference 73.7/72.8/73.4 s). Record gates: cosine ~0.9945, all within tolerance. - Other sides for the configuration before the chunk switch (e79fe452): teacher PASS (/tmp/r3/teacher_final), rollout PASS N1 (/tmp/r3/rollout_final). The chunk switch is train-side only and changes no rollout or teacher code. - Gains by change (train side): dead reference forward + OPD log-probs reuse (~22 s of the reference train window), no recompute (~9 s), trainer CPU threads (~1.5–2 s), chunked log-probs (~0.5–1 s). - Dead ends (cumulative): microbatch packing (memory), teacher chunk 8192 (teacher side 1.02), sleep host-cache skip (none), head-sharded GDN (gradient mismatch, branchwip-gdn-head-shard), FlashQLA (auto-dispatched by fla; cosine 0.957; removed), Qwen3.5 native GDN (parameter names). - Not measured: the full-loop speedup (final evaluation). Projected ~1.2 from the train side and the rollout/teacher results.
9.9 hRound 4 (started 10:50 UTC)
- Lever: rollout side (60% of the full loop). Profile first: judgeprofile --side rollouton the committed configuration (/tmp/r4/profile_rollout). Decode is ~6.4–7.3 ms per step at batch 32 per engine with CUDA graphs on (sglang log), fa3 attention, flashinfer sampling, overlap scheduling enabled.