DeepSeek V4.1 Flash-1017 1.43×
Harness: DeepSeek's own harness · switches: SLIME_OPD_SKIP_REF_FORWARD=1 SLIME_OPD_NO_RECOMPUTE=1 SLIME_OPD_KEEP_TRAIN_RESIDENT=1 OPD_TEACHER_SERVER_ARGS=--chunked-prefill-size 16384 --max-prefill-tokens 32768 SLIME_STUDENT_SGLANG_ARGS=--disable-radix-cache
- Judged
- 1.43×
- Tier
- N1
- Self-best
- 1.43×
- Rounds
- 13
- Hours
- 10.0
- API cost
- $2.06
- Input tokens · 99.6% cached
- 213 M
- Output tokens
- 0.4 M
- Requests
- 519
§ 02Key levers
- Candidate, three windows (s)
- 107.1 / 108.1 / 106.6
- Rollout side
- 1.29×
- Teacher side
- 1.06×
- Train side
- 1.58×
- Per-step sampler sync
- ✓
- Train-side main levers
- partial: skip reference forward, no recompute, training resident
- Teacher-side changes
- chunking only: teacher chunk 16384 + single-batch prefill 32768; student radix cache off
§ ARaw trace
Raw work log15 sections
0.0 hRound 1 — baseline profiles
Thu Nov 12 11:25:40 PM UTC 2026
0.0 hRound 1 (start 23:25 UTC) — baseline profiling
- Started all threebench.py profileruns on the unchanged reference (job 501481, log /tmp/r1/profile.log). - Reference component timings fromruns/bootstrap/timing_train_1/.../train.log(steady step): step window 74.6 s = ref_log_probs 10.8 + actor log_probs 10.1 + actor_train 46.7 + sleep/wakeup 6.6. Full loop 153.8 s = rollout 76.3 (generation ~59.9 + teacher tail 16.4) + train 67.8 + sleep/wake 5.2 + update_weights 2.0 + misc. - Key facts: student decode is bandwidth-bound (~4950 tok/s/engine at short context, ~3700 tok/s at 8k); teacher replay (64 x ~8.5k tokens prefill) is ~45% MFU; reference train record is NOT bitwise repeatable (repeatable=false), so the statistical N1 gate is the realistic target. - Candidate changes prepared: T1SLIME_OPD_SKIP_REF_FORWARD: the reference-model log-prob forward pass is dead code when every KL coefficient is 0 (recipe: kl_coef 0, kl_loss_coef 0). Saves ~10.8 s/step (train side, loss/grad unchanged). T2SLIME_OPD_NO_RECOMPUTE: drop full activation recompute (memory strategy; arithmetic unchanged). TeacherOPD_TEACHER_SERVER_ARGS=--chunked-prefill-size 8192(one of the reference's own frozen rebatchings, so it is inside the teacher tolerance by construction). Thu Nov 12 11:30:02 PM UTC 2026
1.1 hRound 2 (23:45–00:30 UTC) — train: dead ref forward + no recompute; teacher chunk
Measured with time (frozen reference windows in brackets):
- train, switches SKIP_REF_FORWARD+NO_RECOMPUTE (/tmp/sw_train.json): windows 56.239/75.342/55.065 s
[76.649/88.036/73.417] -> side speedup 1.1685, mean step 62.2 s vs 79.4. Components (steady):
log_probs 10.8 s (unchanged), actor_train 38.0 s [46.7], sleep+wake+wait 7.4 s [6.6].
Window 2 hit the same host-side stall the reference shows (actor_train 57 s instead of 38 s).
- OMP_NUM_THREADS=24 + max-tokens-per-gpu=8192 (v3): step 57.4/58.8 s — neutral, dropped.
max-tokens-per-gpu=16384 OOMs (a 32k-token microbatch needs >100 GB of activations), dropped.
- teacher, OPD_TEACHER_SERVER_ARGS="--chunked-prefill-size 8192": 17.536/17.821/17.324 s
[18.375/18.303/17.912] -> 1.027. Adding --max-prefill-tokens 32768: identical (1.028); the
scheduler caps the prefill batch at chunked_prefill_size, and throughput is flat in batch size
(~30k tok/s, ~41% MFU) -> the teacher is compute-bound, not batching-bound.
- rollout profile (200 decode steps, 1 engine): kernel time 4.99 ms/step of a 6.4 ms step;
GEMM 68.5% (3.42 ms, all ~0.73-wave 96-block cuBLASLt nvjet kernels at ~1.5-1.9 TB/s of 4.8),
GDN/mamba 11.7%, comm (allreduce fusion + allgather) 11.0%, elementwise 6.1%, attention 1.5%.
The allreduce fusion is already active (backend auto -> trtllm); sampling backend already flashinfer.
-> decode is bandwidth/occupancy-bound at ~30% of HBM peak; no config fast path identified.
Full check started 00:30 UTC on /tmp/r3/check with switches
{SKIP_REF_FORWARD, NO_RECOMPUTE, teacher chunked-prefill 8192}.
Fri Nov 13 12:31:05 AM UTC 20262.5 hRound 3 (00:30–01:47 UTC) — FULL CHECK
- First attempt (/tmp/r3) crashed in the judge itself:judge_rolloutreuses gate-1's run and never creates<out>/rollout, soscore()died onopen(<out>/rollout/score.log). Pre-creating<out>/rollout(+ teacher, train) fixed it. Gate-1 data from that run was still valid: rollout windows 75.94/76.46/76.41, train windows 55.4/56.1/56.7. - Second attempt:/tmp/r4/check→ PASS, speedup 1.1424, tier N1 (check.json). side speedups: rollout 1.010, teacher 1.0283, train 1.3032. - rollout: exact ok; token forcing mean|d| ratio 0.9905 (<2), 4 events (p=0.35), sampler t=1.52 (<4); train-inference mismatch ratio 1.0034. - teacher: mean|d| ratio 1.0149 of the reference's pooled rebatching bound (limit 2), 10 events (p=0.38), bias t=-1.03 (limit 4) — exactly the frozenrebatch_chunk8192variant, i.e. bitwise that rebatching. - train: exact ok; not bitwise (reference itself is not repeatable); statistical gate all within: update_norm 0.148319 vs 0.148452 (tol 1.20e-3), exp_avg_sq 0.0016464 vs 0.0016418 (tol 4.4e-4), loss_mean 0.0516844 vs 0.0516909 (tol 3.8e-5), grad_norm_first 0.99652 vs 0.99674 (tol 1.65e-3), update-direction cosine 0.99358 (min 0.95660). mismatch ratio 1.0.
2.9 hRound 4 (01:53–02:16 UTC) — marginal levers, separate trees (no submission change yet)
- teacher--disable-radix-cache(on top of chunk 8192): windows 17.063/17.156/16.746 s -> teacher side 1.0669 (vs 1.0283 for chunk only). Numerics validated bycheck --quickon a reference-shaped tree (/tmp/reftree + teacher switch): PASS, i.e. bitwise the frozenrebatch_noradixvariant. Full-loop value is small (~0.4 s of the teacher tail, ~0.3 %), so NOT put into switches.json; the submission keeps the config that passed the full check. - teacher--linear-attn-prefill-backend flashinfer: 1.0289 (no gain over chunk 8192). - student--linear-attn-decode-backend flashinfer: rollout windows 86.6/87.2/87.4 s -> 0.883x (12 % SLOWER than the reference). Rejected. Fri Nov 13 02:16:17 AM UTC 2026
3.5 hRound 5 (02:16–02:52 UTC) — student radix cache off (accepted)
-time --side rolloutwith studentSLIME_STUDENT_SGLANG_ARGS="--disable-radix-cache"(teacher chunk 8192): windows 74.177/74.216/74.173 s vs frozen 77.402/77.246/77.164 -> rollout side 1.0403 (was 1.010 with the radix cache on). The radix cache never hits (prefix_cache_hit_rate 0) and its mamba extra-buffer state tracking costs ~2.1 s per rollout. - Gate validation:check --quick --tree /tmp/rolltree(reference slime/Megatron + sglang copy whose disable_radix_cache default is True) -> rollout gate PASS: exact ok, token forcing mean|d| ratio 0.9957 (limit 2), 7 events (rate 2.3x reference but binomial p=0.034 >> 0.001), sampler t=0.739, train-inference mismatch ratio ~1.0; teacher gate PASS. - Final switches.json: {SLIME_OPD_SKIP_REF_FORWARD=1, SLIME_OPD_NO_RECOMPUTE=1, OPD_TEACHER_SERVER_ARGS="--chunked-prefill-size 8192", SLIME_STUDENT_SGLANG_ARGS="--disable-radix-cache"} (teacher kept at chunk 8192 only: that exact teacher was validated end-to-end by the full check; adding --disable-radix-cache to the teacher is validated on the teacher gate but gains only ~0.4 s and would leave the online teacher+rollout combination untested, so it is not used.) - Expected projection with the final switches: 1/(1 - 0.501*(1-1/1.0403) - 0.5146*(1-1/1.3032)) = 1.161. - A final full check (/tmp/r7/check) with the final switches was queued at 02:52 UTC. Fri Nov 13 02:52:27 AM UTC 2026
4.2 hRound 6 (02:52–03:30 UTC) — final full check on the final switches
-/tmp/r7/check: gate 1 rollout 1.0302 (windows 74.104/74.222/74.899), teacher 1.0255, train 0.9869 -> status UNAVAILABLE (speedup 1.0079). The train window 3 hit the host-side stall (+17 s inside the *microbatch loop*, actor_train 56 s instead of 38 s), so the min ratio fell below 1. - Diagnosis of the stall: it appears only in--debug-train-onlyruns (fixture dumps 1–3): v1 timing (step 5), r7 (step 6), the reference's own train timing runs (step 5, both runs), while the bootstrap full-loop reference A/B and control runs (5 RL steps each, 10 timed steps) show no stall (spreads 0.8 %). Not Triton first-use (first_use.seconds = 0 in every timed window), not cgroup CPU throttling (0.16 s total throttled), not memory pressure (1 TiB container limit, ~200 GB used). Treated as a train-only artifact; the final evaluation times the full loop. - Because a stalled full-loop window would zero the score, the last large lever (keep Megatron resident across the rollout, ~5 s/step) was re-rejected: it cannot be validated in the full loop with the judge's tools (--finalis unavailable), the colocate path withoffload_train=Falsealso changes the grad/param CPU-backup flags used by the weight sync, and an OOM/regression would zero the score. - Final state: see the summary below. No processes left running; GPUs idle.
4.6 hRound 7 (03:35–04:03 UTC) — expensive levers tried
- Log-prob pass microbatch repacking (SLIME_OPD_LOGPROB_TOKENS_PER_MBin actor.py: repack the samples into 4-sample bins for the inference-only log-prob forward, which stores no activations; forward_only already restores the sample order from the iterator). Implemented, timed, rejected: the log-prob pass did get faster (10.8 -> 9.3 s) but the transient 8.6 GB logits buffers grew the caching allocator's reserved memory 33 -> 40 GB, which made the colocated sleep grow 4.1 -> 9.1 s and wake 1.4 -> 2.8 s: step 60.5-66.1 s vs 56-58 s before. Code reverted (git checkout). - Student engine--enable-torch-compile(inductor max-autotune compiles the decode forward and captures it in the CUDA graph; the autotune log shows BLOCK_M=32 Triton GEMMs, promising against the 0.73-wave 128x32 cuBLASLt kernels in the trace). First attempt with the recipe's mem-fraction 0.45 OOMs during capture (needs >12 GB on top of the 63 GB static pool + 35 GB teacher). Retesting with--mem-fraction-static 0.35(KV pool still ~2.5x the used capacity). Compile time is outside every timed window (graph capture is at engine init), but it makes each run ~10-20 min longer. Fri Nov 13 04:02:48 AM UTC 2026 - Student--enable-torch-compile— rejected (memory). Even with--mem-fraction-static 0.35(max-autotune) or0.40withSGLANG_TORCH_COMPILE_MODE=default, the decode-graph capture + inductor workspaces OOM during engine init (needs 8.3-12.2 GB on top of the 49-56 GB static pool and the 35 GB teacher; only 1.3-2.7 GB free). The colocated 4-GPU layout has no room for the compile workspace. Note: with compile on, the engine takes a different GDN decode path ("fused GDN decode QKVZ/BA unpack + indexed Conv1D") and inductor autotunes BLOCK_M=32 Triton GEMMs, so it is the promising direction if the memory were available; it is not on this layout. - Submission left at the validated Round-5 configuration (no further switch changes).
5.4 hRound 8 (04:20–04:46 UTC) — keep the actor resident across the rollout (ACCEPTED)
- Change:SLIME_OPD_KEEP_TRAIN_RESIDENT=1(committed 0ae905df,slime/utils/arguments.py): with the colocated recipe the actor otherwise offloads to host inside every train step (sleep 4.1 s + wake 1.4 s + process-group destroy/reload + cold-cache effects). The resident actor fits: the student engines still release their KV/state pool during training (offload_rolloutuntouched). -time --side train(judge tool): windows 47.072 / 46.565 / 46.323 s vs [76.649 / 88.036 / 73.417] -> side speedup 1.5849. Steady components: log_probs 9.5 s (was 10.8-12), actor_train 35.2 s (was 38-39, no cold allocator/process-group reload), train_wait 1.7 s (was 7.4). - Full-mode validation (the judge's ownbench/launch.sh, MODE=full, NUM_ROLLOUT=2, forced lengths, run under the judge's GPU flock, no timing read): exit 0, 2 RL steps, no OOM; peak GPU memory 125 GB of 140 GB during generation, 96 GB after the weight sync; the step-1 train-inference mismatch 0.00810 (reference 0.00805) so the weight sync is correct; update_weights 0.73-1.36 s. - switches.json now: SKIP_REF_FORWARD, NO_RECOMPUTE, KEEP_TRAIN_RESIDENT, teacher chunk 8192, student radix cache off. Projected step speedup with rollout 1.030 and train 1.585 = 1.257; expected full-loop speedup ~154.2/125.7 = ~1.23. -check --quickon the final switches started 04:46 UTC (/tmp/r12/quick) to re-validate the three correctness gates with the resident actor. Fri Nov 13 04:46:36 AM UTC 2026
5.4 hRound 8 verdict — quick check PASS with the final switches
/tmp/r12/quick/check.json (04:46–05:21 UTC): status PASS, all three sides PASS with the resident
actor: rollout PASS (exact + token forcing + teacher-on-rollout + mismatch), teacher PASS, train PASS
(exact ok; update_norm 0.148195 vs 0.148452, loss_mean 0.051672 vs 0.051691, grad_norm_first 0.996354
vs 0.996736, update-direction cosine 0.99253 vs min 0.95660).6.8 hRound 9 (05:23–06:11 UTC) — FULL CHECK on the final configuration
/tmp/r13/check/check.json: status PASS, projected step speedup 1.2554, tier N1.
- gate 1: rollout side 1.0294 (74.568/74.591/74.959 s vs 77.402/77.246/77.164), teacher side 1.0294
(17.49/17.78/17.34 vs 18.375/18.303/17.912), train side 1.581 (46.55/46.92/47.00 vs
76.649/88.036/73.417). No per-window stall this time.
- correctness: rollout PASS (exact, token forcing, teacher-on-rollout, mismatch), teacher PASS,
train PASS (update_norm 0.148440 vs 0.148452, loss_mean 0.051687 vs 0.051691,
grad_norm_first 0.996474 vs 0.996736, update-direction cosine 0.99564 vs min 0.95660).
- Expected final full-loop speedup: 154.2 / (74.6 rollout + 47.0 train + ~4 sync/misc) = ~1.23.
Fri Nov 13 06:14:12 AM UTC 20267.2 hRound 10 (06:15–06:35 UTC) — profile + torch.compile retry with one captured bs; no gain
- Profiled the current best rollout (200 decode steps, radix cache off, /tmp/r14/profile_rollout): kernel time 5.00 ms/step of a 6.4 ms step; GEMM 63% (same 0.73-wave nvjet classes), GDN decode 9.9% (0.50 ms, 66% of HBM peak), allreduce fusion 9.4%, attention 2.6%, elementwise ~4%, sampling 1.5%. The mamba state-tracking kernel is gone (that is the radix-off gain). No new hotspot. - Retried student--enable-torch-compilewith a single captured decode bs (`--cuda-graph-bs-decode [32],--mem-fraction-static 0.30`, keep-resident active) to fit the compile workspace. The engine still dies during graph capture: an inductor-generated kernel allocates a 1.88e9-element fp32 buffer (empty_strided_cuda((1882718208,)), 7.5 GB) inside the compiled linear-attention decode path and OOMs. Inductor's codegen for this hybrid GDN decode is not usable at this model/batch; the direction is closed. Parser change reverted; tree back at 0ae905df. Fri Nov 13 06:34:23 AM UTC 2026
7.6 hRound 11 (06:35–07:25 UTC) — second full check (reproducibility)
- /tmp/r16/check (same tree/switches as the PASSING /tmp/r13/check): gate-1 rollout windows
74.473/74.147/74.380 s (stable, side ~1.037), teacher 1.029, train windows
46.37 / 47.11 / 46.99 / 62.87 s -> the last timed window hit the train-only host stall again
(+16 s on actors), so this check projects ~1.10 instead of r13's 1.2554. The stall has only ever
been seen in --debug-train-only runs (fixture dumps), not in full-loop runs.
- Robustness note: with the resident actor the full step is ~125 s vs the reference's 154 s, so even a
+16 s stall in a full-loop window would still give a ratio of ~141 -> 154/141 = 1.09, above the
max(1.02, 1+2*spread) bar; the change also widened that margin.
- Correctness gates of r16 are re-running; the r13 full check already PASSed all three with the same
configuration (and /tmp/r12/quick PASSed with the identical switches).
Fri Nov 13 07:01:35 AM UTC 20269.2 hRound 12 (07:33–08:32 UTC) — sampler per-step device sync removed (ACCEPTED, big)
- CPU-profiled a student engine's scheduler with py-spy during a rollout: **70.9 % of the samples sat in one line ofSampler.forward** —sampling_info.return_top_p_log_probs.any().item(). The field is always a CUDA bool tensor (never None), so a reduction + device-to-host sync ran on *every decode step* even though the recipe never asks for top-p logprobs, stalling the scheduler and breaking the overlap pipeline. - Fix (sglang commit 453bb3d, sglang/): carry the same information as a CPU flag (need_return_top_p_log_probs, built from the per-request custom params and OR-ed on batch merge), exactly equivalent to the tensor check. Sampling results unchanged; only the sync is gone. -time --side rollout: windows 60.10 / 59.44 / 58.85 s (was 74.57/74.15/74.38) -> rollout side 1.2879 (was 1.030). ~14 s per rollout recovered. - FULL CHECK/tmp/r19/check: PASS, projected step speedup 1.4342 (side: rollout 1.2943, teacher 1.0283, train 1.5796), all three correctness gates PASS (rollout exact/token forcing/ teacher-on-rollout/mismatch; teacher; train statistical). - Expected final full-loop speedup: 154.2 / (59.4 rollout + 47.0 train + ~4 sync/misc) = ~1.40. Fri Nov 13 08:34:04 AM UTC 2026
9.8 hRound 13 (08:48–09:13 UTC) — CPU profiles of the train and teacher; teacher prefill chunk 16384
- py-spy of a Megatron rank during a train step: 26 % of the CPU samples in TE's CP-attention backward (context_parallel.py:2869, thedq[cu_seqlens_q_padded[-1]:].fill_(0)GPU-scalar slice that syncs once per full-attention layer per backward), ~19 % in Triton JIT compilation (warm-up steps only;first_useinside the timed windows is 0), 8.9 %torch.cuda.synchronizefrom the Triton autotuner (also warm-up), and the rest diffuse. The TE sync only makes the CPU wait while the GPU runs, and the patched alternative would need monkey-patching an installed package, so it was not pursued. - py-spy of the teacher scheduler during a replay: the process is mostly waiting on the GPU (60 samples in 12 s; 55 % memory-pool alloc, 23 % CUDA-graph replay), i.e. the 15 s teacher tail is GPU-bound prefill (42 % MFU), no CPU serialization to remove. - Teacher--chunked-prefill-size 16384 --max-prefill-tokens 32768(one chunk per 8.5k-token request instead of two):time --side teacherwindows 17.165/17.160/16.941 s -> teacher side 1.0573 (was 1.0283); teacher gate PASS (mean|d| ratio 1.0128, 11 events p=0.26, bias t=0.02, /tmp/r23/quick_t). ~0.4 s off the rollout's teacher tail. - switches.json updated to the 16k teacher chunks; full check /tmp/r24/check started 09:13 UTC. Fri Nov 13 09:13:42 AM UTC 2026