Claude Sonnet 5.5 1.96×

Harness: Claude Code · switches: OPT_DECODE_GRAPH=1 OPT_DENOISE_FAST=1 OPT_REF_RESHARD=1 OPT_FUSED=1 OPT_OVERLAP_REF=1 OPT_EARLY_FWD=1 OPT_DECODE_SAMPLE_GRAPH=1 OPT_CLONE_CTX=1 OPT_FUSED_TRAIN=1 OPT_FUSED_GEN=1 OPT_PACK_FAST=1 OPT_GPU_YIELD=3000 OPT_DECODE_SKIP_WARM=1 OPT_FSDP_PREFETCH=1 PYTORCH_ALLOC_CONF=expandable_segments:True OPT_NO_EMPTY_CACHE=1

Judged
1.96×
Tier
N0
Self-best
1.92×
Rounds
9
Hours
10.0
API cost
$37.9
Input tokens · 99.2% cached
209 M
Output tokens
1.1 M
Requests
461
Text-decode CUDA graph
✓ OPT_DECODE_GRAPH, first half of sampling also in the graph (self-estimated ~−9 s per window)
Reward wait hidden behind forward
✓ OPT_OVERLAP_REF + OPT_EARLY_FWD; OPT_GPU_YIELD=3000 yields the GPU every layer; together with skipping decode warmup, full check 1.74× → 1.85×
Rollout-side bitwise fused kernels
✓ Triton: RMSNorm, q/k norm + RoPE, whole denoise layer fused (OPT_FUSED, OPT_FUSED_GEN)
Train-side bitwise kernels / freeing memory
✓ OPT_FUSED_TRAIN (1.7035× → 1.7087×, within noise); OPT_REF_RESHARD peak down to ~104 GiB; FSDP prefetch + expandable segments + no cache clearing 1.85× → 1.92×
Other main levers
packing desync, text CFG context copied directly; pipelined decode, fused MLP tried, no gain
Raw work log9 sections
0.4 hRound 0 — profile of the unchanged reference (2026-11-06 05:46 - 06:06)
Command: bench.py profile --tree /submission/r3 --config /submission/switches.json --out /tmp/prof-ref --from 11 --iters 2
(steps 11 = refine/edit step, 12 = generate step; rank 0; profiler overhead inflates step times ~10%).

Step times seen in the run log (reference, steps 5..11): generate steps 14.6-17.2 s, refine steps 30-35 s.
Per window of 2 steps (one generate + one refine) ≈ 47 s -> the score is dominated by the refine step (~2/3).

Findings from the 54 s Chrome trace (rank 0, 1.44M kernel launches in two steps; GPU kernel time only 26.5 s of 54 s = 49% busy):

| phase (refine step / generate step)        | wall (s) | GPU busy | note |
|---|---|---|---|
| text decode (generate_text_batch)          | ~11 / ~8 | ~18% | 130 decode steps per 6 s => ~46 ms/token-step under profiler; ~1770 kernel launches, ~100 .item() syncs per token-step |
| image denoise (generate_image_mix)         | ~13.5 / ~8 | ~65% | ~80k cudaStreamSynchronize per 10 s (3.5 s blocked), 70k .item(), boolean-mask indexing (nonzero) in _merge_static_kv_prefix_and_query per layer per pass |
| reward wait + glue                         | ~1.5 / ~1.5 | 0% | idle GPU |
| training (ref fwd + policy fwd/bwd + opt)  | ~8 / ~3 | ~100% | cudaMalloc/cudaFree thrash in backward (1.8 s per window total), memory is ~131 GiB at peak |

Hotspot classes: decode = launch/host-sync bound (kind 2: missing graph/sync-free path); denoise = host sync bound;
training = GPU bound (GEMM nvjet kernels 15%+7.7%, flex-attn triton, NCCL RS/AG).
Top GPU kernels overall: nvjet GEMMs (4.0 s + 2.0 s + ...), index_put (1.38 s, 41k launches), elementwise (~3 s over >300k launches).

Plan: (1) sync-free + CUDA-graph decode (same kernels => bitwise), (2) sync-free denoise merge, (3) allocator
(expandable segments / free ref-model unsharded params), (4) misc host overlap.
1.0 hRound 1 — sync-free CUDA-graph decode + sync-free denoise (2026-11-06 06:14 - 06:42)
Phase breakdown of the unchanged reference without profiler (scratch tree with CUDA-event phase timer, judge time:
23.48 s/step vs frozen 23.36, i.e. the timer is free): generate step ≈ 14.3-16 s (decode 4.7-5.4, denoise 5.2-5.7,
reward wait 1.1-2.2, training ≈ 2), refine step ≈ 34.5-35 s (decode 9.4-10.9 for 340-385 token-steps = 28 ms/step,
denoise 10.1-10.3, reward wait 0.65-2.5 + collective idle 1.4-3.6, ref fwd 1.1-1.5, policy fwd 1.4-3.1, bwd 4.2-4.7).

Changes (all N0, each behind a switch, defaults off):
- OPT_DECODE_GRAPH=1: Bagel.generate_text_batch -> fast_paths.generate_text_batch_fast for StaticKVCache: host-side
  bookkeeping (one D2H copy of (next token, prob) per step instead of ~100 .item()), per-active-count CUDA graph of the
  28-layer decode forward (+lm_head) (captured after one eager step per M, thread_local capture, no gc/empty_cache),
  sampling/RNG calls unchanged (eager, same shapes and order), advance_seq_lens unchanged.
  _static_kv_cache_attention got a sync-free branch for decode (cache.fast_decode).
- OPT_DENOISE_FAST=1: torch.tensor([t]*4096) (4096 device syncs per step!) -> t.expand(N).contiguous(), host-side
  cfg-interval test, no unique()/sum() syncs in _forward_flow, and _static_kv_cache_readonly_attention builds the
  merged K/V with torch.cat of [cached prefix | new tokens] per sequence from a host-known plan (cu_seqlens, max_k) instead
  of ~15 syncs + boolean-mask indexing per layer per pass.
Verification: single-GPU harness (real weights, deterministic mode, rollout() gen + refine, rollout_for_eval()):
808/808 outputs bitwise identical (tokens, probs, latents, log-probs, images); harness wall: gen 14.0->7.2 s,
refine 29.7->14.6 s, eval 137->75 s.
Full check #1 (/tmp/check-1): ERROR - CUDA OOM in the backward of step 1 (129.9 GiB allocated, 248 MiB free; the reference
sits at ~131 GiB peak, the ref model's unsharded params stay resident). => memory headroom is mandatory:
added OPT_REF_RESHARD=1 (reshard the reference model after its forward) and OPT_OVERLAP_REF=1 (prepack + pack + ref
forward right after the rollout, overlapping the reward wait; advantages are filled in later via slots, not enabled yet).

### Full check #2 (06:42 - 07:03) — PASS, N0, speedup 1.378
/tmp/check-2/check.json: switches {"OPT_DECODE_GRAPH":"1","OPT_DENOISE_FAST":"1","OPT_REF_RESHARD":"1"} (OPT_OVERLAP_REF off),
commit bebb2ae. Gate 1: windows 15.97 / 16.61 / 16.83 s per step vs frozen 23.11 / 23.78 / 23.20 -> ratios 1.447 / 1.432 / 1.378,
speedup 1.378 (bar 1.0206). Gate 2: bitwise PASS (14 steps, 3200 tensors, 235 reward requests identical). Peak allocated 104.3 GiB
(reference ~131) thanks to resharding the reference model. Steps in the timing run: generate steps 9.7-10.5 s, refine 21.5-22.9 s.
2.1 hRound 2/3 — fused exact kernels, overlap of reward wait, in-graph sampling (07:00 - 07:50)
- OPT_FUSED=1: Triton kernels that reproduce the reference's op chains bit for bit, including ATen's mean(-1) summation
  order (block width W from setReduceConfig: thread-strided vec4 accumulators, halving folds, adjacent-pair warp tree) and
  rsqrt.approx: rms_norm_exact (C=3584, bf16/fp32 in) and qk_norm_rope_exact (head_dim 128, bf16 path for decode/prefill,
  fp32 path + row scatter for the denoise 'gen' mode). enable_fp_fusion=False, do_not_specialize on row counts, all kernel
  variants compiled in warmup() (a first-use JIT compile had cost 3.3 s inside a timed window).
  Tested bitwise against the module chain: 228 + 264 cases (n = 1..60000, bf16/fp32, 3 scales, idx scatter). Kernel time vs
  reference chain: RMSNorm n=4096 213 -> 30 us; q-norm+rope n=4096,H=28 586/756 -> 73 us.
- OPT_OVERLAP_REF=1: prepack + pack_batch (advantages deferred via slots) + reference forward right after the rollout;
  OPT_EARLY_FWD=1: also the policy forward (it does not need the rewards); zero_grad right after the optimizer step.
- OPT_DECODE_SAMPLE_GRAPH=1: the deterministic front half of _sample_next_token (topk/logsumexp/cumsum/mask) and the raw
  softmax run inside the decode graph; only multinomial (RNG) + gathers stay eager.
- OPT_CLONE_CTX=1: the text-CFG context of edit_image copies the image-prefix K/V of the conditional context instead of
  recomputing VAE+ViT+28 layers.
- Phase timer (fused + overlap, /tmp/time-pt3): generate step 8.3-9.0 s, refine 18.2-20 s (was 14.3-16 / 34.5-35).
- Full check #3 (all switches except CLONE_CTX): speedup 1.659 (windows 1.717 / 1.659 / 1.707) but INCORRECT: from the first
  training rollout on, every reward request differed (the 11 step-0 eval requests matched). Cause: my own warmup() called
  torch.randn on the default CUDA generator, which the SDE sampler draws from. The harness had masked it by re-seeding before
  each rollout. Fixed (RNG-free warm-up), harness now seeds once and compares the three RNG states after every rollout.
  Harness (det mode, all switches): 817/817 entries bitwise equal incl. RNG states; gen 14.07 -> 5.42 s, refine 29.8 -> 10.5 s,
  eval 136 -> 60 s.

### Full check #4 (07:53 - 08:12) — PASS, N0, speedup 1.7035  (current best; /tmp/check-4/check.json)
Tree commit a1169c9 (later edits are gated off by default), switches (= /submission/switches.json):
OPT_DECODE_GRAPH, OPT_DENOISE_FAST, OPT_REF_RESHARD, OPT_FUSED, OPT_OVERLAP_REF, OPT_EARLY_FWD, OPT_DECODE_SAMPLE_GRAPH, OPT_CLONE_CTX all "1".
Gate 1: windows 13.11 / 13.77 / 13.62 s per step vs 23.11 / 23.78 / 23.20 -> ratios 1.763 / 1.7275 / 1.7035 (bar 1.0206).
Gate 2: bitwise PASS (14 steps, 3200 tensors, 235 reward requests identical). Peak allocated 103.9 GiB.
Steps in the timing run: generate 8.4-9.1 s, refine 17.8-18.4 s (reference 14.3-16 / 34.5-35).
3.0 hRound 4 — what did not help, reward-service probe, exact fused TRAINING kernels (08:15 - 09:25)
- B1 (OPT_NO_EMPTY_CACHE + OPT_FSDP_PREFETCH, time run on the phase-timer tree, /tmp/time-b1): worse than check #4
  (windows 13.81 / 14.81 / 14.40 vs 13.11 / 13.77 / 13.62; peak reserved 133.7 GiB). ctx_enter shrank 0.25 -> 0.14 s but the
  cached blocks raised fragmentation/peak. Both switches stay OFF. Also the unsharded parameter addresses change every
  step (pointer signature differs each rollout), so decode graphs cannot be reused across rollouts.
- Reward service probe (judge's service, idle GPUs): single request 0.39 s, 4 concurrent 0.97 s each, 16 concurrent makespan
  3.75 s (4 waves) => ~4.3 requests/s whatever the concurrency; each step's 16-image burst needs ~3.7 s to be scored. It is
  hidden by ref+policy forward in refine steps (3.3 s of work) but ~2 s stay exposed in generate steps (only ~0.9 s of
  reward-independent GPU work exists there). Batching images per request is impossible (the judge compares the request set).
- Denoise (non-det harness profile, all switches): 97% GPU busy, GEMM 72%, flash 6.6%, scatter/gather/silu/add ~20%; decode graph
  steady state 6.1 ms/step (GEMM weight streaming floor ~4.0), capture ~35 ms per M + eager first steps ~0.3 s per rollout.
- GEMMs run at 680-800 TFLOPS (q/o 728, gate/up 805, down 678): nothing to gain there.
- OPT_FUSED_TRAIN=1: exact autograd Functions for the training forward: RMSNormExact (fwd stores r; bwd reproduces
  ToCopy/Mul/Rsqrt(-0.5*g*r^3)/Mean(g * fl(1/C))/Pow(2*h)/Add/ToCopy of the reference chain, row reduction in ATen's order,
  weight grad = ATen sum of the bf16 product) and QKNormRopeExact (rotate_half adjoint, bf16 adds, norm bwd for C=128).
  Op-level tests: 60 + 42 cases bitwise equal (outputs, grad_x, grad_w). Whole-model test on one GPU (real weights, AC, det mode):
  loss/logits/mse preds equal and 796/800 gradients equal for generate and refine packings; fwd+bwd 5.27 -> 4.35 s (refine pack).

### Full check #5 (08:44 - 09:04) — PASS, N0, speedup 1.7087  (current best; /tmp/check-5/check.json)
Tree commit fa13351, switches (= /submission/switches.json): the 8 of check #4 + OPT_FUSED_TRAIN=1.
Gate 1: windows 12.56 / 13.46 / 13.58 s per step -> ratios 1.8407 / 1.7671 / 1.7087 (mean s/step 13.20 vs 13.49 in check #4;
the min-window ratio is dominated by window noise ±5%). Gate 2: bitwise PASS (3200 tensors, 235 reward requests). Peak 102.2 GiB.
3.4 hRound 5 — fused gen-mode (denoise) layer, `OPT_FUSED_GEN=1` (09:05 - )
rms_rows Triton kernel (gather rows by index / add a GEMM output row with one bf16 rounding / RMSNorm in ATen order / write
contiguous or scattered / store the residual sum scattered) + Qwen2Model._forward_inference_gen_fused: per layer
[attention GEMMs + fused q/k norm-rope + flash + o_proj] then add+post-norm in one kernel, MLP, and the next layer's input norm
fused with the MLP residual add (final norm scatters the output rows). Removes ~12 full-tensor gather/scatter/zeros/add passes
per layer-pass. Row-kernel test: 36 cases bitwise. Harness (det, everything on): 817/817 bitwise incl. RNG states;
gen 5.42 -> 5.10 s, refine 10.54 -> 9.86 s, eval 60 -> 48 s (single GPU). Full check #6 launched (/tmp/check-6).

### Full check #6 (09:10 - 09:32) — PASS, N0, speedup 1.7457  (current best; /tmp/check-6/check.json)
Tree commit 8514fbf, switches (= /submission/switches.json): the 9 of check #5 + OPT_FUSED_GEN=1.
Gate 1: windows 12.20 / 13.25 / 13.29 s per step -> ratios 1.8946 / 1.795 / 1.7457 (mean 12.91 s/step). Gate 2: bitwise PASS.
Windows 2 and 3 are consistently ~1 s/step slower than window 1 (reference too: 23.78 vs 23.11); phase timers of a similar run show a
larger exposed reward wait in generate steps 12 and 14 (1.4-1.6 s vs 0.0-0.9 s).
4.2 hRound 6 — small items (09:35 - ), check #7 launched with `OPT_PACK_FAST`
- OPT_PACK_FAST=1: add_latent stores Python floats for the 0-d device scalars timestep/dt (they were repeated 20K times per
  latent into lists that torch.tensor converted with one device sync per element): pack_batch 0.51 -> 0.2 s on the refine
  packing; all 30 packed tensors/lists identical (gen and refine structure). Matters on the refine critical path (GPU idle during packing).
- OPT_DECODE_HOIST (per-step integer index math once instead of per layer): bitwise, no measurable gain -> off.
- OPT_FUSED_MLP (exact silu*mul fwd+bwd, whole-model grads identical): fwd+bwd 4.32 -> 4.30 s on the refine packing, rollouts unchanged
  -> no gain, off. (Triton flushes fp32 denormals; PyTorch does not: differs only for |gate| in [87.5, 88.5] / bf16 subnormals.)
- Blocks tried and dropped: flex block-mask cache between ref and policy forward (the ref forward runs in inference_mode, its mask
  tensors are inference tensors and cannot be saved for backward).
- Critical-path analysis of a pair: gen step = rollout 4.4 s + reward service 2.5 s (the 16-image burst, ~4.3 req/s, 2x slower while
  the GPUs are busy) + backward/optimizer 1.05 s; refine step = rollout 9.0 + service/forwards 3-4.5 + backward 3.6. The service
  latency, not my GPU work, is what the post-rollout phase waits for; the remaining controllable parts are rollout and backward.
5.8 hRound 7 — the reward service is slowed by my GPU work: yield the GPU (2026-11-06 10:26 - 11:33)
Profile (per-rank phase timer, judge time runs /tmp/time-pt5 and /tmp/time-pt6 on a scratch copy with CUDA-event marks; all 4 ranks logged):
- The 4 ranks are in lock-step in every phase (rollout end spread < 0.1 s; the "rank imbalance" I suspected does not exist).
- Window 3 (steps 14-15) is the weakest because of the post-rollout wait for the reward service: time from the end of the
  early block (pack + reference fwd + policy fwd, 0.85 s GPU in gen steps, 3.1-3.5 s in refine steps) to the arrival of the 16 rewards =
  gen steps 8/10/12/14: 1.3 / 1.1 / 2.0 / 1.9 s, refine steps 9/11/13/15: -0.6 / -0.6 / +0.7 / +1.45 s.
  Service latency (submission -> last result): gen 2.0-2.8 s, refine 2.4-4.9 s (4.9 s in step 15: the early block ran at the same time).
  In addition the refine text decode grows with the step (et_text_gen 1.4 / 1.8 / 2.1 / 2.6 s for steps 9/11/13/15, data dependent).
- Service probe (standalone judge service, /tmp/work/reward_probe3-5.py; 16 requests like one step): idle makespan 3.8-4.2 s with realistic
  requests (0.9 s with the first 16 prompts: the cost is dominated by colour classification: DataLoader forks + CLIP crops);
  8 busy CPU cores: no effect, 48 busy cores: +50%; continuous GEMM load on the 4 GPUs: 6.2 s (x1.65); the same load with idle gaps:
  30ms/2ms 3.9-4.4 s, 10ms/1ms 3.9-4.5 s, 100ms/10ms 4.0-4.3 s, 300ms/25ms 3.9-4.5 s, 45ms/3ms 4.1-4.3 s (all ~7-9% idle) but
  100ms/2ms 5.1 s, 300ms/5ms 5.2 s, 1000ms/80ms 4.2-4.6 s, 3000ms/250ms 5.0-5.8 s.
  Cause: GPU time slicing between my process and the service workers (one per GPU); while my queue is never empty, each small dependent
  kernel of the service waits for a slice.
- Change OPT_GPU_YIELD=<us> (new modeling/bagel/gpu_yield.py, hook in Qwen2Model.forward_train, begin/end in online_rl.main):
  between the submission of the rewards and the arrival of the results, the post-rollout forwards (reference fwd, early policy fwd)
  synchronise the stream after every decoder layer and sleep <us> (stops by itself when the rank's own futures are done).
  No tensor is touched: N0. OPT_GPU_YIELD_CHUNK_US (adaptive layers per gap) exists too.
- Time runs on the phase-timer copy (all other switches of check #7): no yield 13.14/13.48/13.95 and 12.76/13.60/14.02 (mean 13.53, 13.46);
  OPT_GPU_YIELD=3000 per layer: windows 12.56 / 13.29 / 12.96 (mean 12.94, speedup_vs_frozen 1.806), exposed wait after the early block
  gen 0.02/0.03/0.55/0.53 s, refine -0.61/-0.61/0.22/-0.07 s; gen-step service latency 1.35-2.35 s (was 2.0-2.8), refine 2.5-3.8 s (was 2.4-4.9).
  Chunked variant (8 ms gap per ~100 ms of work): 12.75/13.37/13.26 (13.12): worse in gen steps (service latency 1.9-2.75 s) -> per-layer gaps kept.
  Early block with yields: refine +0.1 s, gen +0.5 s (the gen-step early block is no longer fully hidden in steps 8/10).
- Commit 32f7bb8 (off by default). Full check #8 (11:33 - 11:55) with check-7 set + OPT_GPU_YIELD=3000 + OPT_DECODE_SKIP_WARM=1:
  PASS, N0, speedup 1.8454 (/tmp/check-8/check.json): windows 11.94 / 12.67 / 12.57 s per step -> ratios 1.936 / 1.878 / 1.846;
  gate 2 bitwise PASS (14 steps, 3200 tensors, 235 reward requests). /submission/switches.json updated to this set.
- CPU sampler during the check (/tmp/work/cpu_samples_check8.txt): the 4 training ranks use ~4 cores in total all the time; the service bursts
  use 12-24 cores in the service processes plus short-lived DataLoader workers (machine total up to 60-80 of 96 logical cores): the
  service is CPU heavy by itself, my ranks hardly add to it.
7.3 hRound 8 — profile of the check-#8 configuration, early-block anatomy, allocator/prefetch (2026-11-06 11:56 - 13:04)
Profile (judge profile --from 14 --iters 2 of /submission/r3 + switches of check #8, steps 14 (gen) and 15 (refine), rank 0; profiler inflates ~10%):
- GPU busy per phase: gen: text decode 70-83%, denoise 98%, post-rollout 43-55% (yield gaps + wait for the reward service), backward 95%;
  refine: decode 70-80%, contexts 60%, denoise 98%, early block (pack + ref fwd + policy fwd) ~75%, backward 99%.
- Denoise kernel mix (gen rollout trace): GEMMs 80% of kernel time at ~750 TFLOPS (nothing to gain without changing GEMM kernels),
  flash 7.3%, silu+mul 5.6%, rms/qk-norm-rope 5% (already fused), gather/index/cat 1.6%.
- Decode graph: 777 kernels per step; kernel time 5.9 ms/step, intra-graph gaps 0.36 ms, per-step host gap ~0.85 ms (D2H sync + python + graph launch).
- Blocking allocator calls: cudaMalloc 0.53 s + cudaFree 0.29 s per pair of steps (empty_cache at the end of every rollout context + re-allocation).
- Training fwd+bwd kernels (earlier trace): GEMM 66%, flex attention 12.8%, dispatch index_put/gather/cat/copy ~9.5%; a microbenchmark showed
  index_copy only 10-30% faster than index_put per op (whole dispatch fusion would gain <1%): dropped.
Phase-timer fix: the PT copies run the early policy forward twice (the patch anchor matched the tail of a 20-space line, see apply_pt.py), so
absolute PT step times were ~0.7 s/step too high until pt10; comparisons between PT runs and the early-block numbers stay valid.
Early block anatomy with yields (PT, rank 0, relative to the reward submission): refine: pack 0.3-0.4 s, reference fwd 1.4-1.7 s,
policy fwd 1.2-1.4 s (3.0-3.5 s total, the service answers at 2.4-3.6 s); gen: pack ~0.05 s, ref fwd 0.6-0.7 s, policy fwd 0.6-0.9 s (1.2-1.6 s).
Uncontended service latency measured in situ (diagnostic run waiting for the answers before starting the early block): gen steps 8/10/12/14:
1.31 / 1.34 / 2.00 / 2.15 s, refine steps 9/11/13/15: 1.23 / 1.30 / 3.09 / 2.19 s => in refine steps 9, 11, 15 the early block (not the service)
is the critical path, in gen steps and refine step 13 the service is.
Runs (phase-timer tree, judge time; windows s/step): check-8 config 12.45/12.91/13.04 (with the double-forward bug);
+OPT_FSDP_PREFETCH=1 (corrected tree): 11.86/12.36/12.37 (ratios 1.949/1.925/1.876); +PYTORCH_ALLOC_CONF=expandable_segments:True
+OPT_NO_EMPTY_CACHE=1: 11.73/12.28/12.15 (1.971/1.937/1.910), peak reserved 122 vs 129 GiB, peak allocated 101 GiB, no graph-capture failures.
Full check #9 launched with check-8 set + these three switches (/tmp/work/sw_final2.json).

### Full check #9 (13:04 - 13:26) — PASS, N0, speedup 1.9153  (/tmp/check-9/check.json)
Tree commit 32f7bb8, switches = /submission/switches.json (check-8 set + OPT_FSDP_PREFETCH, PYTORCH_ALLOC_CONF=expandable_segments:True,
OPT_NO_EMPTY_CACHE). Gate 1: windows 11.73 / 12.38 / 12.11 s per step -> ratios 1.971 / 1.921 / 1.915. Gate 2: bitwise PASS (14 steps,
3200 tensors, 235 reward requests). Peak allocated 101.3 GiB, reserved 128 GiB (was 136). Repeat of the same configuration (accidentally
launched twice, 13:50): windows 11.50 / 12.15 / 12.12, speedup 1.9143, bitwise PASS (/tmp/check-10a.check.json) -> the score is reproducible
to ~0.1%; window 3 (steps 14-15) is always the minimum.
8.5 hRound 9 — dead ends of this session (13:26 - 14:15), nothing changes in the submission
- Pipelined decode loop (OPT_DECODE_PIPE, step t+1 launched before the host reads step t; device-side token hand-over, lagged bookkeeping,
  prototype in /tmp/work/tree-pipe, patch /tmp/work/pipe.patch): bitwise (harness 817/817, RNG states equal) but no speed-up: 385 decode
  steps 2.46-2.68 s plain vs 2.43-2.57 s pipelined (single GPU harness). The decode step is GPU bound (graph ~6.4 ms); the 0.85 ms "host gap"
  seen in the profiler trace is mostly profiler overhead. Not committed.
- OPT_FUSED_MLP (exact fused silu*mul) re-measured in the rollout harness: bitwise, but gen 5.0 -> 5.0 s, refine 9.79 -> 9.83 s, eval
  47.6 -> 48.9 s: no gain although silu+mul are 5.6% of the denoise kernel time (the Triton kernel is not faster than the two ATen kernels).
- Remaining controllable time per pair of steps (24.3 s): rollout 13.4 s (denoise 8.8 s at 98% GPU busy with GEMMs at ~750 TFLOPS, decode 4.3 s
  GPU bound, contexts 1.9 s), post-rollout 4.4 s (gen: reward service 1.3-2.2 s uncontended, refine: early block 3.1-3.5 s), backward+opt 6 s.
  Items still left, each <= 1% and not done: exact fused AdamW (opt step 81-97 ms -> ~25 ms), partial activation-checkpoint bypass for the last
  K layers (K/28 of the recompute, memory risk), block-mask sharing between the reference and the policy forward (~0.06 s per refine step),
  image-prefill dedup for the 4 identical refine inputs (short text prefills are NOT batch invariant: flash split heuristic depends on B*H*m_blocks),
  pack image-transform memoisation (~0.06 s), yield tuning (service-bound gen steps cannot gain: measured S' ~= S_idle).
- Background pre-capture of the smaller-M decode graphs (OPT_DECODE_PRECAPTURE, helper thread with its own stream, no_grad, device context,
  shorter GIL switch interval; prototype in /tmp/work/tree-pre): works and is bitwise (817/817 in the harness) but gives no speed-up
  (refine 9.79 -> 9.79-9.83 s, gen 5.00 -> 5.03-5.08 s): the helper's tracing competes with the decode loop for the interpreter. Not committed.
↑ R3