GLM-5.3 1.76×

Harness: Claude Code · switches: GS_OPT_FUSED_ADAM=1 GS_OPT_PIPELINE=1 GS_OPT_NO_EMPTY_CACHE=1 GS_OPT_SEPARATE_SHS=1 GS_OPT_LOG_EVERY=100 GS_OPT_NO_DEPTH=1 GS_OPT_SH_ADAM=1 GS_OPT_BWD_PIXCACHE=1 GS_OPT_GROUP_ADAM=1 GS_OPT_FAST_EXP=1 GS_OPT_ACT_FUSE=1 GS_OPT_STATS_FUSE=1 GS_OPT_CLAMP_FUSE=1 GS_OPT_TIGHT_RECT=1

Judged
1.76×
Tier
N1
Self-best
1.74×
Rounds
21
Hours
10.0
API cost
$55.2
Input tokens · 99.5% cached
187 M
Output tokens
1.2 M
Requests
1,354
Smaller tiles + rewritten binning
✗ tile size unchanged; last round tightened candidate rectangles GS_OPT_TIGHT_RECT=1 (radix sort −39%); binning one warp per Gaussian (1.55 → 1.58)
Bypass autograd, activations folded into raster kernel
✗ activations as a pair of standalone kernels outside rasterization GS_OPT_ACT_FUSE=1 (1.60 → 1.70)
Lazy Adam (update only visible Gaussians)
✗ dense; SH grad and Adam fused into one kernel GS_OPT_SH_ADAM=1 (first version regressed to 1.28, after three reorderings 1.34 → 1.46)
Fused loss / SSIM
only clamp backward fused (−11 µs / step); SSIM only changed to not zero its output
Other main levers
depth skipped, backward pixel state in shared memory (1.46 → 1.53), four small param groups in one Adam kernel, __expf, densification stats kernel (+0.003)
Raw work log21 sections
0.2 hRound 1 — profile + fused Adam (GS_OPT_FUSED_ADAM)
Start: 2026-11-07 05:31 (UTC-ish container time). End: (below)

Profile of the unchanged reference (/tmp/r1/profile, steps 7000–7019, N≈1.3M):
GPU busy ≈ 9.54 ms/step, window idle 16% (36.4/227 ms) — gaps at Adam.step/zero_grad, aten::item, aten::index/nonzero (4 nonzero syncs/step from the visibility .nonzero() + indexed updates).

Top per-step CUDA time:
| kernel | ms/step | share |
|---|---|---|
| PerGaussianRenderCUDA (backward) | 2.62 | 27% |
| Gaussian Adam (foreach multi-tensor ×7) | 2.52 | 26% |
| renderCUDA (forward) | 1.59 | 17% |
| preprocessCUDA fwd | 0.49 | 5% |
| duplicateWithKeys | 0.48 | 5% |
| cub radix sort | 0.22 | 2% |
| index/gather/scatter (densif. stats) | ~0.4 | 4% |
| fill/zero/misc | ~0.5 | 5% |

Change: scene/gaussian_model.py — torch.optim.Adam(..., fused=True) behind GS_OPT_FUSED_ADAM.
Tier: N1 (same Adam rule, different rounding/fusion order; state stays float32, step is 0-dim so the dtype gate item is unaffected).

Time (/tmp/r1/time-fused): speedup_vs_frozen_reference = 1.1432 (total_s 267.797 vs ref 306.144; windows 76.733 / 97.764 / 93.300 s; peak mem 5.31 GiB; run_status ok). Note window 3 < window 2 although N is larger: densification (every 100 iters until 15k) costs real time inside windows 1–2.

End of round 1: ~05:46. Committed as opt: fused Adam for the Gaussian optimizer (N1).
0.2 hRound 2 — host-sync / pipeline batch (GS_OPT_PIPELINE, GS_OPT_NO_EMPTY_CACHE)
Start: ~05:47. Class: host syncs + launch gaps (16% GPU idle in the profile, gaps at aten::item, aten::nonzero/aten::index, Optimizer.step/zero_grad boundaries).

Changes (each off by default):
- gaussian_renderer/__init__.py (GS_OPT_PIPELINE): visibility filter as bool mask instead of .nonzero() (removes 1 sync + gather/scatter sizing); means2D without the extra + 0 copy.
- gaussian_renderer/__init__.py (GS_OPT_SEPARATE_SHS): pass dc / rest SH separately to the rasterizer (skips the per-iteration cat of a ~250 MB tensor); same values, bitwise.
- train.py (GS_OPT_PIPELINE): EMA loss kept on GPU, read back every GS_OPT_LOG_EVERY=100 (no per-iteration loss.item()); training_report/elapsed_time only when it would do something; max_radii2D via torch.maximum (radii==0 for invisible, max_radii2D>=0 → identical).
- scene/gaussian_model.py (GS_OPT_PIPELINE): masked add_densification_stats (invisible Gaussians have exactly-zero mean2D grad → masked add adds zero; no gather/scatter, no sync). (GS_OPT_NO_EMPTY_CACHE): skip torch.cuda.empty_cache() in densify_and_prune (device sync + pool drop every 100 iters; changes no computed value).

Plan: check --quick with all switches on (exact items incl. densification steps), then time.

Quick check (/tmp/r2/quick, pipeline switches on, NO_DEPTH off): PASS (exact_items_pass), 3000 steps in 19.4 s, final count 1,095,984 (reference 1,092,971 — counts are not an exact item; the change steps are, and they match).
0.4 hRound 3 — skip the never-consumed inverse-depth path (GS_OPT_NO_DEPTH, CUDA)
Start: ~05:50. The recipe's cameras carry no depth maps (--depths '' → depth_reliable=False), so the depth term of the loss never runs: the rasterizer's invdepth image and its gradient chain are dead work, computed every iteration.

Changes (off by default; compute_invdepth threaded from GaussianRenderSettings into both kernels as a template parameter COMPUTE_DEPTH):
- forward.cu renderCUDA: skip expected_invdepth accumulation (a divide + fma per contributing pair), the sampled_ard checkpoint writes, the invdepth store, and the depths read in the pair loop.
- backward.cu PerGaussianRenderCUDA: skip the ard/dL_invdepth shuffle cascade, the sampled_ard/pixel_invDepths/dL_invdepths reads, the per-pair depth gradient terms (incl. a 1/(1-alpha) divide), and the final dL_dinvdepths atomicAdd.
- rasterizer_impl.cu: skip the pixel_invDepths D2D copy; out_invdepth stays the zero tensor it was allocated as.
- gaussian_renderer/__init__.py: compute_invdepth = not GS_OPT_NO_DEPTH or camera.depth_reliable (a camera with a real depth map still gets depth).
- Also in this round (pipeline class): scene/gaussian_model.py exposure optimizer _SkipAllNoneGradAdam — step() returns before the per-parameter bookkeeping while every grad is None (torch's Adam updates nothing in that case anyway; the global step hook wraps the class's step, so the step is still counted).

Numerics: the RGB image and every gradient that reaches the loss are value-identical (the removed terms were all multiplied by dL_invdepth = 0); possible ±0.0 sign differences in unused outputs. Tier N1.

Build: 3 attempts (2 signature plumbings fixed); /tmp/gs-ext/6fc0d54b5e15a16ad8ecbda2 builds ok.
Quick check (/tmp/r3/quick, all switches incl. NO_DEPTH): PASS (exact_items_pass), 3000 steps in 18.7 s, final count 1,094,792.

Time (/tmp/r3/time, all switches on): speedup_vs_frozen_reference = 1.3263 (total_s 230.821 vs ref 306.144; windows 63.078 / 85.104 / 82.639 s, each below the fused-Adam-only windows 76.7/97.8/93.3; peak mem 5.27 GiB).

Breakdown so far: fused Adam 1.1432 → +pipeline/no-depth/separate-SHs → 1.3263.

Full check (/tmp/r3/check, finished 06:10): status PASS, speedup 1.3275, tier N1 (candidate_s 230.611, windows 62.99/84.95/82.67). Statistical gate (seed 1, vs 3σ bounds): PSNR −0.13σ, SSIM +1.36σ, LPIPS −1.74σ, log_count +0.38σ — all within. This is the banked verdict for the R1–R3 configuration.
0.7 hRound 4 — remove the rasterizer's second per-iteration sync + render Python overhead
Start: 06:02 (validated 06:10–). Class: host syncs / launch gaps, N0 (bitwise-identical outputs).

- rasterizer_impl.cu: the sample state is sized by the worst-case bucket count (num_rendered/32 + num_tiles, a valid upper bound because Σ ceil(inst/32) ≤ floor(Σ/32) + T), so the bucket_sum device-to-host read that blocked the host between the bucket scan and the render launch is gone; the forward returns the capacity.
- backward.cu PerGaussianRenderCUDA: warps beyond the true bucket count stop on a device-side read of the inclusive-scan total (per_tile_bucket_offset[num_tiles-1]), so the oversized grid is safe.
- gaussian_renderer/__init__.py (GS_OPT_PIPELINE): call rasterize_gaussians(...) directly instead of constructing a GaussianRasterizer nn.Module (and re-validating its args) every iteration; empty_like for the 2D means buffer (its values are never read — the rasterizer recomputes them and only the gradient of that input is consumed).

Quick check (/tmp/r4/quick): PASS (exact_items_pass), 3000 steps in 18.5 s, final count 1,094,097 (counts vary run-to-run within the atomics noise floor; the change steps are the exact item and match).

Time (/tmp/r4/time, all switches on): speedup_vs_frozen_reference = 1.3409 (total_s 228.305 vs ref 306.144; windows 63.132 / 83.908 / 81.265; R3 windows were 63.078 / 85.104 / 82.639 → the gain is in windows 2–3, the post-densification steady state; peak mem 5.27 GiB).

End of round 4: ~06:19. Committed as opt: capacity-sized sample state, no bucket_sum sync; direct rasterize_gaussians call (N0).
1.1 hRound 5 — fused SH-gradient + Adam for f_dc/f_rest (GS_OPT_SH_ADAM)
Start: ~06:25. Class: memory traffic of the SH-coefficient gradient chain, tier N1.

Profile of the R4 tree (/tmp/r5/profile, steps 7000–7019, P≈1.28M): GPU busy ≈ 7.8 ms/step vs 7.6 ms/step wall → the sync removals left essentially no idle; the remaining work is kernel time. Per step: PerGaussianRenderCUDA (bwd) 2.348 ms (30%), renderCUDA (fwd) 1.392 (18%), fused Adam 1.130 (14.5%), preprocess bwd 0.489 (6.3%), duplicateWithKeys 0.484 (6.2%), radix sort 0.225, zeros/fill ~0.30, SSIM 0.134, norm+div ~0.19.

Change (GS_OPT_SH_ADAM, off by default): the SH gradient of a point is basis_j(dir) * (clamp-masked per-point color gradient) — exactly what BACKWARD::preprocess materializes into dL_ddc/dL_dsh (P×1×3 + P×15×3 ≈ 240 MB zeroed + written every step, then read back by Adam). The fused path:
- backward.cu: computeColorFromSH/preprocessCUDA skip the dL_ddc/dL_dsh writes when fusing (the dL_dmeans direction term still runs, so shs is still read).
- rasterize_points.cu: in fused mode the two gradient tensors are 1-element stand-ins (their zeroing disappears); the backward returns the byte offset of clamped in the geom buffer.
- adam.cu (new ADAM::shAdamUpdate): a 2-thread prep kernel advances the two step counters on device (as torch's fused kernel does) and precomputes the bias corrections; the main kernel walks P×(M·3+3) elements, recomputes the SH basis per element (same formulas/values as computeColorFromSH, gated by the active degree like the original's deg > 0/1/2 nesting), and applies torch's fused-Adam update (m = m·β1 + g·(1−β1), v = v·β2 + g²·(1−β2), p −= (lr/bc1)·m/(√v/√bc2 + eps)). Elements beyond the active degree and invisible points get g = 0, exactly the zeros the reference materializes; every element is updated (dense, like the reference).
- diff_gaussian_rasterization/__init__.py: in fused mode the backward stashes (dc, sh, means3D, dL_dcolors, geomBuffer, clamped offset, campos, degree) and returns None as the dc/sh input grads (so .grad stays None); consume_sh_grad(f_dc, f_rest) hands the stash to the optimizer when it matches exactly those parameter tensors.
- scene/gaussian_model.py (_ShFusedAdam, used when both Adam switches are on): step() runs the fused kernel for the f_dc/f_rest groups (torch-style lazy state init, float32 step/exp_avg/exp_avg_sq) and then calls super().step() for the other four groups. When densification/pruning/opacity-reset has replaced a parameter tensor since the backward, the reference optimizer sees a None gradient and skips (no update, no step increment); the identity check reproduces that exactly.

Numerics: the gradient values are the same formulas on the same inputs (per-point, no atomics, same nesting) → bitwise-equal g up to compiler FMA choices; the Adam update replicates torch's fused kernel formula (powf-based bias corrections), so ulp-level differences are possible → N1, same class as the R1 foreach→fused switch that passed gate 3 with margin.

Traffic saved ≈ 0.7 GB/step (dL_dsh/dL_ddc zero + write + Adam grad read), expected ~0.4 ms/step.

Build: first attempt failed (missing step_sh parameter in the binding definition — fixed); second attempt ok, but the first quick check ERROR (illegal memory access at the first optimizer step).

Debugging (repro at /tmp/r5/repro.py passed; the judge's own entry reproduced it under CUDA_LAUNCH_BLOCKING=1; compute-sanitizer only surfaced 660 pre-existing benign shared-memory OOB writes inside stock fused-ssim): an env-gated dump of the exact kernel inputs showed clamped_off = -10203508 — negative. GeometryState::fromChunk takes char*& and *advances* the chunk pointer, so the offset I returned was measured from the chunk *end*. Fixed by capturing the buffer start before fromChunk (rasterizer_impl.cu), rebuilt.

Unit parity (/tmp/r5/parity.py): my kernel vs torch fused Adam given the identically materialized gradient, 3 steps, P=5000, all degrees: max |Δparam| = 2.98e-08 (≈1–3 ulps, rel ~7e-08), step counters identical — the update rule matches torch's up to compiler rounding, and the gradient values are the same formulas on the same inputs (one 5.6% outlier in the first run turned out to be a wrong SH_C3 sign *in my test script*, not in the kernel). Tier N1.

Quick check 2 (after the offset fix, /tmp/r5/quick2): PASS (exact_items_pass), 3000 steps ok — the exact items (camera sequence, densification steps, lrs, dtypes) are unchanged by the fusion.

Time (/tmp/r5/time): speedup_vs_frozen_reference = 1.2777 — a REGRESSION vs R4's 1.3409 (total_s 239.608 vs 228.305; windows 65.897 / 87.670 / 86.042 vs R4's 63.132 / 83.908 / 81.265 — every window slower, the gap growing with the point count: a per-element kernel cost).

Why it lost (working theory, to be confirmed by the R6 profile): the fused kernel parallelizes over the P×(M·3+3) ≈ 61M elements, so j = e/3 varies *within every warp* and the 15-way if-chain that evaluates the SH basis diverges — each warp serially executes most of the 15 branch bodies (~15× the basis ALU). The traffic the fusion saved (~0.4 GB/step ≈ 0.2 ms) is smaller than that divergence penalty. A branch-free (predicated-arithmetic) basis, or a thread-per-point layout like the original preprocessCUDA (sequential over j ⇒ no divergence), should bring the kernel to ~0.5 ms vs the ~0.63 ms it replaced — a ceiling of only ~+0.1 ms/step, so the fix is deferred behind bigger fish.

Decision: GS_OPT_SH_ADAM removed from /submission/switches.json (back to the banked 6-switch config, time 1.3409); the code stays in the tree, off by default, committed for a later cheap retry with a branch-free basis.
1.5 hRound 6 — fix the fused SH-Adam kernel layout (GS_OPT_SH_ADAM, retry)
Start: ~06:58. Class: same hotspot as R5 (the fused SH-gradient + Adam kernel), now with the profile evidence.

Profile (/tmp/r6/profile, R5 tree with all 7 switches, steps 7000–7019): per step — PerGaussianRenderCUDA (bwd) 2.478 ms (30.1%), renderCUDA (fwd) 1.487 (18.1%), shAdamCUDA 1.548 (17.9%), duplicateWithKeys 0.473, torch fused Adam on the other 4 groups 0.298, radix sort 0.249, preprocessCUDA fwd 0.143, div 0.142, preprocessCUDA bwd (SH-write skipped) 0.114, computeCov2D 0.050, SSIM fwd+bwd 0.134.

Root cause of R5's regression confirmed: the kernel is element-parallel (one thread per coefficient), so j = e/3 differs across the lanes of every warp and the 15-way SH-basis if-chain diverges — each warp serializes nearly all 15 branch bodies. A traffic-bound version of this kernel should cost ~0.5 ms, not 1.55.

Bug found while reviewing (latent in R5, never enabled in a scored run): shAdamPrepCUDA used one learning rate for both groups, but upstream gives f_rest feature_lr/20 (0.000125 vs f_dc's 0.0025) — a 20× error on the rest coefficients. The exact-items gate cannot see it (it checks the groups' Python-side lrs), and the R5 parity test used equal lrs for both groups, so it slipped through; gate 3 would have rejected it. Fixed: the binding takes lr_dc and lr_sh, the prep kernel writes each group's own bias-corrected step size, and gaussian_model.py passes gdc['lr'], grst['lr']. The new parity test (/tmp/r6/parity2.py) uses the two different upstream lrs.

Change: adam.cu — one thread per (point, color channel); the basis index j becomes a sequential loop variable identical across the warp (no divergence; with M=15 known at compile time the loop unrolls and each basis formula constant-folds). Same gradient values (identical formulas on the same inputs), same torch fused-Adam update rule, same step-counter and densification-skip semantics. Grid 3P threads instead of P·(M·3+3); traffic unchanged (~1.5 GB/step) but now the intended ~0.5 ms. Tier N1.

Plan: parity → check --quick (7-switch temp config /tmp/r6/sh_switches.json) → time; if it beats 1.3409, put GS_OPT_SH_ADAM=1 back into /submission/switches.json and commit.

Iteration 1 (thread per (point, channel), sequential warp-uniform j loop): build hiccup — rasterize_points.h still had the old arity (undefined symbol at import; ext.cpp binds against the header), fixed. Parity (/tmp/r6/parity2.py, now with the two different upstream lrs): dc max|Δ| 2.98e-08 (~1-3 ulps), rest ≤3.7e-09, step counters identical, at every degree. Quick (/tmp/r6/quick): PASS. Time (/tmp/r6/time): 1.3266 — even with R4's 1.3409 (230.78 s; windows 62.893/84.956/82.931 vs R4's 63.132/83.908/81.265: the fused path costs slightly more at full size).

Profile of iteration 1 (/tmp/r6/profile2): shAdamCUDA = 1.462 ms/step — barely better than the divergent 1.548. The divergence theory was wrong (ptxas had evidently predicated the old chain anyway); ptxas: 48 registers, 0 spills, so occupancy was fine.

Bandwidth measurement (synthetic, flocked): a 460 MB device copy runs in 0.111 ms → this GPU sustains ~4.1 TB/s. The kernel moves only ~1.31 GB, i.e. it *should* take ~0.35 ms, not 1.46. Diagnosis: with one thread per (point, channel), every warp instruction touches eleven 12-byte clusters (2.75× sector amplification), and each warp's 15-iteration working set is ~5.9 KB × 3 arrays; at ~64 warps/SM that is ~380 KB — it thrashes the 256 KB L1, so the amplification goes all the way to DRAM (~3.6 GB effective).

Iteration 2 (element-parallel, branch-free ternary basis, dc/rest split inside the kernel): build ok, parity 1–3 ulps (dc 2.98e-08, rest 3.7e-09, steps identical), but the in-training profile showed shAdamCUDA = 1.322 ms — still 2× its ~0.6 ms of traffic (1.38 GB at ~4.1 TB/s measured streaming).

Bisect (standalone replica /tmp/r6/microbench2.cu = verbatim kernel + harness → 1.175–1.192 ms/call, reproduces the slowness; variants in mb_a/b/c.cu, 30 s compiles):
| variant | ms/call | conclusion |
|---|---|---|
| replica (verbatim) | 1.18 | — |
| A: dc branch deleted (if (true), sh-only) | 0.962 | the dc-vs-rest split costs ~0.22 ms (mixed warps execute both paths serially) |
| B: int idx instead of long long | 1.178 | 64-bit indexing is free |
| C: constant dir/grad/prep (no input loads, no normalize) | 0.704 | per-element direction recompute + input loads cost ~0.48 ms |

The per-element kernel recomputed the normalized view direction (3 amplified means3D loads + sqrtf + three IEEE divides) for every one of the ~61M coefficients instead of once per point — 48× redundant — and the amplified-input theory (2.75× sector amplification thrashing L1, iteration 1) was wrong: it is plain redundant ALU+loads.

Iteration 3 (the fix): hoist the per-point invariants into a new shAdamDirCUDA (one thread per (point, channel), P×3 ≈ 3.8M threads) that computes the normalized direction and the clamp-masked color gradient once per point (bitwise the same values — a float32 store/load round-trip is lossless) into a workspace, and folds the DC update into the same kernel (its basis is the constant SH_C0, so it needs no direction; this also removes the dc/rest branch from the main kernel). The main shAdamCUDA is now sh-only, P×(M·3) threads, thread index = the tensor's flat index (fully coalesced RMW), reading dir/gcol from the workspace. The workspace (P×6 floats ≈ 30 MB) is allocated inside the C++ binding per call (caching allocator, stream-ordered free — Python-facing signature unchanged). ptxas: 26/22 registers, 0 spills.

- Parity (/tmp/r6/parity2.py, extension cf278ebc893ad3e3a9111d73): identical to iteration 2 — dc max|Δ| 2.98e-08, rest ≤3.7e-09, steps identical, every degree. Tier N1.
- Quick check (/tmp/r6/quick3, 7 switches): PASS (exact_items_pass).
- Time (/tmp/r6/time3): speedup_vs_frozen_reference = 1.4605 (total 209.623 s; windows 58.064 / 76.963 / 74.596 vs R4's 63.132 / 83.908 / 81.265 — every window down ~5–7 s ≈ 0.62 ms/step; peak allocated 5.68 GiB).

Decision: GS_OPT_SH_ADAM=1 added to /submission/switches.json (7 switches). End of round 6: ~07:48 (long round — the bisect needed it; logged in full above).
2.8 hRound 7 — shared-memory staging of the backward's per-pixel accumulators (GS_OPT_BWD_PIXCACHE)
Start: ~07:57. Class: the #1 hotspot, PerGaussianRenderCUDA 2.48 ms/step (30%). The kernel launches one 32-thread block per bucket; every cascade iteration lane 0 issues ~10 scalar global loads (sampled_T, sampled_ar×3, pixel colors×3, final_T, n_contrib, dL_dpix×3) that feed the next iteration — a long latency chain where each 4-byte load drags a 32-byte sector (8× amplification), and the pixel-indexed ones are re-read by every bucket of the same tile.

Change (GS_OPT_BWD_PIXCACHE, off by default): PerGaussianRenderCachedCUDA — a 256-thread block covers 8 consecutive buckets; each active warp stages its own per-pixel checkpoint (4 KB) coalesced into shared memory, and the block stages the tile's per-pixel data once for all 8 warps. Values identical (lossless float32 staging, same math) → N0 up to atomic-order rounding. Depth-enabled cameras keep the original kernel (also keeps static smem at 42 KB < the 48 KB limit; 62 registers). pixel_cache threaded GaussianRenderSettings → binding → Rasterizer::backward → BACKWARD::render; use_cache falls back to the original global loads when a block's active warps span two tiles.

First attempt — ERROR at step 10701 (/tmp/r7/time): illegal memory access after a densification step. A/B with the judge's entry (traj mode, 3100 steps, seed 0) showed the training had diverged: final count 10,432,987 vs the switch-off 945,796. Root cause: the per-warp staging wrote s_T[warp_id][threadIdx.x] — each warp staged only the 32 entries at threadIdx.x ∈ [warp_id·32, warp_id·32+32) of its own 256-entry array; 224 of 256 checkpoint entries per bucket were uninitialized shared memory. Garbage T/ar → garbage gradients → runaway densification (the quick gate still PASSED — it checks the *steps* at which the count changes, not the counts; a lesson: check --quick cannot see gradient-value bugs). The garbage params then made the next forward's tile ids undefined → the fault. Fix: each warp's 32 lanes walk e = lane, lane+32, …, lane+224 (all 256 entries, coalesced).

Value validation (the quick gate can't see it, and the backward's atomics make bitwise comparison impossible even run-to-run): three 500-step entry runs — off, off, on (seed 0, before densification). Pairwise max|Δparam|: off-off (pure atomics noise) 1.26e-01/7.3e-01/4.1/1.19/4.9e-01 (xyz/dc/opacity/scaling/rotation) vs off-on 1.22e-01/8.3e-01/4.0/1.16/4.6e-01 — same magnitude on every parameter → the cached kernel is value-equivalent within the noise floor (N0 in exact arithmetic; differences are atomic-order rounding, the same class as any rerun).

- Quick check (/tmp/r7/quick2): PASS.
- Time (/tmp/r7/time2): speedup_vs_frozen_reference = 1.5276 (total 200.404 s; windows 55.811 / 73.431 / 71.162 vs R6's 58.064 / 76.963 / 74.596 — every window down, ~0.31 ms/step; peak allocated 5.68 GiB).

Decision: GS_OPT_BWD_PIXCACHE=1 added to /submission/switches.json (8 switches). End of round 7: ~08:16.
3.1 hRound 8 — conservative exp pre-cull in both render kernels (GS_OPT_EXP_CULL) — NO GAIN, REVERTED
Start: ~08:28. Class: MUFU/exp throughput in the render kernels (forward renderCUDA 1.38 ms, backward cached 2.10 ms; ~355M walked (pixel, instance) pairs per step, each computing exp(power) before the alpha < 1/255 blend test).

Change: skip the exp when power < log(1/(255·opacity)) − 0.01 — a conservative bound that *implies* the original continue (1% margin covers float rounding), so the execution is bitwise-identical (N0); the threshold is one logf per instance (staged in shared, forward) or per splat (registers, backward), not per pair. Plumbed as exp_cull (runtime bool) through settings → both bindings → Rasterizer::forward/backward → both render kernels, behind GS_OPT_EXP_CULL.

- A/B vs noise floor (500 steps, off/off/on, seed 0): off-on deltas at the run-to-run atomics level on every parameter (counts identical) — value-safe.
- Quick check (/tmp/r8/quick): PASS. Time (/tmp/r8/time): 1.5207 vs the banked 1.5276 — a slight regression (201.316 s; windows 55.845 / 73.810 / 71.661 vs 55.385 / 73.532 / 71.486).
- Why it lost: pairs with power ≤ 0 are mostly *inside* the Gaussian's effective ellipse, where opacity·G passes the 1/255 threshold — the power > 0 test already did the spatial culling, so the extra compare + shared load per pair cost more than the few saved exps. The "85–95% culled" estimate was wrong.
- Decision: reverted to the banked tree (git checkout, back at commit 9996eae; the switch was never in switches.json). Dead code that adds a per-pair compare even when off is a pure cost. End of round 8: ~08:40.
3.3 hRound 9 — occupancy of the cached backward kernel (launch_bounds) — NO GAIN, REVERTED
Start: ~08:42. The cached backward runs 62 registers → 4 blocks/SM = 32 warps (half the SM's warp slots), and its 287-iteration cascade is a serial per-warp latency chain — more resident warps should hide more latency. (The forward renderCUDA is already at full occupancy: 32 regs, 8.2 KB smem → 8 blocks/SM.)

Change: __launch_bounds__(256, 5) on PerGaussianRenderCachedCUDA (scheduling only, no value change → N0). ptxas: 48 registers + 16 B stack (spills).

Time (/tmp/r9/time, banked 8-switch config): 1.4713 — a big regression (208.076 s vs 200.404; +7.7 s). The spills land in the hot cascade and 5 × 42 KB of smem nearly fills the SM's 227 KB; the natural 62-register allocation at 4 blocks is the better operating point. Reverted (tree back at 9996eae). End of round 9: ~08:49.
3.5 hRound 10 — fused Adam for the four dense groups (GS_OPT_GROUP_ADAM)
Start: ~08:50. Class: the optimizer step (torch's fused Adam on xyz/opacity/scaling/rotation costs 0.29 ms/step over ~330 MB of traffic — 3.6× its bandwidth floor, one multi-tensor launch per parameter).

Change (GS_OPT_GROUP_ADAM, off by default): groupAdamCUDA — one launch for all four groups, each *block* owning exactly one group (block-uniform selection; the R6 bisect showed a per-lane group split serializes). A 4-thread prep kernel advances the groups' step counters on device and writes each group's bias-corrected step size (per-group lrs: xyz's is scheduled, read from the group at step time). Same torch fused-Adam formula as the SH kernels. scene/gaussian_model.py: _ShFusedAdam.step runs it when all four groups carry a gradient (the common case) and then returns without super().step() when the SH path also ran; on densification steps (replaced tensors → None grads) it falls back to torch's Adam, which skips them exactly like the reference. Mixed configs (group on, SH off) mask the four grads around the fallback so nothing is updated twice.

- Parity (/tmp/r10/parity_grp.py, P=5000, 3 steps, per-group upstream lrs): xyz 2.98e-08, opacity 1.49e-07, scaling 5.96e-08, rotation 2.98e-07... rotation 2.98e-08 — all 1–3 ulps, step counters identical. Tier N1.
- A/B (500 steps, off×3 vs on): counts identical; off-on deltas 1.1–3× the off-off noise floor (xyz 0.31 vs 0.13–0.17, opacity 9.3 vs 3.8–4.9) — the expected ulp-compounding signature of an N1 optimizer change, not a gross bug; gate 3 is the verdict.
- Quick check (/tmp/r10/quick): PASS.
- Time (/tmp/r10/time): speedup_vs_frozen_reference = 1.5534 (total 197.076 s vs 200.404; ~0.11 ms/step off the optimizer).

Decision: GS_OPT_GROUP_ADAM=1 added to /submission/switches.json (9 switches). End of round 10: ~09:00.
3.8 hRound 11 — occupancy of duplicateWithKeys (launch_bounds) — NO GAIN, REVERTED
Start: ~09:05. duplicateWithKeys costs 0.48 ms/step at 53 registers → 4 blocks/SM = 32 warps (half the warp slots). One thread per point walks its tile rect serially (avg ~16 tiles/point → ~800 dependent instructions per thread), so more resident warps could hide that chain.

Change: __launch_bounds__(256, 5) on duplicateWithKeys (scheduling only, values unchanged → N0). ptxas: 48 registers + 8 B stack.

Time (/tmp/r11/time, banked 9-switch config): 1.5537 (197.041 s) — a wash vs the banked 1.5534/1.5592. The occupancy bump bought nothing: the kernel is bound by its serial per-thread instruction chain, not by resident-warp latency. Reverted (git checkout, tree back at commit 3b134bf). End of round 11: ~09:12. Conclusion: occupancy levers are exhausted on both render kernels and the binning kernel — the remaining win in duplicateWithKeys is to shorten the serial chain itself (lane-parallel rect walk, R12).
4.1 hRound 12 — lane-parallel duplicateWithKeys (one warp per Gaussian)
Start: ~09:14. Class: the binning kernel — 0.48 ms/step, one thread per Gaussian walking its tile rect serially (avg ~16 tiles/point ≈ 800 dependent instructions per thread; R11 showed occupancy is not the bottleneck, the serial chain is).

Change (kernel-internal, unconditional — bitwise-identical output): one warp per Gaussian (8 points per 256-thread block, grid (P+7)/8). Lanes walk the rect in row-major rounds of 32 (t = 32r + lane → y = rect_min.y + t/rect_w, x = rect_min.x + t − y·rect_w), each lane runs the same closed-form max_contrib_power_rect_gaussian_float per-tile opacity test, and a __ballot_sync/__popc compaction writes the passing tiles at off + popc(mask & lanes-below-me) — which is exactly the original row-major emission order, so the unsorted key/value arrays are bitwise identical. Sentinel padding of the unused slots (the identifyTileRanges guard skips tile-id 0xFFFFFFFF, so sentinels are never walked) is lane-strided. 39 registers, 0 spills. ptxas note: __lanemask_lt is not declared in this CUDA — used (1u << lane) − 1 instead.

Validation (stronger than the usual noise-floor A/B, because this change claims bitwise identity):
- *Bitwise forward test*: rendered the 6 cameras 0/1/3/9/40/120 from the 500-step model state with the old extension (twice) and the new one — images, radii and depths bitwise identical in all three pairings (the image is a deterministic function of the sorted binning output, so the unsorted arrays are proven identical). /tmp/r12/bw_*.pt.
- *500-step A/B* (traj, seed 0, off/off/on): identical counts (136029) and SH degree; off-on max|Δparam| at or below the off-off atomics noise floor on every parameter (dc 0.17–0.34 vs 0.34; opacity 1.8–2.5 vs 2.5; scaling 0.97–1.06 vs 1.05; rotation 0.43–0.46 vs 0.46). Note: in this configuration xyz is bitwise identical even off-off, so it carries no signal.
- Quick check (/tmp/r12/quick): PASS.
- Time (/tmp/r12/time): speedup_vs_frozen_reference = 1.5834 (total 193.342 s vs 197.076; windows 52.484/71.669/69.189 vs 53.913/72.359/70.069 — every window down, ~0.12 ms/step).

Decision: committed as 7526a5c (unconditional kernel change, tier N0 — no switch needed: identical arrays, no Python surface). Full check (/tmp/r12/check): PASS, speedup 1.5773, tier N1 (PSNR −1.55σ, SSIM +0.93σ, LPIPS −1.24σ, log_count −2.15σ; windows 52.831/71.836/69.428). End of round 12: ~09:45.
4.7 hRound 13 — fast exp (__expf) in the render hot loops (GS_OPT_FAST_EXP)
Start: ~09:45 (profile) → implementation 09:50–10:05. Class: the #1/#2 hotspots — forward renderCUDA 1.40 ms + cached backward 2.10 ms; both compute the ACCURATE multi-instruction expf per walked (pixel, instance) pair (~355M pairs/step each). The R13 profile (steps 7000–7019, /tmp/r13/profile) also showed the torch elementwise/launch soup (fills ×24, zeros ×13, div ×9, norm ×2 per regular step; DivBackward+NormBackward+ExpandBackward 232 µs — the sigmoid/exp/normalize activation chain) and that the aten::index/nonzero spike (~60 calls, 683 µs gathers) is the densify step only (amortized ~120 µs/step).

Change (GS_OPT_FAST_EXP, off by default): __expf(power) (one MUFU.EX2, ≤2 ulp) instead of exp(power) in the forward renderCUDA and both backward cascade kernels, selected as a compile-time FAST_EXP template variant from a new runtime GaussianRenderSettings.fast_exp flag (plumbed like pixel_cache: NamedTuple → binding → Rasterizer → FORWARD/BACKWARD::render explicit-branch dispatch — a runtime bool cannot feed a template argument). Tier N1 (render values change by ~2 ulp).

Validation:
- *Render sanity*: images from the 500-step state differ by mean 2.3e-8 (≈2 ulp; max up to 5.7e-4 where hundreds of splats composite), radii identical — no systematic shift (/tmp/r13/bw_fastexp.pt vs /tmp/r12/bw_old1.pt).
- *500-step A/B* (same build, env off/off/on): identical counts (136029); off-on deltas at the off-off noise floor (dc 0.14–0.16 vs 0.134, opacity 1.89 vs 1.98, scaling 1.22–1.29 vs 1.33, rotation 4.9–5.1 vs 4.3).
- Quick check (/tmp/r13/quick, switch ON): PASS — the count-change steps over the first 3,000 steps still match the reference.
- Time (/tmp/r13/time): speedup_vs_frozen_reference = 1.6039 (total 190.878 s vs 193.342; ~0.08 ms/step off the render kernels — less than the optimistic estimate; the kernels are not purely exp-issue-bound).

Decision: committed as 36d01e2; GS_OPT_FAST_EXP=1 added to /submission/switches.json (10 switches). Full check (/tmp/r13/check): PASS, speedup 1.6125, tier N1 (PSNR +0.45σ, SSIM +0.41σ, LPIPS −1.19σ, log_count −1.37σ — every endpoint comfortable, PSNR/count moved *toward* the reference vs R12's −1.55σ/−2.15σ; more evidence the single-seed σ readings wobble ±0.5σ regardless of the change). End of round 13: ~10:20.
5.0 hRound 14 — fused activations (sigmoid/exp/normalize) forward+backward (GS_OPT_ACT_FUSE)
Start: ~10:20. Class: the torch elementwise/launch soup around the rasterizer — the R13 profile showed the activation chain (sigmoid on _opacity, exp on _scaling, normalize on _rotation, forward + Sigmoid/Exp/Div/Norm/Expand backward) at ~350 µs/step across ~10 launches (DivBackward 113 µs + NormBackward 71 µs + ExpandBackward 48 µs + forward norm/div/sigmoid/exp).

Change (GS_OPT_ACT_FUSE, off by default): cuda_rasterizer/act.cu — actForwardCUDA (one thread per point: sigmoid, exp×3, quaternion normalize with torch's clamp_min(1e-12)) and actBackwardCUDA (composite of torch's backward chain, clamp branch included), bound as _C.act_forward/_C.act_backward, called from a _FusedActivations autograd Function in render() (saves op/sc/raw_ro; backward recomputes the norm). Tier N1 (backward ~1-2 ulp; forward bitwise except 1 ulp on the normalize).

- Parity (/tmp/r14_parity.py, P=20000): forward op/sc bitwise, ro max|diff| 1.79e-07 (norm reduction order); backward rel 7.0e-08 (op), 0 (sc), 1.74e-07 (ro); tiny-norm (< eps) path exact.
- 500-step A/B (same build, env off/off/on): identical counts (136029); off-on deltas at/below the off-off noise floor on every parameter.
- Quick check (/tmp/r14/quick, switch ON): PASS.
- Time (/tmp/r14/time): speedup_vs_frozen_reference = 1.6972 (total 180.377 s vs 190.878; ~0.35 ms/step — above the kernel-time estimate, the removed launches were also stalling the pipeline).

Decision: committed as 7da8173; GS_OPT_ACT_FUSE=1 added to /submission/switches.json (11 switches). Full check (/tmp/r14/check): PASS, speedup 1.6942, tier N1 (PSNR −2.20σ, SSIM −0.51σ, LPIPS −2.12σ, log_count −0.79σ). A second seed-1 draw (check --n1, /tmp/r14/n1b) gave PSNR −1.26σ, SSIM −0.42σ, LPIPS −1.74σ, log_count −1.90σ — the identical-code wobble is ±0.5–0.9σ per endpoint, so the act-fuse systematic is roughly PSNR −1.7σ, LPIPS −1.9σ, count −1.3σ, leaving ~1.1–1.3σ of margin. EV keeps the change (+0.09 speedup vs a few % gate-3 risk); going forward prefer N0 changes and watch the margins in every full check. End of round 14: ~10:40.
5.6 hRound 15 — shAdamCUDA restructure (one thread per (point, basis)) — NO GAIN, REVERTED
Start: ~10:40 → end ~10:55. Class: the fused SH-Adam kernel (R6's, ~640 µs/step at ~30k Gaussians) — its per-thread serial loop over the 15 rest coefficients plus the if-chain SH basis derivative looked issue-bound.

Change (no switch; kernel-internal): one thread per (point, basis-element) — thread (p, b) handles all 3 channels of basis b for point p, basis polynomial evaluated once per thread. Parity (/tmp/r15/parity_sh.py): not bitwise — sh 1.86e-9, m_sh 7.45e-9, v_sh 3.7e-9 max|diff| (FMA contraction in the basis polynomials; ~1 ulp), everything else 0.

Results: 500-step A/B deltas at the off-off noise floor (but that window only exercises the sh path at degree 0); 1600-step A/B: count-change step lists identical 11/11 while counts diverge even off-off (501812 vs 503105 — the atomics noise flips marginal densify decisions). Quick check PASS. Time (/tmp/r15/time): 1.6527 = −0.045 REGRESSION vs 1.6972.

Diagnosis: the kernel is memory/L1-bound, not issue-bound — ~1.05 GB of traffic per call (a ~350 µs floor on this GPU) vs 640 µs measured. The (p, b) layout made the gathers stride-3 (thread b reads channel-interleaved floats), quadrupling L1 transactions; the instruction savings were dominated. A precompute-the-basis variant would ADD traffic. Reverted (git checkout of adam.cu; tree back at 7da8173, whose sources hash to the cached R14 build — no rebuild needed).
5.6 hRound 16 — densify-step host-sync reduction (hoisted mask indices)
Start: ~10:55 → end ~11:10. Class: the densify step's GPU-idle gaps — the R15 trace showed a 16.43 ms span (12.04 ms GPU busy + ~4.4 ms gaps) every 100 steps in the densify window, with ~60 aten::nonzero host-syncing calls per event: every boolean-index gather t[mask] recomputes nonzero() and re-syncs for the output size (16 gathers in densify_and_clone/densify_and_split, 18 in _prune_optimizer, 4 in prune_points, the sum() readback for the prune_filter pad size, and the bool-mask grads[isnan] = 0 assignment).

Change (commit a4cfc6e, no switch — value-identical code path, tier N0): each event computes its index once (mask.nonzero().squeeze(-1)) and every gather uses t[idx] — same values, same ascending order; prune_points passes its index into _prune_optimizer; the pad size comes from selected_idx.shape[0] (host int, no readback); grads[grads.isnan()] = 0.0 → torch.where(torch.isnan(grads), 0.0, grads) (a bool-mask index_put_ also converts via nonzero → sync). ~60 syncs → ~5 per event.

Bug caught by the A/B before any judge run: nonzero() on a 1-D mask returns shape (K, 1) — indexing with it added a trailing dim ((K,1,3) params) and the run died at the first densify ("Tensors must have same number of dimensions: got 2 and 3", step 600). .squeeze(-1) fixes it; the 800-step A/B then completed.

Validation: 800-step A/B (on/off, same build): count-change steps identical (600/700/800 both); count values differ by 1 at the first event and drift afterwards — exactly the established off-off atomics pattern. Quick check (/tmp/r16/quick): PASS. Time (/tmp/r16/time): 1.7009 (windows 48.86/66.69/64.43 vs R14's 49.12/66.79/64.47 — the densify-free window 3 moved −0.04 s = the noise floor, so the real gain is the −0.26 s in w1 + −0.10 s in w2 ≈ 2.4 ms per densify event, ~60% of the trace's gap estimate; the remaining stall is the ~5 surviving syncs plus the gather/cat/nn.Parameter-rewiring soup between them).

Decision: keep (N0, no gate-3 exposure, small but consistent in the affected windows). No full check yet — +0.004 alone doesn't justify holding the GPU ~15 min; bank more N0 gains and run one full check over the accumulated set.
6.2 hRound 17 — densify gather cost: index_select (no gain) + warp-per-row gather_rows kernel + a REAL BUG FIX (frozen xyz)
Start: ~11:05 → end ~11:45 (overran the 30-min pace; the bug discovery forced the full check). Class: the densify event's gather kernels — the R17 profile (/tmp/r17/profile, steps 7000–7019, one densify event) put aten::index at 9.37 ms of CUDA time per event (60 calls; vectorized_gather_kernel<16> 9 × 679 µs on the 45-wide f_rest rows ≈ 90 GB/s).

Attempt 1 — torch.index_select: no gain — profile2 (/tmp/r17/profile2) shows index_select dispatches to the *same* per-element gather kernel (8.41 ms, vectorized_gather 6.11 nested). Time 1.6951 ≈ noise.

Attempt 2 — custom gather_rows (cuda_rasterizer/gather.cu, commit 925bf15): one warp per output row, lane-strided coalesced 4-byte transfers; bitwise parity verified on GPU over all site shapes (N,)/(N,1)/(N,3)/(N,4)/(N,1,3)/(N,15,3), keep-fractions 0–1, permutations, empty. int32 tmp_radii and non-contiguous tensors fall back to torch's path (dispatch order matters: tmp_radii is *contiguous* int32 and reached the C++ TORCH_CHECK before the fix — caught by the quick check's record run).

The bug the A/B surfaced (commit a9c8db1): _gather_rows(self._xyz, ...) reported _xyz as column-major — shape (N,3), strides (1,N). Root cause: fetchPly builds points as np.vstack([...]).T (F-order) and torch.tensor preserves those strides, so _xyz was transposed from creation until the first densify rebuilt it — *in the reference tree too*. Torch ops are stride-aware, but my fused group-Adam wrapper writes through param.contiguous().data_ptr(): for a transposed parameter that is a discarded copy, so xyz (and its exp_avg/exp_avg_sq) was frozen for steps 1..599 while the reference's torch Adam trains it from step 1. Probe: max|xyz(300)−xyz(init)| = 0.0 before the fix, 0.20 after. Present since R10; invisible to gate 2 (it compares no parameter values) — and it explains the systematic gate-3 drift I had been attributing to act-fuse. Fix: .contiguous() at creation in create_from_pcd (+ load_ply); values identical everywhere (every consumer already calls .contiguous() internally), layout only.

Validation: 300-step probe (xyz trains), quick check PASS (/tmp/r17/quick3), full check PASS, speedup 1.6946, tier N1 (/tmp/r17/check): PSNR +0.96σ, SSIM +1.25σ, LPIPS −2.45σ, log_count +0.03σ — versus R14's −2.20/−0.51/−2.12/−0.79. The frozen-xyz bug was the systematic source; margins are now healthy (LPIPS sits ~2σ *below* reference, i.e. better, and inside the gate).

Non-gain noted: windows [49.008, 66.854, 64.796] ≈ R16's — the densify event is *latency*-bound (host-launched kernel chain, ~5 surviving syncs), so making its kernels faster does not shrink the span. The remaining densify lever is fewer launches/syncs/allocations, not faster kernels; treat this path as near-exhausted (~144 events × ~14 ms ≈ 2.0 s ≈ 1.1% of the run, mostly structural).

Decision: both commits kept (a9c8db1 fix — mandatory; 925bf15 gather — bitwise N0, no risk, gates pass). End of round 17: ~11:45.
6.4 hRound 18 — the per-step zero-fill/launch soup: backward arena + empty for fully-written outputs
Start: ~11:45 → end ~11:57. Class: per-step allocation/zeroing overhead — the R17 trace attribution showed each regular step pays ~25 fill_ launches: 13 torch::zeros inside the rasterizer backward's 228 µs CPU span (~122 µs of CPU launch triples), 5 image-sized zeros_like in fused-ssim (4 forward + 1 backward), 3 torch::full in the rasterizer forward, plus the clamp-backward's scalar machinery.

Change (commit 071c81d, N0, no switch — value-identical):
- rasterize_points.cu backward: the eleven P-sized gradient buffers (dL_dmeans3D/means2D/colors/invdepths/conic/opacity/cov3D/dc/sh/scales/rotations) become 16-byte-aligned contiguous views into ONE arena allocation with a single zero fill. First try crashed with *misaligned address*: dL_dconic and dL_drotations are written through float4 casts in backward.cu (lines 405/489) and my offsets were unaligned (P is odd) — fixed by padding every view offset to 4 floats.
- radii: torch::full({P},0) → torch::empty (preprocessCUDA writes radii[idx] for every point before any early return).
- ssim.cu: the five zeros_like(img1) → empty_like (both kernels write every element; the only guard is the pixel bounds check).

Validation: 300-step loss parity vs the pre-change run at the atomics noise floor (1e-6 in the 6th-7th digit, one mark exactly equal), counts identical; quick check PASS (/tmp/r18/quick); time 1.7125 (/tmp/r18/time; windows 48.15/66.42/64.21 — all three windows improved including the densify-free window 3, confirming a per-step saving; total 178.78 s vs 179.99).

Decision: committed. No full check yet — N0-only delta over the R17 full check; bank it and check once more after the next round. End of round 18: ~11:57.
6.8 hRound 19 — input-layout soup: contiguous gt image + view-matrix copy at camera load
Start: ~12:00 → end ~12:18. Class: per-step device clones hiding behind non-contiguous inputs — the R19 profile/trace (/tmp/r19/profile, steps 7000+20) counted 4 aten::contiguous→clone events per step: the SSIM forward cloning the ground-truth image (position 93), the rasterizer forward cloning world_view_transform (164), and two in the rasterizer backward (278/284 — one is the viewmatrix again, the backward re-clones its copy). Direct loader replication confirmed the layout facts: PILtoTorch returns a permuted (H,W,C)-physical view, clamp(0.0,1.0) and .to(cuda) preserve those strides (HWC physical, strides (1, 2937, 3)), and world_view_transform/projection_matrix are transposed views (strides (1,4)).

Change (commit 6ab5045, N0, no switch — layout only):
- Camera.__init__: original_image = gt_image.clamp(0.0,1.0).contiguous().to(data_device) — the clamp is elementwise (value-exact regardless of layout) and .contiguous() is a bitwise relayout, so the stored image is bit-identical; the SSIM kernels now read it directly instead of cloning it twice per step.
- The stored matrices stay EXACTLY as before (a contiguous world_view_transform would feed the bmm/inverse a different layout and could shift full_proj_transform/camera_center by ulps through cublas kernel selection — N1 risk for zero benefit); instead self.raster_viewmatrix = world_view_transform.contiguous() (64 floats, once per camera) and the renderer passes that, with a getattr(..., world_view_transform) fallback for non-Camera callers.

Validation: 300-step probe loss marks 0.2505075/0.1984794/0.1514734 vs R18's 0.2505064/0.1984812/0.1514813 (atomics noise floor), count 136029, dtypes unchanged; quick check PASS (/tmp/r19/quick); time 1.724 (/tmp/r19/time; windows 47.75/66.05/63.78 vs 48.15/66.42/64.21 — all three windows improved ≈ −0.4 s each ≈ 40 µs/step, consistent with removing 3–4 clone launches and their strided reads; total 177.58 s).

Decision: committed. N0-only delta over the R17 full check (R18 arena + R19 layout). End of round 19: ~12:18.
7.4 hRound 20 — fused per-step densification bookkeeping (stats kernel)
Start: ~12:20 → end ~13:05 (two rebuilds: the first norm formula was wrong at the ulp level). Class: the per-step aten soup that is NOT inside the big kernels — the R19 profile's per-op GPU table showed the GPU is saturated (6.23 ms GPU / 5.96 ms wall), so every GPU-µs removed is wall-µs. Between backward and the optimizer step, every iteration of the densification window paid ~7 kernels: radii > 0 (gt), an int32→float32 promotion copy + out-of-place torch.maximum for max_radii2D, a strided (P,2) torch.norm, a bool→float mask product, and two add_ accumulations ≈ 60 µs/step at P≈1.7M.

Change (commit c5460da, switch GS_OPT_STATS_FUSE=1, tier N0): new cuda_rasterizer/stats.cu — densify_stats(radii, grad, max_radii2D, accum, denom) -> vis, one thread per point, in-place state updates; wired through GaussianModel.densify_stats_step (layout-guarded, falls back to the unfused chain), train.py (fused call replaces the maximum + add_densification_stats pair; falls back cleanly if the kernel call declines) and the renderer (skips its radii > 0 under the switch).

The ulp hunt (why two rebuilds): the standalone bitwise parity test — run BEFORE any training — caught the norm's sum-of-squares at 1 ulp off on ~3% of points. torch.norm's two-element reduction is sqrt(round(gx*gx) + round(gy*gy)) with NO FMA contraction (verified against numpy float32 emulation, 0/12345 mismatches); nvcc's default contraction of a plain gx*gx + gy*gy — in either operand order — differs. Fixed with __fmul_rn/__fadd_rn intrinsics that pin the exact form. Also documented: the only inputs where fused ≠ unfused are non-finite screen-space gradients at INVISIBLE points (NaN·0 = NaN in the masked reference) — unreachable, since the rasterizer backward zero-fills them and only atomically writes visible Gaussians.

Validation: standalone parity PASS on all reachable states (P=1..1M, visible fractions 0–1, NaN/Inf at visible points, NaN in max_radii2D; vis/max_radii/accum/denom all bitwise). 800-step A/B on/off/off... plus a same-config second ON run: cross-config loss deltas equal the same-config spread (atomics noise floor), count-change steps identical 600/700/800, cameras identical. Quick check PASS (/tmp/r20/quick). Time 1.727 (/tmp/r20/time; windows 47.71/65.77/63.79 vs R19's 47.75/66.05/63.78 — the −0.29 s sits in window 2, which covers the densification window; w1/w3 at the noise floor).

Honest accounting: +0.003 total, not the +0.013 estimated from the profile — the profile point (step 7000, P=1.7M) over-represents the stats window, whose average P is ~1M (counts: 176k at step 800, 1.7M at 7000). Kept: bitwise N0, gates pass, fewer launches also relieve the CPU side.

Decision: kept. End of round 20: ~13:05.

Full check after rounds 18-20 (/tmp/r20/check, tree = c5460da, all-N0 delta over the R17 check): PASS, speedup 1.7354, tier N1 — gates 0-2 clean, timing windows 47.23/65.52/63.65. Gate-3 margins on seed 1: PSNR −0.09σ, SSIM −0.60σ, LPIPS −0.24σ, log_count +0.25σ (max |0.60σ|) — far healthier than R17's reading (PSNR +0.96, SSIM +1.25, LPIPS −2.45, count +0.03): the −2.45σ LPIPS was run-to-run wobble around a near-zero true bias, not a systematic shift. The final evaluation's single-seed roll now has a wide margin.
7.8 hRound 21 — backward arena split (no gain, reverted)
Start: ~13:06 → end ~13:22. Class: continuation of the zero-fill soup — the backward's single zero-filled arena costs a 41.5 µs fill per step (the largest remaining fill); only the atomically-accumulated buffers actually need their zeros.

Investigation (all facts verified in source before editing): dL_dscales/dL_drotations are written only by computeCov3D (via preprocessCUDA, after its radii guard), dL_dcov3D only by computeCov2DCUDA (full 6-entry write, existing zero branch at line 297), and dL_dmeans3D by computeCov2DCUDA (direct assign — it runs FIRST, before preprocessCUDA's two += contributions). So those four (16 of the arena's 30 floats/point) can be allocated uninitialized if the two kernels store explicit zeros on their culled-point early-return paths. Critical catch: dL_dinvdepths is NOT dead — computeCov2DCUDA *reads* it unconditionally (dL_dinvdepth[idx] * tz2 in the means gradient); with COMPUTE_DEPTH=false it is never written, so its zeros are load-bearing and it must stay in the zeroed arena (also dL_ddc/dL_dsh when the SH gradients are not fused — preprocessCUDA writes them only for visible points).

Change (uncommitted, N0 by construction): split the arena — zeroed 12/pt (means2D, colors, conic, opacity, invdepths, dc/sh stand-ins) + uninitialized 16/pt (means3D, cov3D, scales, rotations, 16B-aligned views); guard-path zero stores in computeCov2DCUDA (9 floats) and preprocessCUDA (7 floats).

Results: 300-step parity at the atomics noise floor (0.2505082/0.1984816/0.1514915, count unchanged), quick check PASS (/tmp/r21/quick), time 1.7354 = exactly the R20 check's number (/tmp/r21/time; windows 47.26/65.54/63.61 vs 47.23/65.52/63.65 — every window within ±0.04 s = noise). No gain.

Diagnosis: at P≈1.7M most Gaussians are culled in any given view, so the explicit culled-point zero stores (~16/pt × culled fraction) eat most of the 16/pt of removed fill traffic; the net (~5-10/pt ≈ 9-17 µs/step) is below a single timing run's noise floor. The fill itself is also only ~40 µs of a 6.2 ms step.

Decision: reverted (git checkout of backward.cu and rasterize_points.cu; tree back at c5460da, the full-checked state). The zero-fill soup is now exhausted: the remaining fill is load-bearing (atomics + the invdepths read). End of round 21: ~13:22.
↑ 3DGS