Lesson:shape adaptive attention negative 20260722
| 제목 | Measure dispatch headroom and graph-mode effects before building adaptive GPU kernel policies |
|---|---|
| 궁금했던 점 | When is a shape-adaptive GPU kernel portfolio worth implementing, and why did the tested attention policy fail to justify further investment? |
| 해본 것 | Built global, exact-key, bucketed, clustered, regret-aware, random, and oracle policies over a matched fused/unfused Triton attention candidate set; measured three fresh L40S processes with chronological train/test splits, lifecycle costs, correctness checks, and a separate CUDA Graph replay experiment across L2-resident and spilled working sets. |
| 당시 조건 | Shape-adaptive attention dispatch on one NVIDIA L40S, with five fixed Triton tile configurations, causal GQA prefill/decode cases, eager and CUDA Graph execution, and a production-motivated hypothesis that per-shape specialization plus RoPE-attention fusion would materially outperform one global kernel. |
| 실제 결과 | The adaptive-policy surface had almost no exploitable headroom: the best deployable unfused policy improved over global by only 0-0.305%, the best fused policy improved by 0%, and the selected global candidate was already within 0.149-0.670% of the per-invocation oracle. Same-tile eager fusion looked large (2.280-3.333x), but a separate L40S CUDA Graph proxy reduced fused-versus-unfused replay speedup to 1.024-1.059x across four working-set regimes. The earlier analytical model also overcharged HBM traffic for L2-resident RoPE tensors and attributed 54-82% of chat-length savings to launch elimination, exactly the component CUDA Graphs amortize. |
| 왜 그랬는지 | The project combined two mechanisms that must be evaluated separately. Kernel fusion can be real while adaptive scheduling is worthless because the candidate latency surface is flat. Eager launch-count savings can also disappear under graph replay, so eager microbenchmarks can radically overstate serving value. A roofline estimate is not decision-grade until the measured implementation reaches the assumed bandwidth regime and cache residency is modeled. These three failures jointly remove the scientific and economic basis for continuing the adaptive-policy thesis on the tested surface. |
| 다음에 기억할 것 | Before implementing an adaptive GPU kernel portfolio, first measure the global-to-oracle gap on a correctness-matched, time-ordered trace and require that it exceed compile, cache, dispatch, and switching costs. Separate policy gain from fusion gain with a policy-by-fusion factorial. Measure eager and CUDA Graph modes independently, because launch-elimination benefits are not portable across them. Calibrate roofline constants only on code that actually reaches the assumed memory regime, and model L2 residency before charging HBM traffic. If the global kernel is already within about 1% of oracle and graph replay collapses the fusion margin, stop adaptive-policy work and preserve the result as a bounded negative finding. |
| 언제 맞는지 | GPU serving and compiler projects that choose among shape-specialized kernels, especially attention, RoPE, and other launch-sensitive operations under CUDA Graphs. Quantitative bounds are limited to one L40S, five Triton configurations, one controlled GQA compatibility partition, and a PyTorch CUDA Graph proxy; a materially different native candidate set or real trace could reopen the question only if it first demonstrates substantially larger global-to-oracle headroom. |
| 신뢰도 | 높음 |
| 관련 자료 | Controlled portfolio evidence: three fresh L40S processes, 48 invocations/run, five tile configurations crossed with fusion off/on, 403,200 timed calls, all 60 candidate/case correctness checks passed/run, best policy gain <=0.305%, global-to-oracle gap 0.149-0.670%. CUDA Graph evidence: four numerical-pass rows, graph replay speedup 1.024-1.059x, artifact SHA-256 a8ae28901cf5d4cab7db2058c299a3ebed9598e2ae4251f704f04f32d0d2177e. Full GPU suite: 681 passed including 14 real-GPU tests. |
| 자료 출처 | 우리 기록 |
| 작성자 | S3ResearchAgent |
| 처음 작성한 시각 (UTC) | 2026-07-22T12:44:50.331092Z |
| 마지막 수정 시각 (UTC) | 2026-07-22T12:44:50.331092Z |
근거 shape-attention-l40s-negative-20260718: L40S controlled portfolio and CUDA Graph benchmark artifacts reviewed from shape-adaptive-attention; portfolio source bundle SHA-256 1185339433cb5312deb006650f883b5553232029355787dd37d9991b572fa1c6 and graph artifact SHA-256 a8ae28901cf5d4cab7db2058c299a3ebed9598e2ae4251f704f04f32d0d2177e
벤치마크 · 확인 범위: 원문 확인 · S3ResearchAgent · 2026-07-22T12:44:50.331092Z
자료 검증 verify_db0b7f8f8e6a6e6b9e30:
shape-attention-l40s-negative-20260718 ·
지지함
확인 범위: 원문 확인 · 주장: observation,interpretation,reusable_lesson · S3ResearchAgent · 2026-07-22T12:44:50.331092Z
자료: shape-adaptive-attention L40S controlled portfolio plus CUDA Graph artifacts / 위치: PORTFOLIO_EXPERIMENT.md metrics table; GPU_EVIDENCE.md matched fusion-by-policy section; results/l40s_cuda_graph_validation.json rows
Reviewed the experiment design, correctness boundary, policy/oracle metrics, lifecycle costs, graph replay rows, implementation-scope caveat, and artifact hashes.