Lesson:research autopilot 20260721t030001z
| 제목 | Research findings 20260721T030001Z: 0 negative/inconclusive, 0 mixed, 1 positive |
|---|---|
| 궁금했던 점 | What did the validated experiments or analyses establish, including useful negative results and the conditions under which they apply? |
| 해본 것 | - mlir-fft-compiler [scientific outcome=positive]: When validating a cost-model rate law whose form is degenerate at the calibration point (here spill reloads=0.685*K vs constant=87.68, which coincide at the only spilled calibration shape K=128), a pre-registered held-out design identifies the law only through shapes that break the degeneracy. In the mlir-fft-compiler spill contract, of four held-out families exactly one (rect-k32-j128, the only family that spills with K!=128) separates the two laws: 4.0x in the local-load NCU counter and 39.3pp in derived timing advantage vs a 5pp tolerance. The direct counter channel separates the laws far more sharply than wall-clock, where spill and instruction terms partly cancel.; validation: 1,232 tests pass (7 new) via .venv/bin/python -m pytest tests/ -q; new tests: tests/test_spill_resolving_power.py (3) and 4 in tests/test_spill_validation_contract.py; Pending preflight receipt confirmed byte-identical to experiments/results/spill_heldout_preflight_v1.json after the change; contract_sha256=c8b3bce... unchanged; git diff --stat over experiments/spill_heldout_contract_v1.json, src/spill_model.py, and the preflight receipt is empty (frozen files untouched); verify_spill_heldout --…; next: Execute the frozen held-out timing and counter campaign on authorized L40S physical GPU3 (all 432 timing rows and 48 counter/compile records), prioritizing rect-k32-j128 measurement quality since it is the sole reload-law discriminator, then populate counter_scores from the real l1tex local-load counters and apply the frozen 5pp tolerance/abstention rule to select the linear-K vs constant reload law.; commit a0a464ea5a5f4f353574da7cd632cb6fcf5241fa; PR https://github.com/mrcha033/mlir-fft-compiler/pull/1 |
| 당시 조건 | Only completed, evidence-backed research findings are included. Operational execution state is intentionally retained outside S3 Research Memory. |
| 실제 결과 | mlir-fft-compiler [scientific outcome=positive]: When validating a cost-model rate law whose form is degenerate at the calibration point (here spill reloads=0.685*K vs constant=87.68, which coincide at the only spilled calibration shape K=128), a pre-registered held-out design identifies the law only through shapes that break the degeneracy. In the mlir-fft-compiler spill contract, of four held-out families exactly one (rect-k32-j128, the only family that spills with K!=128) separates the two laws: 4.0x in the local-load NCU counter and 39.3pp in derived timing advantage vs a 5pp tolerance. The direct counter channel separates the laws far more sharply than wall-clock, where spill and instruction terms partly cancel.; validation: 1,232 tests pass (7 new) via .venv/bin/python -m pytest tests/ -q; new tests: tests/test_spill_resolving_power.py (3) and 4 in tests/test_spill_validation_contract.py; Pending preflight receipt confirmed byte-identical to experiments/results/spill_heldout_preflight_v1.json after the change; contract_sha256=c8b3bce... unchanged; git diff --stat over experiments/spill_heldout_contract_v1.json, src/spill_model.py, and the preflight receipt is empty (frozen files untouched); verify_spill_heldout --…; commit a0a464ea5a5f4f353574da7cd632cb6fcf5241fa; PR https://github.com/mrcha033/mlir-fft-compiler/pull/1 |
| 왜 그랬는지 | These are evidence-backed scientific outcomes. Negative and inconclusive outcomes narrow the hypothesis space; mixed and positive outcomes are reusable only within each finding's recorded applicability bounds. No scheduler, quota, model, authentication, search, or publication failure is represented as research evidence. |
| 다음에 기억할 것 | mlir-fft-compiler: For held-out validation of a cost-model rate law, first run a CPU resolving-power pre-analysis: enumerate held-out points, compute each candidate law's prediction, and confirm at least one point breaks the calibration-point degeneracy by more than the acceptance tolerance. Prefer the most direct measurement channel (a hardware counter linear in the rate parameter) over a derived metric where competing terms cancel, and flag when identification rests on a single point so that point gets the tightest measurement discipline. |
| 언제 맞는지 | mlir-fft-compiler: Pre-registered held-out validation of analytical GPU/compiler cost models where a rate parameter is under-identified by available calibration data; strongest when a hardware performance counter is a linear function of the parameter. Bounds: establishes experiment resolving power and design adequacy, not which law is true; assumes the candidate set is frozen and cannot exclude unmodeled functional forms consistent with the calibration point. |
| 신뢰도 | 중간 |
| 관련 자료 | experiments/results/spill_resolving_power_v1.json: one reload-law-identifying family (rect-k32-j128), counter_divergence_ratio=4.0, timing_divergence_pp=39.33, primary_falsifiable_if_challenger_true=true; other three families divergence 0pp; src/spill_validation_contract.py resolving_power() and score_reload_law(); frozen predictions give linear-K spill-load 23.6/46.0 kB vs constant 94.3/184.1 kB per problem at rect-k32-j128; 1,232 tests pass including selection of the true reload law from the local-load counter under validated fixtures; preflight receipt byte-identical and contract_sha256 unchanged; mlir-fft-compiler commit a0a464ea5a5f4f353574da7cd632cb6fcf5241fa; https://github.com/mrcha033/mlir-fft-compiler/pull/1 |
| 자료 출처 | 우리 기록 |
| 작성자 | S3ResearchAgent |
| 처음 작성한 시각 (UTC) | 2026-07-21T03:17:17.383877Z |
| 마지막 수정 시각 (UTC) | 2026-07-21T03:17:17.383877Z |
근거 research-artifact-dd7bcefc87f2375e: experiments/results/spill_resolving_power_v1.json: one reload-law-identifying family (rect-k32-j128), counter_divergence_ratio=4.0, timing_divergence_pp=39.33, primary_falsifiable_if_challenger_true=true; other three families divergence 0pp
벤치마크 · 확인 범위: 일부 자료 확인 · S3ResearchAgent · 2026-07-21T03:17:17.383877Z