]> 2026-09-08T17:13:16+09:00 Lesson:research autopilot 20260721t030001z 3000 ko mlir-fft-compiler: Pre-registered held-out validation of analytical GPU/compiler cost models where a rate parameter is under-identified by available calibration data; strongest when a hardware performance counter is a linear function of the parameter. Bounds: establishes experiment resolving power and design adequacy, not which law is true; assumes the candidate set is frozen and cannot exclude unmodeled functional forms consistent with the calibration point. - mlir-fft-compiler [scientific outcome=positive]: When validating a cost-model rate law whose form is degenerate at the calibration point (here spill reloads=0.685*K vs constant=87.68, which coincide at the only spilled calibration shape K=128), a pre-registered held-out design identifies the law only through shapes that break the degeneracy. In the mlir-fft-compiler spill contract, of four held-out families exactly one (rect-k32-j128, the only family that spills with K!=128) separates the two laws: 4.0x in the local-load NCU counter and 39.3pp in derived timing advantage vs a 5pp tolerance. The direct counter channel separates the laws far more sharply than wall-clock, where spill and instruction terms partly cancel.; validation: 1,232 tests pass (7 new) via .venv/bin/python -m pytest tests/ -q; new tests: tests/test_spill_resolving_power.py (3) and 4 in tests/test_spill_validation_contract.py; Pending preflight receipt confirmed byte-identical to experiments/results/spill_heldout_preflight_v1.json after the change; contract_sha256=c8b3bce... unchanged; git diff --stat over experiments/spill_heldout_contract_v1.json, src/spill_model.py, and the preflight receipt is empty (frozen files untouched); verify_spill_heldout --…; next: Execute the frozen held-out timing and counter campaign on authorized L40S physical GPU3 (all 432 timing rows and 48 counter/compile records), prioritizing rect-k32-j128 measurement quality since it is the sole reload-law discriminator, then populate counter_scores from the real l1tex local-load counters and apply the frozen 5pp tolerance/abstention rule to select the linear-K vs constant reload law.; commit a0a464ea5a5f4f353574da7cd632cb6fcf5241fa; PR https://github.com/mrcha033/mlir-fft-compiler/pull/1 S3ResearchAgent medium Only completed, evidence-backed research findings are included. Operational execution state is intentionally retained outside S3 Research Memory. 2026-07-21T03:17:17Z 2461242.6370023 experiments/results/spill_resolving_power_v1.json: one reload-law-identifying family (rect-k32-j128), counter_divergence_ratio=4.0, timing_divergence_pp=39.33, primary_falsifiable_if_challenger_true=true; other three families divergence 0pp; src/spill_validation_contract.py resolving_power() and score_reload_law(); frozen predictions give linear-K spill-load 23.6/46.0 kB vs constant 94.3/184.1 kB per problem at rect-k32-j128; 1,232 tests pass including selection of the true reload law from the local-load counter under validated fixtures; preflight receipt byte-identical and contract_sha256 unchanged; mlir-fft-compiler commit a0a464ea5a5f4f353574da7cd632cb6fcf5241fa; https://github.com/mrcha033/mlir-fft-compiler/pull/1 These are evidence-backed scientific outcomes. Negative and inconclusive outcomes narrow the hypothesis space; mixed and positive outcomes are reusable only within each finding's recorded applicability bounds. No scheduler, quota, model, authentication, search, or publication failure is represented as research evidence. Research findings 20260721T030001Z: 0 negative/inconclusive, 0 mixed, 1 positive mlir-fft-compiler [scientific outcome=positive]: When validating a cost-model rate law whose form is degenerate at the calibration point (here spill reloads=0.685*K vs constant=87.68, which coincide at the only spilled calibration shape K=128), a pre-registered held-out design identifies the law only through shapes that break the degeneracy. In the mlir-fft-compiler spill contract, of four held-out families exactly one (rect-k32-j128, the only family that spills with K!=128) separates the two laws: 4.0x in the local-load NCU counter and 39.3pp in derived timing advantage vs a 5pp tolerance. The direct counter channel separates the laws far more sharply than wall-clock, where spill and instruction terms partly cancel.; validation: 1,232 tests pass (7 new) via .venv/bin/python -m pytest tests/ -q; new tests: tests/test_spill_resolving_power.py (3) and 4 in tests/test_spill_validation_contract.py; Pending preflight receipt confirmed byte-identical to experiments/results/spill_heldout_preflight_v1.json after the change; contract_sha256=c8b3bce... unchanged; git diff --stat over experiments/spill_heldout_contract_v1.json, src/spill_model.py, and the preflight receipt is empty (frozen files untouched); verify_spill_heldout --…; commit a0a464ea5a5f4f353574da7cd632cb6fcf5241fa; PR https://github.com/mrcha033/mlir-fft-compiler/pull/1 What did the validated experiments or analyses establish, including useful negative results and the conditions under which they apply? lab mlir-fft-compiler: For held-out validation of a cost-model rate law, first run a CPU resolving-power pre-analysis: enumerate held-out points, compute each candidate law's prediction, and confirm at least one point breaks the calibration-point degeneracy by more than the acceptance tolerance. Prefer the most direct measurement channel (a hardware counter linear in the rate parameter) over a derived metric where competing terms cancel, and flag when identification rests on a single point so that point gets the tightest measurement discipline. Draft 2026-07-21T03:17:17Z 2461242.6370023 2026-07-21T03:17:17Z 2461242.6370023 research autopilot 20260721t030001z 3000 research-artifact-dd7bcefc87f2375e 2026-07-21T03:17:17Z 2461242.6370023 S3ResearchAgent experiments/results/spill_resolving_power_v1.json: one reload-law-identifying family (rect-k32-j128), counter_divergence_ratio=4.0, timing_divergence_pp=39.33, primary_falsifiable_if_challenger_true=true; other three families divergence 0pp benchmark partial_source research autopilot 20260721t030001z# eaede6fd21efa6dda3340c9e5e0a197a 3000 commit_a0a464ea5a5f4f35 2026-07-21T03:17:17Z 2461242.6370023 S3ResearchAgent GitHub mrcha033/mlir-fft-compiler commit a0a464ea5a5f4f353574da7cd632cb6fcf5241fa code Commit emitted by this cycle; validation scope is recorded in the Lesson. partial_source research autopilot 20260721t030001z# f2e5bb53163fe3143f1d6c9c9f856c79