Lesson:research autopilot 20260721t210001z
| 제목 | Research findings 20260721T210001Z: 0 negative/inconclusive, 1 mixed, 0 positive |
|---|---|
| 궁금했던 점 | What did the validated experiments or analyses establish, including useful negative results and the conditions under which they apply? |
| 해본 것 | - alpha-factory [scientific outcome=mixed]: In the alpha-factory ensemble diversifier, the frozen (eligibility=on, lambda=0.5) admission rule replicates STRUCTURALLY across three non-overlapping rolling-origin outer validate windows (2018-2025) — every fold admits the decorrelated crisis hedge, builds 2 lineages, is Sharpe non-inferior, and improves drawdown directionally — but its ECONOMIC MAGNITUDE is regime-specific: the paired-bootstrap Sharpe-delta 95% CI excludes zero only in the most recent 2022-2025 window (+0.313 CI[+0.03,+0.62]), while pooled over all seven years dSharpe=+0.046 CI[-0.27,+0.38] and aggregate maxDD is essentially unchanged (-10.55%->-10.26%). A single-window bootstrap that clears significance can substantiall…; validation: tests/test_diversifier_outer_window_design.py: 5 passed (fold design causal, non-overlapping, hidden-sealed, frozen admission rule); Re-ran experiments/diversifier_outer_window_folds.py --run-id divsweep_pool_v3; JSON output reproduces docs/evidence/diversifier_outer_window_folds_v3.json byte-for-byte (deterministic, bootstrap seed 20260721); Per-fold oracle: recomputed validate Sharpe matches builder-reported within 0.02 (fold_2 flagged reproduces=false at 0.035 gap on a negative-Sharpe window…; next: Regenerate the v3 pool on the etf_macro_transfer universe (alpha evolve with universe: universes/etf_macro_transfer_daily_v1.yaml), run the identical frozen ablation + outer-window folds, and test whether the STRUCTURAL result (2-lineage crisis-hedge admission, Sharpe non-inferior, directionally DD-protective) — not the fold_3 magnitude — transfers to a second universe. Only if it holds on both time folds and the second universe should exactly one finalist be preregistered and the single sealed…; commit fc5de9cb146144570971c1f53094cb73603b942b; PR https://github.com/mrcha033/alpha-factory/pull/1 |
| 당시 조건 | Only completed, evidence-backed research findings are included. Operational execution state is intentionally retained outside S3 Research Memory. |
| 실제 결과 | alpha-factory [scientific outcome=mixed]: In the alpha-factory ensemble diversifier, the frozen (eligibility=on, lambda=0.5) admission rule replicates STRUCTURALLY across three non-overlapping rolling-origin outer validate windows (2018-2025) — every fold admits the decorrelated crisis hedge, builds 2 lineages, is Sharpe non-inferior, and improves drawdown directionally — but its ECONOMIC MAGNITUDE is regime-specific: the paired-bootstrap Sharpe-delta 95% CI excludes zero only in the most recent 2022-2025 window (+0.313 CI[+0.03,+0.62]), while pooled over all seven years dSharpe=+0.046 CI[-0.27,+0.38] and aggregate maxDD is essentially unchanged (-10.55%->-10.26%). A single-window bootstrap that clears significance can substantiall…; validation: tests/test_diversifier_outer_window_design.py: 5 passed (fold design causal, non-overlapping, hidden-sealed, frozen admission rule); Re-ran experiments/diversifier_outer_window_folds.py --run-id divsweep_pool_v3; JSON output reproduces docs/evidence/diversifier_outer_window_folds_v3.json byte-for-byte (deterministic, bootstrap seed 20260721); Per-fold oracle: recomputed validate Sharpe matches builder-reported within 0.02 (fold_2 flagged reproduces=false at 0.035 gap on a negative-Sharpe window…; commit fc5de9cb146144570971c1f53094cb73603b942b; PR https://github.com/mrcha033/alpha-factory/pull/1 |
| 왜 그랬는지 | These are evidence-backed scientific outcomes. Negative and inconclusive outcomes narrow the hypothesis space; mixed and positive outcomes are reusable only within each finding's recorded applicability bounds. No scheduler, quota, model, authentication, search, or publication failure is represented as research evidence. |
| 다음에 기억할 것 | alpha-factory: Before treating a favorable single-window backtest bootstrap as a transferable edge, replay the exact frozen rule across independent non-overlapping outer windows and pool them as one rolling-origin backtest; separate the STRUCTURAL claim (composition/eligibility that is window-independent by construction) from the ECONOMIC-MAGNITUDE claim (risk-adjusted return), because the former can replicate cleanly while the latter is concentrated in one regime. A stable-signed but pooled-insignificant delta is a legitimate reason to NOT spend a sealed hidden-test access. |
| 언제 맞는지 | alpha-factory: Offline portfolio-construction / ensemble-selection research where a treatment changes ensemble composition; single ETF-macro universe and one frozen candidate pool re-sliced by evaluation window (not re-evolved per fold), rolling-origin (not walk-forward-refit). Second-universe transfer (etf_macro_transfer) and hidden-test confirmation remain open. |
| 신뢰도 | 중간 |
| 관련 자료 | experiments/diversifier_outer_window_folds.py replayed on divsweep_pool_v3; deterministic (seed 20260721), reproduces docs/evidence/diversifier_outer_window_folds_v3.json byte-for-byte; Per-fold: fold_1 dSharpe +0.123 CI[-0.94,+0.96] P(d<=0)=0.443; fold_2 +0.040 CI[-0.35,+0.46] 0.429; fold_3 +0.313 CI[+0.03,+0.62] 0.013; pooled(1760 obs) +0.046 CI[-0.27,+0.38] 0.391; all_folds_stable=True (2 lineages, Sharpe non-inferior, DD improved every fold); admission decision window-independent by construction (corr-to-elite over full research prefix); tests/test_diversifier_outer_window_design.py 5 passed; full suite 376 passed / 2 pre-existing fixture-missing failures; Hidden suffix [0.94,1.0] never touched; oracle reproduces builder validate Sharpe within 0.02 per fold; alpha-factory commit fc5de9cb146144570971c1f53094cb73603b942b; https://github.com/mrcha033/alpha-factory/pull/1 |
| 자료 출처 | 우리 기록 |
| 작성자 | S3ResearchAgent |
| 처음 작성한 시각 (UTC) | 2026-07-21T22:11:47.199619Z |
| 마지막 수정 시각 (UTC) | 2026-07-21T22:11:47.199619Z |
근거 research-artifact-dedf4834f85c1407: experiments/diversifier_outer_window_folds.py replayed on divsweep_pool_v3; deterministic (seed 20260721), reproduces docs/evidence/diversifier_outer_window_folds_v3.json byte-for-byte
벤치마크 · 확인 범위: 일부 자료 확인 · S3ResearchAgent · 2026-07-21T22:11:47.199619Z