]> 2026-09-08T17:13:19+09:00 Lesson:research autopilot 20260721t210001z 3000 ko alpha-factory: Offline portfolio-construction / ensemble-selection research where a treatment changes ensemble composition; single ETF-macro universe and one frozen candidate pool re-sliced by evaluation window (not re-evolved per fold), rolling-origin (not walk-forward-refit). Second-universe transfer (etf_macro_transfer) and hidden-test confirmation remain open. - alpha-factory [scientific outcome=mixed]: In the alpha-factory ensemble diversifier, the frozen (eligibility=on, lambda=0.5) admission rule replicates STRUCTURALLY across three non-overlapping rolling-origin outer validate windows (2018-2025) — every fold admits the decorrelated crisis hedge, builds 2 lineages, is Sharpe non-inferior, and improves drawdown directionally — but its ECONOMIC MAGNITUDE is regime-specific: the paired-bootstrap Sharpe-delta 95% CI excludes zero only in the most recent 2022-2025 window (+0.313 CI[+0.03,+0.62]), while pooled over all seven years dSharpe=+0.046 CI[-0.27,+0.38] and aggregate maxDD is essentially unchanged (-10.55%->-10.26%). A single-window bootstrap that clears significance can substantiall…; validation: tests/test_diversifier_outer_window_design.py: 5 passed (fold design causal, non-overlapping, hidden-sealed, frozen admission rule); Re-ran experiments/diversifier_outer_window_folds.py --run-id divsweep_pool_v3; JSON output reproduces docs/evidence/diversifier_outer_window_folds_v3.json byte-for-byte (deterministic, bootstrap seed 20260721); Per-fold oracle: recomputed validate Sharpe matches builder-reported within 0.02 (fold_2 flagged reproduces=false at 0.035 gap on a negative-Sharpe window…; next: Regenerate the v3 pool on the etf_macro_transfer universe (alpha evolve with universe: universes/etf_macro_transfer_daily_v1.yaml), run the identical frozen ablation + outer-window folds, and test whether the STRUCTURAL result (2-lineage crisis-hedge admission, Sharpe non-inferior, directionally DD-protective) — not the fold_3 magnitude — transfers to a second universe. Only if it holds on both time folds and the second universe should exactly one finalist be preregistered and the single sealed…; commit fc5de9cb146144570971c1f53094cb73603b942b; PR https://github.com/mrcha033/alpha-factory/pull/1 S3ResearchAgent medium Only completed, evidence-backed research findings are included. Operational execution state is intentionally retained outside S3 Research Memory. 2026-07-21T22:11:47Z 2461243.4248495 experiments/diversifier_outer_window_folds.py replayed on divsweep_pool_v3; deterministic (seed 20260721), reproduces docs/evidence/diversifier_outer_window_folds_v3.json byte-for-byte; Per-fold: fold_1 dSharpe +0.123 CI[-0.94,+0.96] P(d<=0)=0.443; fold_2 +0.040 CI[-0.35,+0.46] 0.429; fold_3 +0.313 CI[+0.03,+0.62] 0.013; pooled(1760 obs) +0.046 CI[-0.27,+0.38] 0.391; all_folds_stable=True (2 lineages, Sharpe non-inferior, DD improved every fold); admission decision window-independent by construction (corr-to-elite over full research prefix); tests/test_diversifier_outer_window_design.py 5 passed; full suite 376 passed / 2 pre-existing fixture-missing failures; Hidden suffix [0.94,1.0] never touched; oracle reproduces builder validate Sharpe within 0.02 per fold; alpha-factory commit fc5de9cb146144570971c1f53094cb73603b942b; https://github.com/mrcha033/alpha-factory/pull/1 These are evidence-backed scientific outcomes. Negative and inconclusive outcomes narrow the hypothesis space; mixed and positive outcomes are reusable only within each finding's recorded applicability bounds. No scheduler, quota, model, authentication, search, or publication failure is represented as research evidence. Research findings 20260721T210001Z: 0 negative/inconclusive, 1 mixed, 0 positive alpha-factory [scientific outcome=mixed]: In the alpha-factory ensemble diversifier, the frozen (eligibility=on, lambda=0.5) admission rule replicates STRUCTURALLY across three non-overlapping rolling-origin outer validate windows (2018-2025) — every fold admits the decorrelated crisis hedge, builds 2 lineages, is Sharpe non-inferior, and improves drawdown directionally — but its ECONOMIC MAGNITUDE is regime-specific: the paired-bootstrap Sharpe-delta 95% CI excludes zero only in the most recent 2022-2025 window (+0.313 CI[+0.03,+0.62]), while pooled over all seven years dSharpe=+0.046 CI[-0.27,+0.38] and aggregate maxDD is essentially unchanged (-10.55%->-10.26%). A single-window bootstrap that clears significance can substantiall…; validation: tests/test_diversifier_outer_window_design.py: 5 passed (fold design causal, non-overlapping, hidden-sealed, frozen admission rule); Re-ran experiments/diversifier_outer_window_folds.py --run-id divsweep_pool_v3; JSON output reproduces docs/evidence/diversifier_outer_window_folds_v3.json byte-for-byte (deterministic, bootstrap seed 20260721); Per-fold oracle: recomputed validate Sharpe matches builder-reported within 0.02 (fold_2 flagged reproduces=false at 0.035 gap on a negative-Sharpe window…; commit fc5de9cb146144570971c1f53094cb73603b942b; PR https://github.com/mrcha033/alpha-factory/pull/1 What did the validated experiments or analyses establish, including useful negative results and the conditions under which they apply? lab alpha-factory: Before treating a favorable single-window backtest bootstrap as a transferable edge, replay the exact frozen rule across independent non-overlapping outer windows and pool them as one rolling-origin backtest; separate the STRUCTURAL claim (composition/eligibility that is window-independent by construction) from the ECONOMIC-MAGNITUDE claim (risk-adjusted return), because the former can replicate cleanly while the latter is concentrated in one regime. A stable-signed but pooled-insignificant delta is a legitimate reason to NOT spend a sealed hidden-test access. Draft 2026-07-21T22:11:47Z 2461243.4248495 2026-07-21T22:11:47Z 2461243.4248495 research autopilot 20260721t210001z 3000 research-artifact-dedf4834f85c1407 2026-07-21T22:11:47Z 2461243.4248495 S3ResearchAgent experiments/diversifier_outer_window_folds.py replayed on divsweep_pool_v3; deterministic (seed 20260721), reproduces docs/evidence/diversifier_outer_window_folds_v3.json byte-for-byte benchmark partial_source research autopilot 20260721t210001z# 458336fb0722afb7b29dff24135e50da 3000 commit_fc5de9cb14614457 2026-07-21T22:11:47Z 2461243.4248495 S3ResearchAgent GitHub mrcha033/alpha-factory commit fc5de9cb146144570971c1f53094cb73603b942b code Commit emitted by this cycle; validation scope is recorded in the Lesson. partial_source research autopilot 20260721t210001z# f6a75e80e8dc0f478398fca57534c03d