]> 2026-09-08T17:13:15+09:00 Lesson:research autopilot 20260721t090001z 3000 ko alpha-factory: Offline portfolio/backtest research where a train/select/validate/hidden split is percentage-based and non-overlapping, and a headline OOS metric is computed on a window that does not touch the sealed suffix. The CSCV PBO gate is only weakly informative when the search collapses to few distinct strategies; use it as a sanity gate, not a full search-space overfitting test. - alpha-factory [scientific outcome=positive]: On the alpha-factory real ETF-macro pool, the diversifier-track finding (ensemble diversity requires BOTH an upstream eligibility track admitting a decorrelated quality-gate-failer AND a downstream diversification-lambda>0) survives regeneration under hidden-suffix-sealed evaluator semantics v3, and survives a pre-registered single-window selection-sensitivity gate. The (eligibility=on, lambda=0.5) cell adds seed_crisis_hedge, improving ensemble_validate max-drawdown from -4.07% to -2.57% and raising Sharpe from 1.31 to 1.62. Paired stationary block bootstrap: Sharpe delta +0.314 (95% CI [+0.029,+0.62], P(delta<=0)=1.3%), P(drawdown better)=100%; CSCV PBO=0.000 over 252 partitions. The cand…; validation: pytest: 371 passed, 2 failed (known fixture-missing: data/etf_macro_daily_v1/SPY.csv absent; matches prior state's 2 known deselections); v3 regeneration: divsweep_pool_v3 artifacts carry evaluator_semantics id close-t-signal_..._hidden-suffix-sealed_v3; hidden_test_included=False; 2x2 ablation v3 (docs/evidence/diversifier_track_ablation_v3.json): only (on,0.5) is 2-lineage, validate Sharpe 1.6218 maxDD -2.57%; claim supported=True; selection sensitivity (docs/evidence/diversifier_selection_se…; next: Run the frozen v3 pool-and-ablation protocol across >=3 non-overlapping outer time folds (or the etf_macro_transfer universe) with the admission rule and lambda held fixed, report per-fold drawdown/Sharpe deltas with paired intervals, and only if the advantage sign is stable across folds preregister exactly one finalist and spend the single sealed hidden-test access. Do not calibrate lambda before the outer-window evidence exists.; commit f8e17cd2a2c1705365044fb3c5866fffc3d03030; PR https://github.com/mrcha033/alpha-factory/pull/1 S3ResearchAgent medium Only completed, evidence-backed research findings are included. Operational execution state is intentionally retained outside S3 Research Memory. 2026-07-21T09:27:52Z 2461242.8943519 docs/evidence/divsweep_pool_v3_leaderboard.csv (v3 pool, 3 gate-passers unchanged from v2); docs/evidence/diversifier_track_ablation_v3.json (2x2: only (on,0.5) is 2-lineage, Sharpe 1.6218, maxDD -2.57%); docs/evidence/diversifier_selection_sensitivity_v3.json (Sharpe delta +0.314 CI95 [+0.029,+0.62], P(delta<=0)=1.3%, CSCV PBO=0.000); docs/evidence/divsweep_pool_v3_diversifier_track.json (seed_crisis_hedge admitted, fit 0.50, mean\\|corr\\| 0.27); pytest 371 passed / 2 known fixture-missing; alpha-factory commit f8e17cd2a2c1705365044fb3c5866fffc3d03030; https://github.com/mrcha033/alpha-factory/pull/1 These are evidence-backed scientific outcomes. Negative and inconclusive outcomes narrow the hypothesis space; mixed and positive outcomes are reusable only within each finding's recorded applicability bounds. No scheduler, quota, model, authentication, search, or publication failure is represented as research evidence. Research findings 20260721T090001Z: 0 negative/inconclusive, 0 mixed, 1 positive alpha-factory [scientific outcome=positive]: On the alpha-factory real ETF-macro pool, the diversifier-track finding (ensemble diversity requires BOTH an upstream eligibility track admitting a decorrelated quality-gate-failer AND a downstream diversification-lambda>0) survives regeneration under hidden-suffix-sealed evaluator semantics v3, and survives a pre-registered single-window selection-sensitivity gate. The (eligibility=on, lambda=0.5) cell adds seed_crisis_hedge, improving ensemble_validate max-drawdown from -4.07% to -2.57% and raising Sharpe from 1.31 to 1.62. Paired stationary block bootstrap: Sharpe delta +0.314 (95% CI [+0.029,+0.62], P(delta<=0)=1.3%), P(drawdown better)=100%; CSCV PBO=0.000 over 252 partitions. The cand…; validation: pytest: 371 passed, 2 failed (known fixture-missing: data/etf_macro_daily_v1/SPY.csv absent; matches prior state's 2 known deselections); v3 regeneration: divsweep_pool_v3 artifacts carry evaluator_semantics id close-t-signal_..._hidden-suffix-sealed_v3; hidden_test_included=False; 2x2 ablation v3 (docs/evidence/diversifier_track_ablation_v3.json): only (on,0.5) is 2-lineage, validate Sharpe 1.6218 maxDD -2.57%; claim supported=True; selection sensitivity (docs/evidence/diversifier_selection_se…; commit f8e17cd2a2c1705365044fb3c5866fffc3d03030; PR https://github.com/mrcha033/alpha-factory/pull/1 What did the validated experiments or analyses establish, including useful negative results and the conditions under which they apply? lab alpha-factory: When an evaluator-semantics fix (excluding a hidden suffix) invalidates prior artifacts, regenerate and re-test rather than assuming the old conclusion breaks: check whether the affected quantity (here candidate-level total_return/fitness over the full range) actually feeds the reported conclusion (here an OOS window that structurally never overlapped the hidden suffix). Pair the regeneration with a seeded paired block bootstrap plus CSCV PBO on the sealed research window to report selection sensitivity WITHOUT spending a one-shot hidden-test access. Draft 2026-07-21T09:27:53Z 2461242.8943634 2026-07-21T09:27:53Z 2461242.8943634 research autopilot 20260721t090001z 3000 research-artifact-fe5656eeeadb895d 2026-07-21T09:27:52Z 2461242.8943519 S3ResearchAgent docs/evidence/divsweep_pool_v3_leaderboard.csv (v3 pool, 3 gate-passers unchanged from v2) benchmark partial_source research autopilot 20260721t090001z# e45e09f2f3fdc4fd2e0e3bd25174057e 3000 commit_f8e17cd2a2c17053 2026-07-21T09:27:53Z 2461242.8943634 S3ResearchAgent GitHub mrcha033/alpha-factory commit f8e17cd2a2c1705365044fb3c5866fffc3d03030 code Commit emitted by this cycle; validation scope is recorded in the Lesson. partial_source research autopilot 20260721t090001z# b10c9ef382e9a4f8a3a08198e15544ba