Synthesis of Phases 1–7. Generated artefacts:
reports/equity_state_phase{1,2,3,4,5,6,7}.md; ledger
reports/equity_state_ledger.jsonl (857 rows); code
validation/equity_state/{data,features,phase1,phase2,phase3,phase4,phase5,phase6,phase7,monitor}.py;
tests tests/test_equity_state.py.
Promotion recommendation:
DO NOT PURSUE(as a tradeable overlay or signal for the existing sleeves). Equity-state variables are genuine, incremental forward-risk forecasters, but they do not improve the frozen sleeves out-of-sample net of the sleeves' own risk controls. No production change was made.
1. Hypotheses tested
- H1 (Phase 1): market-wide breadth / dispersion / correlation / concentration / leadership carry forward information about market return, market vol, cross-sectional selection, and the frozen sleeves' forward outcomes.
- H2 (Phase 2): conditioning sleeve performance on those states materially changes the forward distribution, incrementally to trailing SPY return/vol.
- H3 (Phase 3): a simple, no-look-ahead state-conditioned exposure overlay improves the frozen sleeves' OOS drawdown/tail without destroying return.
- H4 (Phase 4): (a) the incremental forward-vol signal converts into a state-augmented volatility target that beats a naive vol target; and (b) a sleeve's own lagged equity-curve state predicts its forward return/vol/drawdown incrementally over market state.
- H5 (Phase 5): the state index adds incrementally to the incumbent D1 SPY-200d gate; and cross-sleeve simultaneous deterioration (spec §12) predicts aggregate book risk with a tradeable de-risk overlay.
- H6 (Phase 7): the state index improves the incumbent C7 stress-hedge (the tail/drawdown channel, which does not give up upside the way de-risking does).
2. Data and universes
- Yahoo total-return
_ADJdaily store: 501 current S&P 500 names, 2006-08-14 → 2026-08-12, masked to point-in-time S&P 500 membership (monthly snapshots, step-function; no forward-looking). - SPY total-return benchmark; GICS sector map (
data/cache/sectors_full.json). - Frozen baselines (unchanged):
data/baselines/eqmom_live_v1.csv("Equity Momentum"),flagship_base_v1.csv("ML Rotation"). - Limitations:
_ADJholds only current members, so PIT removes late-addition look-ahead but not delisted-name survivorship; mid-caps excluded (2020+ and split-only adjusted); true market cap unavailable, so dollar-volume share is used as a concentration proxy.
3. Features / method
39 trailing features across five families (breadth, cross-sectional dispersion, correlation, concentration, leadership) and 14 forward targets (SPY return 1/5/21d, SPY vol 5/21d, momentum long-short spread, and the two sleeves' forward 5/21d return, 21d vol, 21d max-drawdown). Methods: rank-IC (full/IS/OOS), effective-n (Newey–West lag 21) p-values, partial IC over trailing SPY return/vol, Benjamini–Hochberg FDR, block-bootstrap CIs / permutation, PIT rolling-rank quintile/tercile partitions, rank-OLS incremental R², and a chronological walk-forward with an untouched holdout (≥ 2023-01-01). Automated future-perturbation look-ahead guards on features and PIT buckets.
4. Strongest findings
- Forward risk is predictable, incrementally. High concentration
(
top10_dvwt,hhi_dv) and high cross-market correlation (avg_corr20,corr_spy60) forecast higher forward sleeve volatility and deeper forward drawdowns (BH-significant). Incremental over trailing SPY ret/vol:top10_dvwt → eqmom_vol_fwd21ΔR² +0.100 (partial IC +0.32);top10_dvwt → eqmom_mdd_fwd21ΔR² +0.050 (partial −0.22);corr_spy60 → flagship_mdd_fwd21Q5−Q1 −0.014 (q≈0). - Joint state matches the hypothesised success condition. The worst forward-drawdown cell is low breadth × high correlation / high concentration / high dispersion (flagship_mdd ≈ −6% vs ≈ −4% elsewhere).
- Correlation → momentum-spread. High correlation predicts a weaker raw momentum long-short spread, sign-stable across all four sub-periods and specific to correlation (dispersion carries no plateau). But the raw spread itself has negative OOS Sharpe and the no-tuning gate fails its placebo (p=0.16).
5. Failed hypotheses (recorded, not hidden)
- Sleeve return timing: no feature robustly times eqmom/flagship forward returns; strongest OOS ICs ~0.18–0.22 with BH q>0.05 and partial IC ≈ 0.
- Risk overlay (Phase 3): walk-forward A/B/C/D overlays all reduce Sharpe
(eqmom best 0.90 vs base 0.97; flagship 0.83 vs 0.84) and give back large
return for at best a comparable drawdown; parameter perturbation flips the
drawdown effect sign; block-placebo p=0.92 (no better than a randomly
permuted state assignment). A single holdout cell
(
eqmom/dispersion, Sharpe 1.56 vs 1.49, maxDD −21.4% vs −26.0%) is dominated by the multiple-testing and walk-forward failures. - State-augmented vol targeting (Phase 4): adding the state index to an EWMA vol forecast does not improve a de-risk-only vol-target overlay; state-VT is worse than naive-VT for eqmom and equal for flagship (block-placebo p=0.95 / 0.55).
- Equity-curve feedback (Phase 4, spec §11): no own-curve feature robustly
times forward returns incrementally over market state. Surviving pairs are
risk-clustering (
downside vol → future vol, IC +0.52) and drawdown persistence (flagship deeper drawdown → lower forward return, partial −0.23) — diagnostics, not timing. - State on top of D1 (Phase 5): the state index improves the rolling walk-forward over D1 (Sharpe 1.01 vs 0.87, block-placebo p=0.00) but the map fitted on all pre-holdout data selected the no-overlay option, so the untouched holdout shows no state benefit — period-concentrated, not robust.
- Cross-sleeve deterioration (Phase 5, spec §12): simultaneous sleeve drawdown predicts higher forward book vol (IC +0.33) but the de-risk overlay loses Sharpe (WF 0.80 vs 0.97; holdout 1.71 vs 1.76) — same de-risk-fails-to-pay pattern.
- State-conditional hedge (Phase 7): hedging in the fragile state looks best OOS 2016-23 (Sharpe 1.13 vs C7 1.05, base 0.98; maxDD −20.4% vs −34.1%) but significantly hurts the untouched holdout (C7+state 1.39 vs C7 1.51, block-bootstrap p=0.02). The apparent hedge skill is entirely the Phase-6 high-vol-regime concentration; in the calm 2023+ holdout the hedge is a drag. Both channels (exposure and hedge) now fail OOS.
- Attribution caveats:
avg_corr20largely proxies trailing SPY vol for vol prediction (partial IC flips sign), and breadth levels predict lower forward market returns (post-rally mean reversion), not a timing edge.
6. Candidate ranking (best available, for the record)
| rank | feature(s) | incremental OOS info | robustness | economic significance | impl. simplicity | drawdown impact | verdict |
|---|---|---|---|---|---|---|---|
| 1 | top10_dvwt / hhi_dv (concentration) | ΔR² +0.10 vol, +0.05 DD | conditional BH-robust; overlay not | strong risk forecast | high | predicts DD; overlay fails OOS | INTERESTING (research) |
| 2 | corr_spy60 / avg_corr20 (correlation) | ΔR² +0.06 vol; spread IC −0.21 | 4/4 sub-periods sign-stable | strong risk + spread | high | predicts DD; spread gate p=0.16 | INTERESTING (research) |
| 3 | disp20 / disp1 (dispersion) | ΔR² +0.04 SPY vol | BH-robust | moderate | high | one holdout cell only | WEAK |
| 4 | hl_spread_52w / breadth_sma200 | partial −0.23/−0.15 | BH-robust vs vol | moderate | high | joint worst cell | WEAK |
| 5 | breadth50_chg20 | ΔR² +0.02 DD | BH-robust | small | high | improving breadth → shallower DD | WEAK |
6b. Robustness (Phase 6, spec §13D/E/F)
All seven headline relations keep their sign across full / large-cap-100 / two random halves, and rolling 3y windows are sign-stable: breadth→vol 100%, dispersion→vol 99%, correlation→vol 91%, correlation→momentum-spread 96%, breadth→spread 85%.
But two caveats matter for any overlay:
breadth_sma200 → spy_vol_fwd21is concentrated in high-vol regimes (IC −0.37 high-vol vs +0.02 low-vol) — the "incremental" breadth info is largely redundant once vol is already elevated, exactly where an overlay would act.corr_spy60 → flagship_mdd_fwd21is regime-unstable (bull +0.09, bear −0.17; 84% rolling-window sign consistency) — the drawdown link is the fragile part of the story.
This is why the risk forecast is robust but the risk overlay could not pay.
6c. Observe-only monitor (the actionable output)
The only surviving output is a diagnostic. validation/equity_state/monitor.py
computes a daily PIT fragility state (breadth / correlation / concentration /
dispersion) and writes reports/equity_state_monitor.json + .csv.
- Current reading (2026-08-12): NEUTRAL, state index −0.72, breadth200 0.72, avg_corr 0.009, disp 0.100, top-10 DV weight 0.358.
- Historical state mix: NEUTRAL 42%, PANIC 24%, HEALTHY 18%, FRAGILE 11%, RECOVERY 6%.
- Observe-only: computes no orders, scales no positions, is wired into no bot or the order path. It exists to log "when the book is in a statistically riskier state", not to trade.
7. Recommendation
DO NOT PURSUE
Equity-state internals are a useful risk monitor / research diagnostic
(e.g. an observable "fragile market" flag: low breadth + high correlation +
high concentration) but are not a promotable sizing signal for the
existing equity sleeves under realistic walk-forward, holdout, cost and
placebo testing. Both the exposure channel (Phases 3-5) and the hedge
channel (Phase 7) were tested and fail out-of-sample; every apparent gain is
concentrated in high-vol regimes and reverses in the untouched holdout. No
candidate survives all eight promotion criteria. The only defensible use is
the observe-only fragility monitor (validation/equity_state/monitor.py),
not an overlay.
Two out-of-scope findings worth separate tickets (no state involved):
- The incumbent D1 SPY-200d gate is a DD-cut, not an SR uplift, on the
current frozen baseline. This was already documented in
reports/eqmom_overlay_live_vote.md(D1 OOS SR 1.14 vs base 1.17) — Phase 5 reconfirms it (WF 0.87 vs 0.97; holdout 1.28 vs 1.49). The live101 SR 1.53 belongs to the raw-12m config, not the live residual-momentum one. No new ticket; D1 remains shadow-gated as documented. - Plain EWMA volatility targeting (no market state) improved eqmom walk-forward risk (Sharpe 0.76 → 0.95, maxDD −45% → −22%) but flat holdout Sharpe with a large return give-back — reconcile against the prior D4/BS-C result (book-matching target → constant SR) before use.