"Learning Overlays for a Validated Multi-Sleeve Book: RL, Markov, and NN Fine-Tuning"

Date: 2026-09-07 Scope: four RL allocator generations (v2–v5), Markov-regime timing, pooled-NN fine-tuning, and edge-hunt rounds 16–19 — all aimed at beating validated baselines (equal-weight book, EqMom, PEAD, gate60) that already run live on paper accounts. Status: 0 promotions. 1 MARGINAL de-risk variant (PEAD double gate). 1 retracted PASS (spike reversal). Companion to reports/edge_research_paper.md (the factor-zoo era).

#Abstract

The production book (flagship + EqMom + PEAD + Treasury + crypto-trend, equal-weight monthly) beats every sleeve alone by diversification. This program asked whether learning could add a second layer: RL allocation across sleeves, Markov-regime timing within sleeves, and NN fine-tuning of the pooled stock-selection model. Answer: no, not in any formulation tried. RL allocators failed 4× (best median uplift −0.002, latest −0.168 over 35 cells). A Markov-GMM timing gate failed formally and conditionally. Pooled-NN fine-tuning degraded train below its own ablation. Two regime-gate extensions are MARGINAL de-riskers (halved drawdowns, lower Sharpes). One apparent PASS (VIX-spike reversal, p≈0.002) was retracted when the cost pass exposed a non-tradable return construction — the program's most valuable catch. The only learning system that earns its keep remains supervised stock selection (flagship h5, live).

#1. RL allocators v2–v5: the overlay that converges to its baseline

Formulation (all generations): PPO allocates across sleeve-return streams; compared against deterministic equal-weight (EW) and risk-parity baselines net of a four-component cost model (commission + spread + slippage + sqrt-impact), walk-forward with embargo, multi-seed, bootstrap p on paired OOS diffs. Acceptance: Sharpe uplift ≥ +0.10, p ≤ 0.05, turnover/maxDD/behavioral gates.

Mechanism (v4 audit): the agent's mean allocation sat within 0.001 of EW on every sleeve — constraints + costs pushed the optimum to "don't move." v5 made it worse, informatively: the agent did tilt (BTC 0.179 vs ETH 0.153) and traded 10× more than v4 — and lost by more. Its worst window was the 2022 bear (agent −5.2 vs EW −4.6): it learned a procyclical tilt, overweighting beta into drawdowns. Richer state did not help because there is no persistent cross-sleeve rotation signal to find: all sleeves are long-biased drift streams, and EW is the minimum-regret aggregator of such streams.

Code: models/rl_env_v4.py, models/rl_env_v5.py, models/rl_v5_baseline.py, validation/rl_v5_oos.py, validation/rl_v5_smoke.py, scripts/rl_v4_sweep.py.

#2. Markov-regime timing (Round 16): GMM gate, formal + conditional REJECT

Design: GaussianMixture(3) over trailing-252d (spy_ret20, spy_vol20, vix_ratio, credit_z), refit monthly on trailing data only, stress = highest-VIX component; lag-1 flat-in-stress gate on EqMom, PEAD, SPY (validation/edge_hunt16.py).

Result: Sharpe lifts everywhere (+0.05 EqMom, +0.32 SPY, +0.06 PEAD) with positive residual Sharpes — but paired p ≈ 0.50 throughout (REJECT). Follow-up conditional test (mean return on calm vs stress days): gaps all positive yet insignificant (SPY +3.84bps/d p=0.216; EqMom/PEAD p≈0.47). The state weakly separates the market but not stock-selection sleeves — expected in hindsight: momentum/PEAD returns come from which names, not market direction. (reports/edge_hunt16.md)

#3. Pooled-NN fine-tuning: the adapter that dilutes (REJECT)

Problem (P3a): a pooled ensemble trained on 51 flagship names transfers zero-shot to 20 unseen names (PF 1.33 vs per-symbol 1.24) but dilutes train (CAGR 7% vs 17%). Fix attempted (validation/pooled_finetune.py): per-cutoff pooled EnsembleModel plus a light per-symbol GradientBoostingClassifier(100) adapter, blended by trailing-252d IC share, frozen for the OOS block; 12 train / 8 holdout names, 12y, region-A simulation.

Result (reports/pooled_finetune.md): MARGINAL → REJECT after ablation. Blend: train CAGR −0.90%, holdout PF 1.37. Pooled-only rerun in the same harness (scores replayed from cache, validation/pooled_only_from_cache.py): train +3.34%, holdout PF 1.25. So the small pool/short window explains part of the P3a gap — but the adapter itself degrades train (−0.90% vs +3.34%) while only lifting holdout PF on 6 names with a worse 2023–26 (+37% vs +51%). Trailing-IC blend weights chase noise; an unconstrained 100-tree adapter memorizes. Follow-up (2026-09-07): constrained adapter (pooled floor 0.6, 25 trees, min_samples_leaf 20, same harness) scored train −0.35% / holdout PF 1.26 (+23.3% 2023–26) — worse than the unconstrained blend on holdout and still far below pooled-only on train. Shrinkage does not rescue an anti-predictive adapter signal. Macro-feature variant (+vix_ratio, credit_z, t10y2y_z force-included): MARGINAL, holdout PF 1.34, train −2.5% — macro neither fixes dilution nor (at smoke scale) harms pooled (+4.2% pooled-with-macro vs +3.3% without). Full 51-name/20y scale-up (constrained + macro): REJECT, train −1.0% / holdout −0.5%. Pooled-only replay from the same cache also fails (−4.4%/−5.0%) — but that implicates the harness, not just the adapter: P3a's +7%/+10% came from older simulator code+config (since co-evolved: chop_scale, trend_loose, vol_target gates), a different window, and possibly different names; plus the scale-up cache mixed two date grids after a fetch-window shift (recovered via positional keys, but OOS blocks carry ±1-day training-window jitter). A clean P3a replication was judged not worth ~15h to settle calibration. Adapter verdict stands on the smoke-scale evidence (3 variants, same harness, same code: adapter always degrades train).

#4. Regime-gate extensions (Rounds 17–19)

Round 17 — PEAD × credit (reports/edge_hunt17.md). Single credit gate on PEAD: REJECT (1.61 vs 1.97, corrEqMom 0.71 — no diversification). EqMom × credit: REJECT on Sharpe (0.96 vs 1.06) despite the maxDD cut (−26% vs −39%). Double gate (VIX calm AND credit calm): MARGINAL — p=0.014, maxDD halved (−10.3% vs −20.7%), corrEqMom 0.40, at the cost of Sharpe (1.14 vs 1.97). Same de-risk profile as the round-15 VIX gate. Promotion study (reports/pead_v3_promotion.md): the 50/50 EqMom blend wins on maxDD (−13.7% vs −23.0%) but loses on Sharpe (1.72 vs 1.88) → NO PROMOTION as a sleeve; kept as a documented de-risk variant.

Round 18 — crypto X-sec rotation (reports/edge_hunt18.md). Top-k trailing-momentum rotation across SOL/XRP/LINK/AVAX/DOGE (~2y daily-from-1H, lag-1, 10bps): REJECT — every cell negative in a −49% alt universe, p≈0.5. Diagnostic: mean pairwise alt correlation 0.73 (LINK-AVAX 0.84) — the equity edge_audit verdict repeated: rotation among near-identical betas cannot work. Parked until correlations break.

Round 19 — gate60 × equity regime + spike reversal (reports/edge_hunt19.md). gate60-ETH/BTC × credit/VIX/double-calm: all REJECT (best: BTC × credit-calm uplift +0.11, p=0.47). The spike leg deserves its own section:

#5. Retraction anatomy: the VIX-spike reversal PASS that wasn't

Initial finding: SPY 1-day reversal held only when vix_ratio > 1.2 — OOS Sharpe +0.88 vs −0.88 unconditional, p_self 0.002–0.009 at every threshold 1.1–1.5, IS/OOS consistent. Apparent PASS.

The cost-drag pass (validation/spike_reversal_costs.py) exposed the flaw: the study series never multiplied the lagged signal by day-t return — it averaged the signal (−r[t−1] on spike days), which harvests drift bias in down markets rather than reversal alpha. It was not tradable P&L. Correct construction (w[t] = −sign(r[t−1])·spike, P&L = w·r, turnover-costed) gives OOS Sharpe +0.53 gross (p=0.041), +0.43 at 2bps (p=0.078), dead at 5bps → amended verdict MARGINAL, no sleeve. The file validation/rev_sign_check.py holds the corrected battery, and the retraction is recorded in §"CORRECTION" of reports/edge_hunt19.md.

This is the program's most valuable catch: a result that passed significance, consistency, and threshold-robustness checks — and was still wrong, in a way only position-exact cost accounting reveals.

#6. Lessons

1. Allocation across drift streams has no learnable signal. Four RL generations confirm: the optimum of a cost-aware allocator over long-biased sleeves is EW itself. Stop formulating RL as allocation.
2. Regime states must separate sleeve returns, not market returns. The GMM state weakly separated SPY yet left EqMom/PEAD untouched — selection sleeves are immune to market timing by construction.
3. Blend weights from trailing IC chase noise. The fine-tune adapter had a causal, sensible design and still diluted. Adapters need floors, shrinkage, or full-scale validation — never naive IC weighting.
4. The cost pass must mirror the exact tradable position series. Significance + consistency + threshold-robustness all passed on a non-tradable construction (§5). Book the position series first, Sharpe second.
5. Paired bootstrap is underpowered for sparse gates. A gate active 25% of days dilutes its diff series with zeros; supplement with the self>0 test and the conditional (calm-vs-stress) test, and report all three as Round 16 did.
6. Correlation diagnostics are cheap verdicts. The 0.73 alt-correlation matrix settled Round 18 faster than any backtest. Run it before the study, not after.
7. Selection is where learning works. The only ML that earns its keep in this repo is supervised stock selection (flagship h5: 21d Spearman +0.053, CAGR 18.85%, PIT-clean, live). Every allocator/timer built on top of selection has failed; selection itself never has.

#7. What remains positive / next

Live edges unchanged: flagship selection, EqMom, PEAD (+VIX-gated v2 sleeve), Treasury diversifier, carry×basis sizing, depth filter, maker execution. MARGINAL shelf (de-riskers, not replacements): PEAD double gate, spike reversal at 1bp, EqMom+GLD_bh blend (+0.23 Sharpe, maxDD −23.5% vs −38.6%, paired p=0.51 — diversifier properties only), marginals blend (PEAD-double-gate + spike-reversal + GLD, vol-scaled EW: Sharpe 1.84 vs best leg 1.36, maxDD −5.5%, legs ~uncorrelated — but paired p=0.51, so economics without significance).

Round 20 (reports/edge_hunt20.md) also closed three more families: GLD 126d-TSMOM REJECT (0.76 vs 0.90 buy-hold — trend adds nothing; GLD buy-hold itself is the diversifier, OOS 0.90/corrEqMom 0.15); EqMom × fast-credit-change REJECT (good IS 0.93, decayed OOS 0.95 vs 1.06); EqMom × curve-inversion REJECT (never fires IS, hurts OOS). P4 vol-weight question found already answered: combined_book2 has EW 1.62 vs risk-parity 0.66 — equal-weight stays.

Round 21 (reports/edge_hunt21.md): marginals blend REJECT-formal (best economics in the program, p=0.51); BTC/ETH lead-lag REJECT (cross-corrs −0.05/−0.07, all variants maxDD −95%); crypto spike reversal REJECT (0.02/−0.25, equity VIX does not gate crypto). Done since: constrained adapter REJECT, macro variant MARGINAL, 51-name scale-up REJECT (§3). Open: news-weight re-run at 60+ corrtrack days (currently 3), OKX-depth entry timing at 90-day warmup.

Round 22 (reports/edge_hunt22.md): overnight/intraday EqMom split REJECT as sleeve (both legs positive: overnight 1.37/−6.7%, intraday 1.00/−14.0%); earnings-exclusion REJECT decisively (earn-window days earn +13.3bps/d vs +5.7bps/d, 51% of cum P&L — exclusion would remove the alpha); TLT/GLD lead-1 SPY overlays REJECT (−0.54/−0.27).

Round 23 (reports/edge_hunt23.md): pre-earnings drift REJECT (1.05 vs market 1.01 — beta participation); vol-scaled EqMom REJECT (0.92 vs 1.06, same maxDD); 5-sleeve +GLD book MARGINAL (1.27 vs 1.10, maxDD −21.8% vs −25.9%, paired p=0.50).

#References

• RL: reports/rl_v4_report.md, reports/rl_v4_oos.md, reports/rl_v5_oos.md
• Timing: reports/edge_hunt16.md
• Fine-tune: reports/pooled_transfer.md, reports/pooled_finetune.md
• Gates/rotation: reports/edge_hunt17.md, reports/edge_hunt18.md, reports/edge_hunt19.md, reports/spike_reversal_costs.md, reports/pead_v3_promotion.md
• Code: models/rl_env_v{4,5}.py, models/rl_v5_baseline.py, validation/rl_v5_oos.py, validation/edge_hunt1{6,7,8,9}.py, validation/pooled_finetune.py, validation/pooled_only_from_cache.py, validation/{spike_reversal_costs,rev_sign_check,rev_spike_check,pead_v3_promotion}.py
• Prior paper: reports/edge_research_paper.md