"RL v3 Meta-Allocator — REJECT (final ledger entry)"

#Scope

Train a PPO agent to PICK from a 5-rule hand-coded rule set (EW, risk-parity, vol-target, credit-gate, trend-filter) instead of outputting a continuous weight vector. The meta-allocator changes the problem formulation: discrete action (5-way), simpler than the continuous 4-dim v3 action, and the rules are pre-validated candidates the agent can choose between.

#Verdict: REJECT

15 cells (5 walk-forward windows x 3 seeds). The agent picked EW 62% of the time and was statistically indistinguishable from the equal-weight baseline on every cell. The rule-switch cost alone (35.7 switches per 126-day OOS cell at 25 bps/switch = ~7 bps/day = 17% annualized drag) makes the net result worse than gross.

#Pre-registered pass bar (from rl_v3_hypothesis.md)

ADOPT requires ALL THREE:

• OOS Sharpe > equal-weight baseline Sharpe + 0.10
• Bootstrap p (agent > baseline) < 0.05
• OOS max drawdown < 0.30

#Pipeline summary

#OOS scoreboard (15 cells)

Aggregate: median uplift -0.254, max bootstrap p 0.646, worst maxDD -20.10%.

#Mechanism audit

The mean agent allocation is within 0.02 of EW on every sleeve. The 5-17% rule choices (TREND_FILTER, VOL_TARGET, RISK_PARITY, CREDIT_GATE) collectively moved weights by ~1.5% per sleeve. That magnitude is too small to register against the noise in 6mo OOS windows.

The agent does WORSE in bear regimes. The plan's pre-registered mechanism prediction (the agent should win in bear regimes) is not observed. This is a stronger failure than v3, where the agent was at least better in bear regimes (marginally).

#Net-cost analysis

The agent switched rules 35.7 times per OOS cell on average (35.7 / 126 = 28% of OOS days). At 25 bps per switch, that's ~7 bps/day or ~17% annualized drag. This single factor explains why the net result is uniformly worse than gross: even when the gross was slightly positive (window 0 seed 1: +0.08 uplift), the cost overwhelms it (net: -1.67).

The cost could be reduced by adding a hold_period (action frozen for N bars after any rule change) — but that is a different design than what was tested here.

#Stop conditions (hypothesis doc)

• S1 (baseline Sharpe > 0.3): NOT triggered. EW median OOS Sharpe on the 13-window baseline was +0.61, well above the 0.3 floor.
• S2 (median agent uplift >= 0): TRIGGERED. Median uplift -0.254.
• S3 (net uplift >= 50% of gross): TRIGGERED. Net is 0% of gross (in fact, deeply negative).

#Combined RL study verdict (v3 + v4 + meta-allocator)

This is the third RL arm in this repo to close at REJECT. The pattern is now consistent:

Across all four arms:

• The agent learns something (mechanism PASS in v3, v4)
• But the magnitude is too small to clear p<0.05 on OOS
• And the cost (whether modeled in env or post-hoc) overwhelms any marginal edge
• And EW is genuinely hard to beat (median Sharpe +0.61 on the 13-window baseline, maxDD well within 30%)

This is now a settled finding for this sleeve set and data window. A future RL attempt would need either a different sleeve set (structural alpha the agent can exploit) or a different baseline (EW is the right null to beat; the question is whether the agent's edge is real or noise).

#What this means for the live book

Nothing changes. The validated equal-weight baseline (production/book_allocator.py) stays the production default. No production/ runner is registered. No live orders. No new data dependency. The v3, v4, and meta results all converge on the same conclusion: an RL allocator does not add value to this sleeve set / data window / cost model.

#What we WOULD change before another RL attempt

• Different sleeve set. The eqmom flagship, gate60 crypto trend, and SPY bench are all positively correlated equity-like bets. The agent has no source of decorrelated alpha to exploit. A sleeve set that includes a genuine diversifier (trend-following on TLT, carry on FX, volatility-selling) would give the agent something to allocate.
• Different baseline. EW is hard to beat. A weaker baseline (e.g. fixed-50%-equity) would be easier to improve on, but would not produce a useful live deployment.
• A cost model that the agent can SEE during training. The meta-allocator's high switching cost was a surprise; the env didn't include it. Baking the cost into the reward (as v4 did) is necessary for the agent to learn to hold positions.
• A hold-period constraint. If the action is discrete, the agent should be required to hold for N bars after any switch. This is a small change but it bounds the cost drag.

#Files this study produced

#Date locked

2026-09-04