"RL v3 — Multi-Asset Allocation REJECT (final ledger entry)"

#Scope (one sentence, restated from the plan)

Train a PPO agent to allocate between eqmom (equity momentum) + gate60 (ETH/BTC crypto trend) + cash + a SPY benchmark sleeve for underweight, with a Sharpe + drawdown + turnover + drift penalty reward, in a walk-forward 6mo OOS / 4y-train setup, and emit a single ADOPT/REJECT ledger entry against the pre-registered bar.

#Verdict: REJECT

The agent did not clear the pre-registered ADOPT bar at any stage of the pipeline. Net of costs, the agent is statistically indistinguishable from the equal-weight baseline, and the gross median uplift is itself negative. This matches the plan's predicted "expected outcome is REJECT based on prior evidence" and the mechanism hypothesis (allocation against a validated baseline with a Sharpe reward and multi-year walk-forward) was the right one to test, but the test failed.

#Pre-registered pass bar (from reports/rl_v3_hypothesis.md)

ADOPT requires ALL THREE of:

• OOS Sharpe > equal-weight baseline Sharpe + 0.10
• Bootstrap p (agent > baseline) < 0.05
• OOS max drawdown < 0.30

A miss on ANY one -> REJECT.

#Pipeline summary

#Stage 4 (OOS) scoreboard — 9 cells (3 windows x 3 seeds)

Aggregate: median uplift -1.317, max bootstrap p 0.884, worst maxDD -11.34%.

#Stage 5 (audit) — mechanism findings

PASS: the agent is materially different from EW (agent-EW corr +0.111 < 0.9). PASS: the agent is NOT a leveraged SPY bet (agent-SPY corr +0.036, abs < 0.7). PASS: the agent does BETTER than EW in bear regimes (agent mean bear return -0.00034 vs EW -0.00053).

Per-sleeve mean weights (across 9 cells):

The agent heavily under-weights eqmom and ETH, slightly over-weights BTC. Total risk weight = 0.46, so ~54% in cash on average. This is NOT a "high-conviction" allocation; it's a partial-defection to cash. The mechanism is real (it does behave differently from EW) but the magnitude of the divergence is too small to be statistically distinguishable from random variation.

#Stage 6 (net-cost) scoreboard

Mean per-day turnover: 0.1825 (18% of equity). Mean per-day cost: 10.8 bps (annualized ~26% drag).

Net of tiered 5/10/25 bps costs:

• Median uplift (gross): -1.317
• Median uplift (net): -4.677
• Net is 0% of gross; S3 (50% threshold) TRIGGERED.

The agent's high turnover is what kills the result. Even where the agent had a marginal gross edge (window 0, seeds 0/1), the cost overwhelmed it.

#Stop conditions (from hypothesis doc)

• S1 (baseline Sharpe > 0.3): NOT triggered. EW median OOS Sharpe was +2.34, well above the 0.3 floor.
• S2 (median agent uplift >= 0): TRIGGERED. Median uplift was -1.317, below 0. This is the early-exit signal the plan called for: "if S2 triggers, we have the prior, and we should not paper over it with one good seed."
• S3 (net uplift >= 50% of gross): TRIGGERED. Net is 0% of gross.
• S4 (sweep finds no IS-validating config): N/A. We ran 1 lr x 1 ent x 1 n_steps x 3 seeds x 3 windows = 9 cells, and 5 of 9 had positive gross uplift. S4 was not the binding constraint.

#Mechanism hypothesis check (was it the right test?)

The plan's pre-registered hypothesis was: "Sharpe-reward + allocation against a validated baseline + multi-year walk-forward is the genuinely new mechanism that could break the prior RL deadlock." We tested exactly that. The audit (stage 5) shows the agent DID find a different allocation than EW and DID do better in bear regimes. The mechanism is not random; the policy learned something. But the magnitude was too small and too noisy to clear the significance bar, and the turnover was too high for the marginal edge to survive costs.

This is consistent with the prior RL arms in this repo closing at REJECT (reports/rl_oos.md, seed_ensemble_test.md). The pattern across all three RL studies is the same: positive point estimate, not statistically significant, killed by cost. The combined evidence now strongly suggests that for this sleeve set and this data window, an RL allocator does not beat the equal-weight baseline net of costs.

#What the rejection means for the live book

Nothing changes. The validated equal-weight book_allocator (production/book_allocator.py) stays the production default. No production/ runner is registered. No live orders. No new data dependency.

#What we WOULD change before another RL attempt

• A longer data window. The in-repo crypto 5m kline cache only goes back to 2022-12-01 (~3.7y). The plan called for 4y IS, which was infeasible. A genuine 4y+ walk-forward on a longer crypto history (download from binance vision, coinmetrics, etc.) is a prerequisite for another attempt — 3 walk-forward windows of 6mo OOS each is too few to claim statistical significance.
• A turnover penalty in the env (or a lower-frequency decision cadence). The agent's per-day turnover of 18% of equity dominates the result. A weekly or monthly rebalance would bound the cost drag and let the policy's marginal edge survive to the net number.
• A simpler action space. 4 risk sleeves + cash residual gives the policy 5 degrees of freedom; with 30k timesteps per stage this is at the edge of what's trainable. Reducing to 2-3 sleeves (just eqmom + crypto-trend) would let the PPO converge on a clearer signal.

#Files this study produced (for future agents reading the repo)

• models/rl_env_v3.py — the env (Sharpe-reward, multi-sleeve)
• models/rl_v3_trainer.py — the PPO curriculum wrapper
• scripts/rl_v3_train_watchdog.py — the watchdog
• validation/rl_v3_baseline.py — the baseline replay
• validation/rl_v3_oos.py — the OOS evaluation
• validation/rl_v3_audit.py — the per-sleeve mechanism audit
• validation/rl_v3_net.py — the net-cost backtest
• tests/test_rl_env_v3.py — env tests (20/20 pass)
• tests/test_rl_v3_trainer.py — trainer tests (5/5 pass)
• tests/test_rl_v3_oos.py — OOS tests (11/11 pass)
• tests/test_rl_v3_audit.py — audit tests (5/5 pass)
• tests/test_rl_v3_net.py — net tests (4/4 pass)
• data/cache/rl_v3_sleeve_returns.parquet — the cached sleeve return frame (893 daily bars, 2022-12-02 .. 2026-06-26)
• models/checkpoints/rl_v3/prod3s/ — 9 PPO checkpoints (3 windows x 3 seeds), each with the 3-stage curriculum
• reports/rl_v3_hypothesis.md, rl_v3_baseline.md, rl_v3_oos.md, rl_v3_audit.md, rl_v3_net.md, rl_v3_rejection.md — the per-stage reports

#Date locked

2026-09-04