Synthesis review. No new backtests here — it compiles the verdicts of the
daily-horizon crypto signal program (edge-hunt 24 F-family, 27 small-edge,
28–31, and this session's 36–41) and distills the recurring failure modes into
a pre-screen bar. Sources are the individual reports/ cited inline and
docs/ROADMAP.md.
Why this exists
This session added six daily crypto studies (36–41) and all six were REJECT. That is not bad luck: the SAME three or four failure modes recur across almost every prior crypto candidate. Writing them down once means the next candidate can be screened in minutes instead of rediscovering them in a full study.
Scoreboard (daily crypto, adopted = drawdown-neutral or better)
| id | idea | data | verdict | failure mode |
|---|---|---|---|---|
| F1 | tape-toxicity skip | Binance tape | REJECT | lift -0.21, p 0.86 |
| F2 | OI-collapse + taker exhaustion | Binance OI/taker | REJECT | wrong sign (stress precedes bounces) |
| F2b | OI-expansion sizing | Binance OI | REJECT (near-miss) | lift +0.06, p 0.065, sparse |
| F4 | VIX-regime gate | FRED/VIX | REJECT | IS-only, OOS lift 0.00 |
| F4b | credit-spread gate | FRED | REJECT | inverted signature |
| F6 | depth-filter on alts | Binance depth | REJECT | p 0.60, WF VOID |
| F7 | F&G contrarian skip | F&G | REJECT | placebo worse than random (p 0.99) |
| F8 | DVOL-spike veto | Deribit DVOL | REJECT | IS sign flip |
| F9/F9b | stablecoin vetoes | DeFiLlama | REJECT | never fires / both tails beat middle |
| S1 | basis-only sizing | Hyperliquid | REJECT | p 0.053 and IS -0.67 (leakage pattern) |
| S2 | F&G skip | F&G | REJECT | p 0.148 (placebo kills it) |
| R1 | session/vol/noise gate | klines | REJECT | IS wrong sign, p 0.44 |
| R2 | daily swing breakout | klines | REJECT | textbook IS mirage (1.26→0.35) |
| R5 | BTC→alts diffusion | klines | REJECT | cost-killed (179 turns/yr) |
| R6 | funding-ride | Hyperliquid | REJECT (near-miss) | placebo = momentum proxy |
| R7 | weekend sizing | calendar | REJECT | cuts winners |
| R8 | stablecoin directional | DeFiLlama | REJECT | decisive wrong direction |
| 36 | SPY→crypto diffusion | SPY | REJECT | lookahead trap (window overlap) |
| 37 | on-chain PUELL timing | on-chain | REJECT (near-miss) | 0.79-corr trend proxy |
| 38 | Coinbase premium | Coinbase/Binance | REJECT | IS/OOS sign flip |
| 39 | VRP / low-vol timing | BVOL | REJECT | IS-heavy, placebo p 0.18 |
| 40 | macro factors (DXY/GLD/TLT/SPY/VIX/TNX) | ETF prices | REJECT | clean null, |IC|≤0.03 |
| 41 | cross-sectional premium carry | premium index | REJECT | cost (turnover 1.3x/day) |
The five failure modes (and the tell for each)
-
Alignment leakage. A signal and its target share a window: the close-to-close join of a same-calendar session (36: naive SPY→BTC gave a spurious OOS SR ~3.0 / IC +0.32), or an unshifted basis/flow series (S1: +0.38 → +0.08 and IS flips once lag-1). Tell: the effect dies when you move to a held window (
[t, t+1]) or lag by one bar. Rule: every study prints the lag-1/held-window version alongside the unshifted one. -
Trend / momentum proxy. Crypto daily returns are dominated by one common trend factor. F2/F4b/F9 found stress quartiles have the best forward returns; R6 funding-ride matched a frequency-matched 7-day momentum gate; 37 PUELL is 0.79-correlated with price/1y-MA. Tell: corr(signal, MOM365) is high and partial-IC over momentum is ~0. Rule: report corr(signal, MOM365) and partial-IC | momentum; on-chain/funding/flow transforms of price are especially suspect.
-
Cost kill. R5 diffusion (179 turns/yr), 41 cross-sectional premium (1.3x/day at 15bps ≈ 20bps/day), D1 ORB (12.6bps/trade < 20bps RT). Tell: gross IC positive, net negative, and turnover × cost ≥ gross edge. Rule: net@10/25bps with turnover is reported before any promotion talk.
-
IS/OOS sign flip. 36 (IS/OOS edges differ), 38 (IS +0.46, OOS -0.66), F4/F8, the equity VIX/credit veto family. A long/flat overlay in a bull IS window gives up return; a bear OOS window rewards de-risking. Tell: IS and OOS edge signs disagree. Rule: IS same-sign is mandatory.
-
Placebo / sparseness. F7 is worse than skipping random days (p 0.99); R6/D1/41 do not beat a permutation/random null; F9 fires 0 times, F2b has 17 boost days, R4 23. Tell: the candidate never faced a random null, or has too few active events to have power. Rule: permutation (or random-skip/rank) p < 0.05 and an explicit active-event count.
Pre-screen bar for a new crypto daily candidate
Run these five checks BEFORE a full study. Any hard fail = reject on the spot.
- A. Alignment: PIT/held-window assert plus the lag-1 sensitivity. (mode 1)
- B. Incrementality: corr(signal, MOM365) and partial-IC | momentum/EW-beta. If it does not beat its own price-momentum proxy, stop. (mode 2)
- C. Economics: gross, net@10bps, net@25bps, turnover; net must survive. (mode 3)
- D. Stability: IS<split/OOS>=split, same sign, bootstrap p < 0.05. (mode 4)
- E. Null + power: permutation/random-null p < 0.05 and ≥ ~50 active events (or state the study is underpowered). (mode 5)
This is the same discipline the track already uses for equity candidates
(placebo_audit.md); the point is that it must apply to crypto too.
What is NOT closed by this
- Genuinely exogenous series that are not price transforms: the only untested family. The feeds are now in place (on-chain, ETF flows, Coinbase, BVOL, premium index) but each has been used as a level/transform of the same underlying, which is why they proxy price. A new candidate needs a mechanism whose PIT series is not a function of price.
- Cross-sectional with genuinely persistent signals. 41 was cost-bound at daily turnover, but the 5-day rebalance also turned gross negative (-1.11), so the premium-z edge was fast/noise, not slow. A slower book only helps if the underlying carry is genuinely persistent — the premium z is not; a different carry construction would be needed.
- Execution vs signal alpha. Mid-price microstructure signals exist (microstructure batch-1: OFI mid +0.19bps at 300s), but the spread eats them. That is an execution problem (maker fills), not a signal problem.
Recommendation
- Treat modes 1–5 as the entry gate: no candidate reaches a full study without a one-page pre-screen (A–E) in the ledger.
- Prefer exogenous mechanisms over price transforms; if the signal is a monotone function of price/funding/flow level, assume mode 2 until proven otherwise.
- The remaining real gap is a signal whose PIT series is genuinely exogenous (not a price/funding/flow level) or structurally persistent; 41's premium-z was neither (fast and cost-bound). Everything else tested to date is a price transform and should be assumed mode 2.