Scope: add TypeSafe's Jev decision model as a validator on top of the LLM
market-research watchlist (production/llm_research.py). Built:
production/jev_client.py, production/jev_verdict.py (one parallel
system_one call/day, veto/per-name Nouls + regime/size/trust), shadow hook
in the llmresearch cycle, keyed and live (JEV_ENABLED=1). Enforcement
flags (JEV_*_APPLY) deliberately NOT wired.
Verdict
CONDITIONAL CONTINUE — no credible positive evidence yet. No enforcement.
The veto mechanism is technically sound, but the only large positive screen
(341 wks) is a momentum-echo artifact (reports/jev_deepdive.md), and the
only genuine forward test (4 live days) was negative. Keep the shadow as pure
data collection; capital unchanged.
Evidence by trustworthiness
| Test | n | Result | Caveat |
|---|---|---|---|
Live forward replay (recorded Gemma artifacts, validation/jev_backtest.py) | 4 days | D −0.27%, hit 25%; vetoes killed winners (+0.38% vs −0.13%) | tiny; wrong judge (gemini-3.8-flash) |
Proxy-history screen 2020-01..2026-07 (validation/jev_proxy_history.py) | 341 wks | H21 D +1.02%, block-CI [+0.37%, +1.60%], hit 59%; vetoed +1.19% vs kept +3.19% | largely a momentum-echo artifact — see reports/jev_deepdive.md; upper-bound hindsight |
Threshold sweep (validation/jev_sweep.py) | 36 configs | veto 0.30–0.45 all SIG, 0.35 mid-plateau; strict mode collapses to ~0 | grid = researcher DF (logged) |
| Aux questions regime/size/trust | 341 | regime 37% agreement; size backwards; trust max 0.31 | reject — log-only |
Deep-dive (validation/jev_deepdive.py) | 341 | proxy veto ≈ "drop worst-momentum" (D +1.02% vs +0.88%); p≈inverse momentum rank (−0.917); 2020-only; vetoed names +1.25%; regime no signal | the proxy screen does not test Jev |
Calibration: pooled H21 forward rises across Jev-p deciles (+1.56% bottom → +4.37% top) but pooled IC is only +0.06 — the edge is the veto tail, not ranking.
Why the two positive/negative tests disagree
Only the 4-day replay is truly forward; the 341-week screen lends Jev its own training data (tailwind) and starves the state of news (headwind). Neither is promotion-grade. The cost of the disagreement is bounded by construction: confirmed-empty days fail open to the Gemma book, so live enforcement would have been neutral on all 4 replay days.
Decisions
- Keep shadow running (zero risk, fail-open, log-only). First live row 2026-09-21; promotion-grade series is Gemma-backed (≥30 trading days).
- Keep veto=0.35, fail-open (weak/unjudged kept). Strict keep-weak=False is rejected (D→~0, 1–4% hit).
- Never gate on regime/size/trust until reworded and re-measured.
- Jev's role is veto only; ranking signal too weak to act on.
Kill criterion
At 30 live Gemma-backed days: reject if D≤0 OR the veto tail stops separating (vetoed H21 ≤ kept H21). Otherwise proceed to the enforcement vote.
Open follow-ups (this report's session)
- Risk-adjusted readout: is the edge return or risk (Sharpe/vol/maxDD)?
- Relative (per-day rank) vs absolute veto threshold — Jev p is compressed (median 0.20), so absolute 0.35 may be the wrong lens.
- Orthogonality: is Jev p just re-deriving momentum?
- Regime wording failure diagnosis (37% ≪ 50%).
Resolved in reports/jev_deepdive.md: (1) edge is concentration/return,
not risk (Sharpe 2.41 vs 2.21); (2) rank veto ≈ absolute, both ≈ momentum
drop; (3) yes — p ≈ inverse momentum rank (−0.917), so the proxy screen is an
echo; (4) regime has no signal (risk_off weeks up 63%). Only (4) remains
open as a live-series question.