Purpose: find temporary stock-level mispricing created by liquidity/order-flow shocks. This document is the mandatory pre-implementation audit. No strategy exists yet; this only records what data is actually available and reliable.
Verdict up front
| Question | Answer |
|---|---|
| Do we have true order flow (trades+NBBO, signed volume, OFI, spread)? | YES, via Alpaca SIP (data/equity_ticks.py), entitled and verified 2026-09-20. |
| Is it cached broadly? | NO. Cache is AAPL only, 22 symbol-days (data/parquet_micro/). |
| Do we have Level-2 depth? | NO. Alpaca has no historical deep book; only live polled snapshots. |
| Broadest reliable panel? | 499 symbols × 3y of 30-min bars (data/parquet/daybot/30Min_3y.parquet). |
| Minute bars? | 100 symbols × 1y (5-min); 24 symbols × ~4mo (1-min). |
| Sector / industry? | GICS sector+industry for 497 names, but only 97 of the 499 30-min universe. |
| PIT universe (survivorship)? | YES — monthly S&P 500 membership 2000-2026 (pit_sp500_membership.json). |
| News? | Alpaca NewsClient probe (data/alpaca_news.py); cache currently empty. |
| Market/sector ETFs? | SPY daily + 30-min + 1-min; QQQ/IWM/VIX/TLT daily; SPY/QQQ/VIX 1-min. No sector ETFs cached. |
Conclusion: true order flow exists but is a single-name, 22-day sample. Every broad panel is OHLCV-only. Per the brief this means two separately labelled tracks, never conflated:
- Track A — OHLCV liquidity-pressure PROXY (broad: 499 sym × 3y, 30-min; plus 100 sym × 1y, 5-min). Not order flow. Factors may only use price, volume, dollar-volume.
- Track B — TRUE microstructure (AAPL 1-second NBBO/trades: spread, size imbalance, Cont OFI, Lee-Ready signed volume). Single name, 22 days → underpowered by construction; used to validate/reject Track A's proxy, not to claim a cross-sectional edge.
Dataset inventory
True order flow — data/parquet_micro/{SYM}_{YYYY-MM-DD}_1s.parquet
- Source:
data/equity_ticks.py— Alpaca SIP historical trades + NBBO. - Cached:
AAPLonly, 22 files, 2026-08-03 … 2026-09-15 (non-contiguous). - Grid: 1-second RTH (09:30:00–15:59:59 ET), 23,400 rows/day.
- Columns:
mid bid ask bsize asize spread imbalance ofi nq vol sv tcount notional micro vwap.spread= ask−bid (mean/sec),imbalance= (bsize−asize)/(bsize+asize),ofi= Cont et al. event-level OFI summed per second,sv= signed volume (Lee-Ready vs prevailing NBBO, tick-rule fallback),micro= size-weighted microprice.
- Raw density ~670k quotes + ~660k trades per liquid symbol-day; pull ~2.3 min/symbol-day. Fails closed on 401/403.
- No Level-2 depth. No continuous multi-level book. No historical deep book.
OHLCV bars — data/parquet/daybot/
| File | Symbols | Window | Resolution |
|---|---|---|---|
30Min_3y.parquet | 499 | 2023-08-09 → 2026-08-13 | 30-min (incl. extended hours → filter RTH) |
5Min_365d.parquet | 100 | 2025-08-13 → 2026-08-13 | 5-min |
1Min_120d.parquet | 24 | 2026-04-14 → 2026-08-12 | 1-min |
1Day_600d.parquet | 24 | — | daily |
Columns: timestamp open high low close volume symbol. No bid/ask, no trade | |||
| size, no direction, no depth. |
Daily adjusted — data/parquet/{SYM}_ADJ.parquet
- 503 symbols, ~6.1y (2020-07 → 2026-09),
open high low close adj_close volume dividend split. Delisting-inclusive pool (framework.load_close_volume).
Auxiliary
data/cache/sectors_full.json— 497 names, GICSsector+industry. Overlap with the 499-name 30-min universe: 97. Sparse.data/cache/pit_sp500_membership.json— monthly PIT membership 2000-2026.data/parquet/{SPY,QQQ,IWM,VIX,TLT}_*.parquet,SPY_30Minvia daybot.data/alpaca_news.py— timestamped news fetch (earnings/analyst/macro filters noted);data/cache/event_study/news/is empty.- Fundamentals companyfacts present (market-cap computable); no direct mktcap cache.
Reliability notes / hazards
- Do not manufacture order flow from OHLCV. Track A factors are named
*_proxyand kept separate from Track B. - 30-min cache contains extended hours; RTH filtering is mandatory.
- All bar timestamps are UTC in the raw repo convention, but the daybot pooled
caches are already tz-aware
America/New_York. - Sector confirmation is limited by 97/499 GICS coverage; use correlation peers or residual-vs-market as fallback.
- Survivorship: backtests over the fixed 499/503 lists exclude delistings
except the
_ADJpool; use PIT membership for OOS honesty where it matters.
Plan implied by the audit
- Track A event study on
30Min_3y(499×3y): return/volume/impact proxy shocks, forward returns, market + sector confirmation, costs. This is the power test of whether the phenomenon exists at all. - Track B event study on
parquet_micro(AAPL 1s): true OFI/spread/depth channels, validates the proxy and tests reversal-vs-continuation directly. - Only if an event-level effect survives → cross-sectional portfolio + OOS + placebo. No production changes regardless of result.