Options Buyer ML: Why One Model Fails (and the V2 Fix)

Lessons from a real rebuild of an options-buyer prediction system. No profit claims —
just the architecture that fixes the chronic bugs of V1.

The Core Mistake in V1

V1 asked one XGBoost model one big fuzzy question: "CE ya PE?" — directly from raw
CE/PE premium data. Premium is a transformed signal (underlying move × delta × gamma × IV ×
theta × spread × strike distance × liquidity). The model learned noise as much as signal.

Concrete evidence from the research logs:

Balanced accuracy stuck at 51–61% for months — hyperparameters were never tuned (lr=0.02, depth=3 defaults used throughout; Optuna existed but was never run).
A partition bug (iv_change_1d shift inside single-row groups) silently zeroed a whole feature for the entire history.
A rollup config flag compressed 15-minute bars into 1 row/day, destroying 760× of training volume (387 sequences instead of 295K+).
Live paper trading: 31.6% win rate, −₹90.3k PnL, entry confidences only 55–64%.

V2 Principle: Split the Question

underlying mechanics  -->  side, range, ETA, invalidation
option chain scanner   -->  is the buyer contract worth paying for?
XGBoost (many heads)   -->  thin calibrated learner on clean mechanics

Rule: underlying decides side; option contract decides execution eligibility. CE/PE
premium is validated against, never learned as, direction.

Many Shallow Heads, Not One Deep Model

Instead of one CE/PE answer, V2 trains separate narrow heads:

underlying_up/down_touch_{15,30,60}m
ce_1p3x / ce_1p5x / ce_2p0x and pe_1p3x / pe_1p5x / pe_2p0x (SEPARATE CE and PE)
no_trade_quality

This single change removes most of the CE/PE confusion V1 fought for months.

The Shallow Regularized Grid (the actual fix for overfit)

learning_rate = 0.015–0.035   n_estimators = 800–2000 (early stop)
max_depth = 2–3               min_child_weight = 12–40
gamma = 0.1–2.0               subsample = 0.65–0.90
colsample_bytree = 0.55–0.85  reg_alpha = 0.5–3.0
reg_lambda = 6.0–20.0         scale_pos_weight = min(neg/pos, 8.0)

V1's intraday head had only 8 of 1280 features with non-zero gain — most of the bloat
was pure noise the regularizer had to prune. Shallow + hard-regularized is the answer.

Overfit Gate (Hard Promotion Rule)

overfit_gap = train_metric − test_metric. Flag if > 0.15. A model is NOT promoted just
because train metrics look good. Log the gap automatically on every head, every retrain.

The Honest Verdict

V2 is a cleaner architecture, but it is still research. The lesson that transfers: stop
asking fuzzy questions, declare your nulls, keep trees shallow, and gate promotion on
out-of-sample gap — not training score.

Research only. Not investment advice.

DEV Community