Backtesting Pitfalls in Options: 7 Ways You Lie to Yourself

By Shakti Tiwari · Educational only · Not investment advice

A backtest is a story you tell yourself about the future. The danger is not that the story is false — it is that a convincing backtest can survive contact with reality for years before it quietly blows up your account. This article lists seven of the most common ways options backtests lie, why each one happens, and how to structurally prevent it. No live market numbers are quoted; the point is the methodology, which does not expire. If you have ever presented a backtest you later distrusted, at least one of these seven was in it.

1. Look-ahead bias through indicator leakage

The cleanest lie is also the most common: an indicator uses information that would not have been available at the decision time. In options, this shows up as using the day's closing volatility to size a position taken at the open, or computing a feature from the full bar when the signal fired mid-bar.

Structural fix: timestamp every feature with the bar it could have been computed at, then shift features so that at decision time t, only data from t-1 or earlier is visible. Walk-forward: train on [0, k), validate on [k, k+m), never the reverse. If your pipeline cannot tell you the exact bar a feature was computed at, it has leakage by default.

2. Survivorship bias in the universe

If you select constituents that exist today and backtest on history, you have deleted every company that delisted, bankrupted, or was acquired unfavorably. In indices this is milder because the index is maintained, but for single-name options or a self-built universe it is fatal.

Structural fix: use point-in-time membership. For Nifty, use the index provider's historical constituent list at each rebalance, not today's 50 names. For single names, source delisted instruments too. A backtest that only trades survivors is measuring a portfolio that never existed.

3. Ignoring transaction costs and slippage

A short-option strategy that "wins" on mid prices can lose money the moment you pay the bid-ask spread twice per round trip plus brokerage plus STT (securities transaction tax on the sell side in India, which is not symmetric). Options spreads are wide; ignoring them is the difference between a Sharpe of 2 and a loss.

Structural fix: model the worst realistic fill — assume you cross the spread, add a fixed per-lot fee, and add the regulatory tax explicitly. If the strategy's edge is smaller than its round-trip cost, the backtest is entertainment, not analysis.

4. Overfitting via too many parameters

Seven indicator parameters, three regime filters, and a vol threshold tuned on ten years of data will fit the noise. Out-of-sample performance decays the moment you stop optimizing. This is leakage of a different kind: not temporal, but parametric.

Structural fix: count your free parameters. The more you have relative to the number of independent trades, the more suspicious the result. Prefer structurally motivated rules (e.g., "sell vol after a 2-sigma move") over a grid-searched optimum. If you must tune, use a single holdout block you never touch during development.

5. Path dependency and settlement timing

Options expire at a fixed time; cash settles on a fixed cycle. A backtest that marks positions at close but ignores the gap between expiry and settlement can misstate both P&L and margin requirements. Assignment, exercise, and physical settlement are discrete events, not continuous marks.

Structural fix: simulate the actual event calendar. Model expiry → assignment probability → settlement date. If your P&L jumps on a day nothing traded, you have a settlement bug.

6. Volatility regime blindness

A vol-selling strategy looks genius from 2013 to 2017 and suicidal in 2020. Backtesting on one regime and assuming it generalizes is a lie of omission. The backtest is not wrong; it is incomplete.

Structural fix: report performance per volatility regime separately (low/normal/high/ crash), not just an aggregate. If the strategy loses money in any regime you might live through, size for that regime, not the average.

7. Selective reporting (p-hacking the narrative)

The last lie is you: you run twelve variants, keep the one with the prettiest equity curve, and present it as "the strategy." The other eleven never appear. This is not a code bug; it is a discipline bug. The fix is boring and effective: write the losers down. A strategy you can defend is one whose failures are visible, not one whose successes are curated.

Structural fix: pre-register the hypothesis. Write down what you will test and what would falsify it before running the code. Report the losers alongside the winner. A research log that only contains successes is a diary of confirmation bias.

A minimal governed backtest loop

A defensible options backtest does five things structurally:

1. Point-in-time universe and features (no look-ahead, no survivorship).
2. Walk-forward splits with a frozen holdout.
3. Explicit cost model including STT and spread crossing.
4. Per-regime reporting, not just aggregate.
5. A research log that records failed variants.

Any one of these missing means the number you are proud of is, at best, an upper bound on what you will earn.

How this connects to the ML pipeline

The companion articles cover the building blocks: idempotent tick ingestion so your feature store is reproducible, leakage-free XGBoost features so your model does not cheat, and a data pipeline from WebSocket to clean store. None of those matter if the backtest that consumes them has the seven lies above. The model is only as honest as the loop around it.

A worked example: the vol-seller that "always wins"

Consider a strategy that sells a strangle when realized vol is "high" and buys it back when vol normalizes. On mid prices from 2015 to 2019 it shows a 95% win rate and a beautiful curve. Then you add the three things above:

• Spread crossing on entry and exit (options spreads are wide, especially away from at-the-money).
• STT on the sell leg (asymmetric, India-specific, cannot be ignored).
• A 2020 crash block (vol exploded, assignment happened, margin called).

The 95% win rate survives, but the two losing trades now exceed all the winners combined. The backtest did not lie about the win rate; it lied by omission about the regime. This is why per-regime reporting is not optional — it is the only honest summary.

Why "more data" does not fix this

A common coping response is to extend the backtest to thirty years. That helps only if the extra years contain the regimes you fear. If your thirty years still exclude a vol crash, a liquidity freeze, and a corporate-action edge case, you have a longer story, not a truer one. Length is not a substitute for coverage of failure modes.

Position sizing is part of the backtest

A backtest that assumes you always deploy 100% of capital is testing a position, not a policy. Real risk comes from sizing: the same edge with 2% risk per trade survives a crash, with 50% risk it does not. The backtest must simulate the sizing rule, not just the signal, because the sizing rule is where ruin lives.

The research log is the missing artifact

Most backtests are reproducible in code but not in thought. You can re-run the notebook and get the same curve, but you cannot see the eleven discarded variants that would have falsified your confidence. Keep a log: hypothesis, data window, parameters, result, and what would have changed your mind. A notebook without this is a result; a log with it is analysis.

What "good enough" looks like

You do not need a perfect backtest — you need one whose known limitations are written down. State the regimes you did not cover. State the costs you approximated. State the parameters you tuned. A backtest with honest scars beats a flawless one that hides them.

A pre-publish checklist for your own backtest

Before you trust a number, answer each:

• Can you name the exact bar every feature was computed at? (No → leakage.)
• Does the universe include delisted names? (No → survivorship.)
• Is STT modeled on the correct leg? (No → cost lie.)
• How many parameters did you tune, and on what holdout? (Many, none → overfit.)
• Are expiry, assignment, and settlement simulated as events? (No → path bug.)
• Does it report per-regime, not just aggregate? (No → regime blindness.)
• Is the discarded-variant log attached? (No → selective reporting.)

Seven questions map to the seven lies. If any answer is "no" or "I don't know," the backtest is a draft, not a decision.

Code smells that signal a lying backtest

• Features built from df.shift(0) or computed after the target row.
• groupby on symbol without point-in-time awareness.
• A future column that leaks the next bar's close.
• Costs set to zero "for simplicity."
• A single aggregate Sharpe presented without drawdown-by-regime.
• random_state tuned until the curve looks good.

These are not stylistic complaints; each one is a mechanism for the lies above. Reviewing for them takes minutes and saves accounts.

What to do instead: a defensible minimal loop

If you take one thing from this article, take the loop. A defensible options backtest, even a simple one, does five things structurally and refuses to skip any:

1. Point-in-time everything. Universe, features, and targets are all stamped with the bar they were observable at. No future information enters a past decision.
2. Walk forward, frozen holdout. Train on the past, validate on the immediate future, and keep one block you never touch during development. Report on it last.
3. Costs are real. Spread crossing, brokerage, and STT on the correct leg. If the edge is smaller than the round trip, the strategy does not exist.
4. Regimes reported separately. Low, normal, high, crash. Aggregate Sharpe is a sales pitch; per-regime is analysis.
5. Failures logged. The discarded variants are part of the result, not an embarrassment to hide.

None of this requires a PhD or a cluster. It requires refusing the seven comforts above. The backtest that survives this list is still not a guarantee — no backtest is — but it is a number you can defend when someone asks how you knew.

The honest bottom line

Backtesting does not predict the future; it exposes whether your logic survives its own assumptions. Most backtests fail not because the market is unknowable, but because the tester quietly removed the parts that would have disagreed. Lie to the backtest and it will lie back, on the worst possible day. Build it to argue with you instead, and the number you get is at least yours.

A 12-line mental model you can reuse

When in doubt, simulate this order and never reorder it:

for each day t:
    features = compute(df[ : t-1 ])        # only past visible
    signal  = model.predict(features)      # no future column
    if signal and risk_budget_ok():
        fill = cross_spread(signal)        # pay the cost
        pnl   = mark(fill, df[t]) - cost   # realized, not mid
    if expiry_event(df[t]):
        assign_or_expire()                  # discrete, not continuous
    log_variant(outcome)                    # keep the losers too

The discipline is in the brackets: df[: t-1] not df, cross_spread not mid, expiry_event not a smooth mark. Every lie in this article is a violation of one of those brackets. If your code cannot be squeezed into this shape, the shape is telling you something is leaking.

Related

• XGBoost feature engineering for Nifty options
• Nifty options data pipeline: from WebSocket tick to clean feature store
• What is the Nifty 50 — a structural explainer

Continue Reading

• Interactive Tools for Nifty Options Traders
• AI Trading Myth-Buster

Shakti Tiwari writes about systematic options trading and ML. Follow on X · LinkedIn · GitHub · DEV. #ShaktiTiwariOnAI #NiftyOptions #QuantML #Backtesting #SystematicTrading