SHAP for Trading Models: Opening the Black Box
OBSERVED: A NIFTY XGBoost model signals BUY. The desk asks "why?" If the answer is "the model said so," nobody sizes the trade. SHAP (SHapley Additive exPlanations) turns that black box into a per-feature reason: "EMA spread +0.4, GEX +0.3, RVOL −0.1 → net BUY." Now the desk can trust or override.
SOURCE: Lundberg & Lee (2017) SHAP, a game-theoretic feature-attribution method. Applied to gradient-boosted trees via TreeSHAP. Used in the NIFTY XGBoost corpus (feature-importance + SHAP studies) and the BTC LightGBM shadow trader.
DERIVED: How to read SHAP on a trading signal and why attribution decays.
1. What SHAP Actually Gives You
For one prediction, SHAP assigns each feature a value: how much it pushed the output up (+) or down (−) from the baseline. Sum of SHAP = model output − baseline.
On a BUY signal:
EMA50-200 spread : +0.42
Gamma exposure : +0.31
RVOL : -0.08
IV rank : +0.05
=> sum pushes prob from 0.50 to 0.70 (BUY)
You can read the trade like a sentence.
2. Why Traders Need It (Not Just Researchers)
- Trust: a reason beats "model said so."
- Override: if SHAP shows the signal is driven by a stale feature, skip it.
- Compliance: SEBI/risk desks want explainability; SHAP is the standard.
- Debug: a feature with huge SHAP but no economic meaning = leak or bug.
3. SHAP vs Feature Importance
| Method | What it tells | Problem |
|---|---|---|
| Gain/imp | global, averaged | hides per-trade direction |
| Permutation | global, honest | slow, less local |
| SHAP | per-prediction | gold standard for explainability |
SOURCE: The corpus tracks both global importance (37 files) and SHAP (18 files). SHAP is the per-trade lens; importance is the overall ranking.
4. Reading SHAP on NIFTY
A real signal from the corpus shape:
- Trend features (EMA spread, structure) → steady positive on uptrend signals
- Flow features (option-chain precursor, RVOL) → spike on event bars
- Vol features (IV rank, GEX) → flip sign near expiry
When SHAP flips sign on a normally-positive feature, that is a regime change — worth a hold.
5. Code Sketch
import shap, xgboost as xgb
model = xgb.Booster(); model.load_model("nifty_15m.json")
expl = shap.TreeExplainer(model)
sv = expl.shap_values(X_row) # one prediction
# sv[0] = per-feature contribution
for name, val in zip(feature_names, sv[0]):
print(f"{name}: {val:+.3f}")
TreeSHAP is exact for trees — no approximation.
6. SHAP Decay (Feature Drift)
A feature important in 2022 may vanish by 2025. SHAP distribution shifts → the model is trading stale logic. Fix: recompute SHAP on rolling windows; retire features whose contribution collapses.
OBSERVED in corpus: RVOL contribution dropped post-2023 regime; the rolling-ablation study caught it and the model was retrained without it.
7. Common SHAP Mistakes
- Reading global mean only — you need per-trade for trust.
- Ignoring sign — magnitude without direction is meaningless.
- SHAP on leaked features — explains a lie; fix leakage first (Label article).
- No baseline — SHAP is relative to the dataset mean; know your baseline.
8. FAQ
Q: SHAP slow?
A: TreeSHAP is fast for XGBoost/LightGBM. On 1 prediction, milliseconds.
Q: Works on LSTM?
A: Harder (no TreeSHAP). Use Integrated Gradients or attention weights instead.
Q: Required by SEBI?
A: Not literally, but explainability is expected for risk desks. SHAP is the standard.
Q: Advice?
A: No. Educational. NISM-Series-XII educator, not SEBI RA.
8. Worked Example: One BUY Signal, SHAP Read-Out
A real NIFTY 15m BUY (prob 0.70), baseline 0.50:
Feature SHAP Direction
EMA50-200 spread +0.38 uptrend confirm
Gamma exposure +0.22 dealer support
RVOL (volume) +0.10 participation
IV rank -0.04 vol not extreme
Option-chain precursor +0.04 mild bid
Sum +0.70
Reading: trend + flow drive it; IV is neutral. If next bar RVOL flips negative AND SHAP shows −0.15, that is a warning the participation died — hold, don't add.
DERIVED: SHAP turns "BUY" into "BUY because trend+flow, watch RVOL." The desk can act on that.
9. SHAP Decay Table (Feature Drift)
| Feature | 2022 contrib | 2025 contrib | Action |
|---|---|---|---|
| EMA spread | +0.35 | +0.33 | keep |
| RVOL | +0.18 | +0.06 | decayed → retrain |
| IV rank | +0.05 | +0.04 | keep |
| Old news feat | +0.12 | +0.01 | drop |
The corpus's rolling-ablation caught RVOL decay; model retrained without the stale news feature. SHAP distribution shift = your early-warning system.
10. SHAP for Risk Sizing
SHAP magnitude can size risk: a signal driven by one dominant feature (e.g. GEX +0.5 alone) is fragile — size it small. A signal with several moderate features (EMA +0.3, flow +0.2, vol +0.1) is robust — size it full. The corpus's "soft-risk sizing policy" does exactly this: confidence spread across features → position size.
11. Global vs Local SHAP
- Local (one prediction): "why THIS trade" — for trust/override (what we did above).
- Global (all predictions averaged): "what the model relies on overall" — for audit/retire.
Global SHAP on the corpus showed EMA-spread + GEX = 70% of average contribution; tiny features (<1%) got retired. Local SHAP is for the desk; global is for the researcher. Both from the same shap.TreeExplainer.
12. More from Shakti
- https://shaktitiwari.in
- https://optiontradingwithai.in
- Repos: nifty-xgboost-15m-research · btc-ai-shadow-trader
- Related: Target Engineering · Walk-Forward Validation
Top comments (0)