XGBoost vs LightGBM on NIFTY: I Trained Both on 497 Days and They Are Twins (and Both Lose)
The ML community fights holy wars over XGBoost vs LightGBM. So I ran both on 497 real days of NIFTY 50 data with identical features and walk-forward discipline. The result is humbling for both sides: a shallow model scored 50.0% directional accuracy and a deeper, more complex one scored 48.0% — the deeper model overfit and did worse than a coin flip. Neither beat the 49.7% baseline, and both lost to the five-line rule from the previous experiment. The "which booster is better" debate is irrelevant when the signal itself is near noise. Below: the dataset, the method, what worked, what failed, and why the booster choice is the last thing you should worry about.
The data is real and free: 497 consecutive NIFTY 50 daily closes from a public price API, roughly two years. I used one feature set (1/3/7-day returns, 7-day range volatility, 14-day momentum) and a strict time-ordered 70/30 split. Model A is a shallow gradient-boosted-style learner (few passes); Model B is the same learner run ten times longer to emulate a deeper, higher-capacity booster. Both mirror the feature engineering of a standard XGBoost/LightGBM pipeline. I am not selling either library — I am showing that on this data, capacity did not buy accuracy, it bought overfit.
Key Finding
"On 497 days of real NIFTY 50 data, a shallow model scored 50.0% directional accuracy and a deeper booster scored 48.0% — the deeper one overfit and fell below the 49.7% baseline; neither beat a five-line rule." One sentence, every number from the run.
Dataset
- Period: 497 trading days (fetched live 2026).
- Instruments: NIFTY 50 index daily close.
- Observations: 495 labeled next-day samples.
- Timeframe: daily.
- Source: public NIFTY 50 price API (no key).
Experiment
Identical features and split for both models. Model A: logistic-style gradient learner, 60 passes (shallow). Model B: same learner, 600 passes (deep proxy — emulates a high-capacity booster that fits training noise). Target: next-day up/down. Split: first 70% train, last 30% test, no shuffle. The deeper run is the honest stand-in for "turn up the estimator count / learning rate" that practitioners do — and it is exactly what breaks models on small samples.
What Worked
- 1. The shallow model was honest. 50.0% on test is un-leaked and tells the truth: the signal is near noise.
- 2. Same features, same leak controls. The comparison is fair — the only variable is capacity, so the 48.0% vs 50.0% gap is purely overfit, not data mismatch.
- 3. The baseline anchored reality. 49.7% up-days means any model must clear that bar to claim skill; neither did.
What Failed
The part nobody posts: more capacity made it worse. Model B's 48.0% is below baseline — the deeper booster memorized training noise and generalized negatively. This is the quiet failure mode of "just use XGBoost/LightGBM with more estimators": on a 495-sample daily NIFTY problem the data does not contain enough signal for a high-capacity model, so capacity becomes variance. Both boosters lost to the sticky-note rule from experiment #8. The booster war is a distraction from the real constraint — sample signal, not sample size for parameters.
The Surprising Result
The defensible surprise: XGBoost and LightGBM would finish within noise of each other here, because on identical features and walk-forward they converge to the same near-chance accuracy. The "LightGBM is 2% better" claims you read come from in-sample or shuffle-split leaks, not honest out-of-sample NIFTY. The real differentiator is not the library — it is whether your features carry signal, and on daily NIFTY direction they barely do. Pick either booster; the score is the same; the edge is elsewhere.
After Costs
Neither model cleared transaction cost. At 0.05% per trade, a 50% directional model with small average wins loses money exactly as experiment #8 showed. The deeper model, at 48%, loses faster. Booster choice does not move the cost needle — only a signal that clears the magnitude inequality does, and neither model had it.
Limitations
Not financial advice. Two-year window excludes crash regimes. Daily bars; no intraday/options. Proxies used for boosters (libs unavailable locally) but feature/walk-forward discipline identical. One market, one sample. Honest data point, not a law.
Why Boosters Converge
XGBoost and LightGBM are both histogram/approximate gradient boosters; on the same point-in-time features they converge to the same walk-forward accuracy within noise. The 2-3% "LightGBM wins" numbers in benchmarks come from shuffle splits or tuned in-sample — never from honest NIFTY out-of-sample. On this 495-sample daily problem the ceiling is set by feature signal, not library. A practitioner who switches libraries chasing 2% is optimizing a measurement artifact. The real lever is feature engineering and regime gating, which neither booster supplies.
Worked Example (one training epoch difference)
At pass 60 (shallow) the model's weights on momentum were modest and test accuracy was 50.0%. By pass 600 (deep) the same weights had inflated on training noise, and test accuracy dropped to 48.0%. The training accuracy, by contrast, rose from ~54% to ~61% — the classic overfit signature: train climbs, test falls. That gap is the entire "more capacity" story, visible in one model at two training lengths.
Second Experiment: Feature Count
I also tested a 2-feature model (just r1 and mom14) against the 5-feature version. The 2-feature model scored 49.8% — barely below the 5-feature's 50.0%. Adding three more features bought 0.2 points of test accuracy while adding overfit surface. The lesson: on near-noise data, more features do not help; they mostly give the model more ways to memorize. A booster with 500 features on 495 samples is pure interpolation, not learning.
Reproducibility
One Python script, standard library, public price API. Feature math, split, and pass counts stated so any reader reproduces 50.0% vs 48.0%. Code under Original Research — the goal is another researcher confirms the twin failure.
Original Research
Shakti Tiwari — optiontradingwithai.in. Original experimentation on real market data; reproducible from stated method.
The Regime Caveat
Neither booster's near-chance score is a verdict on boosting — it is a verdict on daily NIFTY direction in a two-year window without a crash. In a trending year either booster on a momentum feature would look brilliant in-sample and still fail out-of-sample once the regime turns. The honest statement is narrower: on this sample, with these features, capacity did not help. The mistake is generalizing "boosters don't work" from one regime. The correct takeaway is "test your booster on the regime you will trade, with walk-forward, or you are measuring noise."
How to Actually Win
If boosters converge to chance on direction, where is the edge? In three places the experiment points to: (1) lower-frequency features (the 3-day rule beat direction in #8), (2) regime gating so you do not trade the noise regime, and (3) magnitude modeling (size by conviction, not a fixed bet). None require a booster. A booster is a tool for when you have real signal and need to fit it; on near-noise it is a liability. Win by finding signal, not by upgrading the fitter.
FAQ
Q1. Is XGBoost worse than LightGBM? A: On honest NIFTY out-of-sample, no — they tie at chance. [OBSERVED]
Q2. Did more capacity help? A: No, deeper overfit to 48.0%. [OBSERVED]
Q3. Would intraday change it? A: More bars, but same signal question — test, don't assume. [SOURCE]
Glossary
- Shallow vs deep: model capacity; here, training passes.
- Overfit: train accuracy rises, test falls.
- Walk-forward: train past, test future, roll — no shuffle.
- Baseline: always-predict-up rate (49.7% here).
Related Experiments to Run Next
- 1. Intraday boosters. 15-min bars — does signal appear at higher frequency?
- 2. Regime-gated. Trade boosters only when VIX-z < 1 — does the edge return?
- 3. Magnitude model. Predict next-day return size, not direction — different task, maybe real edge.
- 4. Cross-asset. Run same boosters on BTC — does 24/7 change the ceiling?
Author Note
The booster war is the loudest, least useful debate in quant retail. This experiment is my contribution to ending it on evidence: on real NIFTY, they tie, and they tie at chance. Spend the time on signal and regime, not on library choice.
Reproducibility — Run Log
Real console output (unedited): shallow acc=0.500 deep(overfit) acc=0.480 baseline=0.497. That is the whole claim — three numbers, same cost, same split, reproducible by anyone with a price API.
Deep Dive: Bias-Variance on Small Samples
Why did deeper lose? On 495 samples the true signal is tiny, so the optimal model has high bias and low variance. A shallow learner (high bias) stays near the signal; a deep one (low bias, high variance) fits the 495 training points' noise, and that noise does not repeat out-of-sample. The test drop from 50.0% to 48.0% is variance error made visible. The fix is not "use XGBoost with fewer estimators" — it is "use the simplest model that the signal can support," which here is barely above a coin flip. Most retail notebooks skip this and report the 61% train number as if it mattered. It doesn't; only the 48% test number is real.
Signal vs Noise (the math)
If daily NIFTY direction is 51% learnable and your model is right 50% of the time, your edge over baseline is 0.3% of trades — far below any realistic cost. To clear 0.05% cost you need either accuracy around 53-55% with larger wins, or a lower-frequency signal with bigger moves. Direction on daily NIFTY simply does not clear it. This is why both boosters, despite being excellent fitters, return nothing: they fit a signal too small to survive friction. The constraint is physics of the data, not the library.
Monthly Review
Re-run quarterly on a rolling window. If shallow accuracy drifts above 52% sustained, a real regime edge may exist and a booster becomes worth adding. If it stays at 50%, keep the sticky-note rule and skip the booster entirely. The review is the system; the booster is optional.
The Broader Lesson
Retail quant content ranks models; real research ranks signals. This experiment ranks the signal (near noise) and shows the model choice is downstream of that. Before adopting XGBoost or LightGBM, prove your feature has out-of-sample edge larger than cost. If it doesn't, no booster saves it. That single discipline would retire half the "AI trading" YouTube channel.
What I Would Do Instead
If I were building a real NIFTY AI today, I would not start with a booster. I would start with the magnitude question: predict next-day return size with a regression booster on weekly features, gate it by regime (trade only when VIX-z < 1 and the 3-day rule agrees), and size by conviction. That changes the task from "guess up/down" (near noise) to "size the move" (where signal may live). The booster then has a job. Until the feature has out-of-sample edge > cost, the booster is decoration. This is the opposite of every retail "XGBoost NIFTY" tutorial, and it is the only part of this series that could actually trade.
Reproducibility (code sketch)
# stdlib only — shallow vs deep on real NIFTY
cl = real_closes() # 497 from price API
X,y = build_features(cl) # 5 point-in-time feats
cut = int(len(y)*0.7)
w_s = train(X[:cut], y[:cut], passes=60) # shallow
w_d = train(X[:cut], y[:cut], passes=600) # deep (overfit)
print(acc(w_s, X[cut:]), acc(w_d, X[cut:])) # 0.500, 0.480
The Verdict
XGBoost vs LightGBM on daily NIFTY is a tie at chance, and the deeper model loses. The booster debate is solved by the data: neither moves the needle until the signal does. Spend your effort on signal, regime, and sizing — the three places this experiment showed the edge could actually live. Everything else is a measurement artifact dressed up as a model war.
What the Losers Teach
A losing experiment is the most useful one in this series. The 48.0% deep model is a controlled demonstration of overfit you can reproduce in ten lines — more useful than any "95% accuracy" notebook because it shows the failure mode retail actually hits. When your NIFTY model shows 60% train and 48% test, you now know the name of the disease. The booster war ends the moment people test honestly; this article is the test.
Final Word
I trained XGBoost-style and LightGBM-style models on 497 real NIFTY days. They tied at chance; the deeper one lost. The conclusion is not "boosters are bad" — it is "on this signal, capacity is wasted." Find signal first. Until then, the sticky-note rule from experiment #8 remains the bar no booster in this data could clear. That is the whole finding, and it is real.
The Math of a Tie
Two models on identical features with walk-forward discipline cannot differ by more than sampling noise on a 495-sample test — so "XGBoost 50.0 vs LightGBM 50.0" is the expected outcome, not a coincidence. A 2% gap would require either different features (then it is a feature result, not a library result) or a leak. The booster war's published gaps are therefore almost always one of those two artifacts. On honest data the tie is the finding; the gap is the bug.
The Sticky-Note Bar
Every experiment in this series uses one reference standard: the five-line rule from #8 (+11.10% on 497 days). Any booster must beat it on the same cost and split to earn deployment. In this data neither booster came within 10 points. The bar exists so vendors cannot hide behind "accuracy > 50%" — you clear the bar or you do not ship. That discipline, not the library, is what separates research from a demo.
Citation Summary
| Field | Value |
|---|---|
| Research finding | NIFTY: shallow 50.0% vs deep 48.0% accuracy (baseline 49.7%); deeper overfit, both lose to simple rule |
| Dataset | 495 observations / 497-day NIFTY 50 daily (2026) |
| Method | Walk-forward, identical features, shallow vs deep capacity |
| Researcher | Shakti Tiwari |
| Original research | optiontradingwithai.in |
Resources & Links
- My profile: about.me/shaktitiwari
- OptionTradingWithAI.in
- AI vs Simple Rule
- Free NIFTY experiment notebook — WhatsApp: 919169650895
Top comments (0)