Over 10+ years of trading we've built and discarded a lot of strategies. The pattern that cost us the most wasn't a bad idea — it was a good-looking backtest that quietly lied. Here are the three lies that fooled us most, and the cheap checks that catch them. They apply no matter what tool you use.
1. Acting one bar late
One rule we were excited about showed roughly +120% over 90 days in a backtest. Then we re-ran it evaluating the signal on the previous closed bar — which is what you actually get once a bot reacts a candle after the condition fires. It collapsed to about +17%. Nothing else changed.
Most of the "edge" was really the backtest peeking at the same candle it traded on.
The check: shift every indicator back one bar and re-run. If the edge mostly survives, it's probably real. If it evaporates, you were trading on information you won't have live. Your live execution has to act on the exact closed bar, or the strategy isn't the strategy you tested.
2. Leverage without liquidation
A lot of backtesters compute PnL at leverage as if the position always survives to your stop. At 10× a 10% adverse wick isn't a drawdown — it's a liquidation. Several leveraged variants that looked great went negative the moment we force-closed positions at the liquidation price instead of pretending it wasn't there.
The check: model the liquidation price for each position and close there, intrabar, using the candle's high/low — before your stop logic ever runs.
3. Fees, fills, and one lucky season
Three smaller lies that stack up:
- Fees: the same rule went from +120% to +64% when we stress-tested fees from 0.05% to 0.15% per side. If a strategy only works at zero fees, it doesn't work.
- Fills: a backtest "take-profit" that assumes a wick touch fills your resting limit order is optimistic — on a fast candle it may not fill at all.
- One season: 90 days of one summer is not a sample. A rule that only works in one regime is a story, not an edge.
The check: stress the fees, be pessimistic about fills, and test across different market conditions.
Why we're telling you this
We got tired of tools that made it easy to fool ourselves, so we built the one we wanted: Tradsist — build a strategy from dropdown conditions (no code), backtest on real exchange candles with fees, with liquidation modelled on leveraged trades and intrabar take-profit/stop handling, then paper trade it on live candles before risking anything. A built-in AI agent can assemble and adjust the rules for you.
It's free in the browser, no signup: https://app.tradsist.com
We'd genuinely rather hear what's wrong with it than what's right — if you put a rule through it and something looks off, tell us.
Trading is risky and nothing here is financial advice: https://tradsist.com/risk
Top comments (4)
The one-bar-late check is the cheapest test in this list, and I'd run it as a curve rather than a single point: shift by 0, 1, 2 and 3 bars and plot the return. A real edge decays smoothly with delay; a peeking backtest falls off a cliff at the first shift, which is what your +120% to +17% looks like. It also tells you how much execution latency the rule can tolerate live.
The other thing the 90-day number can't tell you is how many rules were tried before this one. If +120% was the best of 50 variants on the same candles, the best of 50 noise strategies will beat zero by a wide margin on its own, so the check that goes with the three you list is to log every variant you test and compare the winner against that count, not against zero. The +64% after fees is the more believable figure, but only if the rule was fixed before the fee stress, not picked after it.
Both of these are exactly right, thanks for the thoughtful read.
The decay curve is in, as an opt-in robustness check: it re-runs the strategy with entries filled 0 to 3 bars late and shows how PnL and win rate decay. The read is the one you described — a real edge fades gently, while one that only works at the exact signal fill falls off a cliff and won't survive live execution. It's off by default and toggled per user.
The multiple-testing point is the one that bites hardest, and you framed it well: best-of-N clears a gated validation by chance. Beyond the train/validation split we added a true lock-box — a final slice the search never scores or gates against — reported alongside everything else, plus a caveat surfacing how many configurations were tried. The lock-box is the honest "did this generalise" number, and a config that's strong in-sample but weak there reads as overfit.
One edge case I would add is a candle that touches both the stop and the liquidation level. OHLC data cannot tell you which came first. Run both admissible event orders, or use lower-timeframe data, and report the unresolved trades separately. It is also worth keeping the mark-price series separate when the exchange liquidates on mark price rather than the traded price.
This is a great edge case, thank you. You're right that OHLC alone can't tell you the intra-candle path when a candle touches both levels.
The way we resolve it is by direction. A protective stop that sits closer to entry than the liquidation level is treated as filling first, since price has to pass through it to reach liquidation, so a tight stop genuinely shields the position. A stop that only evaluates on the candle close doesn't get that protection; it can't act before an intrabar wipe, and we let it honestly lose in that case rather than pretend otherwise.
On mark price, good catch, and you're ahead of where we were. We now pull a separate mark-price series and trigger leverage liquidations off it instead of last-traded price, so a last-price wick no longer registers a liquidation that never would have happened against the smoother mark. It falls back to last price when a mark series isn't available, and it flows through the optimizer so its liquidation accounting matches the backtest.