Every trading bot post ends the same way: a green equity curve and a suspicious silence about whether it made real money. This post is the other one. I spent months building an algorithmic trading system, tested it as honestly as I know how, and here is the headline result:
My best strategy returned +4.33% over three years of out-of-sample testing. Buying Bitcoin once and holding it returned +127.77%.
I'm publishing that number instead of hiding it, because getting to a number I couldn't argue with turned out to be the entire project.
What I built
TradingAI is about 4,000 lines of Python: a strategy engine, a backtester, a walk-forward validation harness, a live paper-trading bot with Telegram alerts, a real-time dashboard, and 130+ tests running in CI. The bot trades a simple, transparent rule (a Donchian channel breakout: buy 20-day highs, exit on a trailing stop) on real exchange data via ccxt.
The strategy itself is a few lines of logic. Everything around it exists because the first version of this project lied to me, and I rebuilt it so that lying was impossible.
How backtests lie: a field guide
My original backtester said the strategy was profitable. It was wrong in four separate ways, and all four are endemic in hobby trading bots.
- Look-ahead bias (trading on tomorrow's newspaper) The old code computed a signal from a candle, then "bought" at that same candle's close. It acted on information that didn't exist until the moment was already over. The fix is structural, not a patch:
engine.py: bar t executes the decision made at bar t-1's close,
filling at bar t's open. One line of policy, enforced everywhere.
One engine enforces this for every strategy, and a test feeds in a deliberately cheating strategy (it peeks one bar ahead) and asserts it cannot profit. If someone adds a leaky strategy later, CI catches it.
- Fees are the boss fight The old backtester charged zero fees. Kraken's taker fee is 0.26% per side, which sounds tiny per trade and is catastrophic in aggregate. When I re-ran my system at different speeds with honest costs, fees wrote the whole story:
Timeframe Trades Fees (% of account/yr) Net result
1 day 14 1.3% +4.33%
4 hours 61 11.4% −6.01%
1 minute 52 in 45 days 23.5% in 45 days −25.18%, zero wins
Read that last row again: fifty-two trades, zero winners. The signal was identical at every speed; the fees were not. On 1-minute bars the fee is roughly six times the average move you're trying to catch, so every trade opened underwater. A real signal and a losing strategy are not contradictory. The gap between them is transaction costs.
Grading your own homework (overfitting)
If you tune a strategy's parameters on 2023's data and then test it on 2023's data, you've measured memorization, not skill. The honest version is walk-forward analysis: optimize on a training window, trade the next window the optimizer never saw, roll forward, repeat. Only the unseen windows count. My +4.33% is that number: five years of data, scored only on days the parameters had never met."It's profitable" vs "it's lucky"
Even an honest backtest can be a fluke. So the harness runs a permutation test: shuffle the market's daily returns 200 times (destroying the time structure while keeping the same statistical distribution) and re-run the strategy on every scrambled history. If it scores as well on noise, it has no edge. Mine beat 97.5% of the shuffles (p = 0.025). The edge is real. It is also small, and smaller than the fees.
The experiments that humbled me
Three things I was sure would help, all executed by the same harness:
"Add more indicators." My original bot summed seven indicators into a score. Tested honestly against a one-rule strategy, the 7-indicator version came dead last (−2.4% vs +3.0%). Indicators on one asset are correlated opinions; each new one adds more ways to memorize history than ways to read it.
"Trade faster for more profit." See the table above. Every step down in timeframe was strictly worse, entirely because of fees. The math doesn't negotiate.
"Diversify across coins." The one idea with real evidence behind it: trend funds do this. I ran the same rule on BTC + ETH + SOL, equal sleeves. Result: portfolio Sharpe 0.30 vs 0.38 for BTC alone. Diversification needs uncorrelated assets, and major cryptos turn out to be one macro wave in three sizes. My best idea, killed by my own machine. I kept the feature anyway; a test bench that spares its author's ideas is worthless.
What the bot is good at
One defensible result: during a stretch where Bitcoin drew down 53%, the bot's worst drawdown was 4%. It spends about 80% of its time safely out of the market and only surfs confirmed trends. It's a stay-solvent machine, not a get-rich machine. Whether that trade-off is worth +4% vs +127% is a question about your stomach, not your code.
The engineering I'm proudest of
One engine for backtest and live: the simulator and the live bot execute the same code path, so live behavior can't silently diverge from tested behavior. A test asserts both paths land on the same equity, to the cent.
A data layer that refuses to lie: Kraken's API silently ignores your start date and returns only the last ~720 candles. My old backtester printed "testing 180 days" while testing 2.5. The new fetch layer paginates from venues that support it and fails loudly on truncation.
A live mode that can't run unprotected: the old live path's stop-loss function was literally pass. The new one places an exchange-side stop at entry and closes the position immediately if that order fails.
Replay mode: a live 4h bot makes six decisions a day, which looks identical to a crashed program. cli.py replay runs the real engine over two years of history in a minute while the dashboard animates every trade. Same code, same numbers, just watchable.
What I'd tell past-me:
Build the lie detector before the strategy. You can't evaluate ideas without it, and with it, bad ideas die for free instead of expensively.
Fees are not a detail. They are the main character.
A negative result you can defend beats a positive result you can't.
"Would this survive someone trying to debunk it?" is the only standard that matters, in trading and probably everywhere else.
The whole thing is open source: github.com/nar1-frames/tradingai.
Clone it, run make setup && make doctor, and python cli.py walkforward regenerates every number in this post from live exchange data. If you find a way the harness can still be fooled, open an issue. That's the game.
Nothing here is financial advice. The measured finding is literally that you shouldn't trade like this.
Top comments (0)