DEV Community: Aurora Systems

I Built an Honest Backtester in Python — the Architecture, the Code, and the Real (Losing) Numbers

Aurora Systems — Mon, 13 Jul 2026 15:08:39 +0000

In the last article I walked through the four ways a backtest lies to you — lookahead, overfitting, ignored costs, survivorship. The natural follow-up is the constructive question: what does a backtester look like when it's built so those lies are hard to tell?

This is the architecture I use. The core is one small file of plain Python, but every line placement in it is a design decision, and the design goal is the opposite of what most people build:

A backtester is not a machine for making strategies look good. It's a machine for killing bad ideas cheaply.

Every choice below follows from that sentence.

The contract

Four rules the architecture has to enforce — not suggest, enforce:

The strategy can only see the past. At every decision point it gets exactly the data it could have had at that moment. Never the whole series.
Costs exist from run one. Fees aren't a refinement you add later. They change verdicts.
Every result reports absolute return, max drawdown, and buy & hold — together. One number is marketing. Three numbers are a measurement.
The optimizer never grades its own homework. Whatever data you tune on, you judge on different data.

Rule 1 is structural, not a code-review checkbox

The most common way a backtest lies is lookahead — the strategy acting on data it couldn't have had yet. The last article covered how it sneaks in; this is the architectural answer:

signal = strategy(closes)           # WRONG: the strategy sees the whole series
signal = strategy(closes[: i + 1])  # RIGHT: only the past, one bar at a time

You can hunt the first version in code review, forever, on every change. Or you can make it impossible: the backtest loop hands the strategy a growing prefix of history, so at decision time the future is physically not in the list. The strategy can't cheat because there is nothing to cheat with. That's the entire trick of a walk-forward backtester, and it's the most important line in the file.

The code

Self-contained — standard library only, data fetch at the end:

"""A small, honest, walk-forward backtester.

The cardinal rule: at every decision point, the strategy may only see data
it could actually have had at that moment. We enforce that structurally by
feeding the strategy a growing *prefix* of closes - never the whole series.
"""

from dataclasses import dataclass, field


# --- Strategy: the classic SMA crossover, deliberately unremarkable ---

def sma(values: list[float], n: int) -> float:
    return sum(values[-n:]) / n


def sma_crossover(closes: list[float], fast: int = 9, slow: int = 21) -> int:
    """Return +1 (buy), -1 (sell) or 0 (hold).

    Receives ONLY the prefix of history the backtester hands it. It cannot
    look ahead, because the future literally isn't in the list.
    """
    if len(closes) <= slow:
        return 0
    fast_now, slow_now = sma(closes, fast), sma(closes, slow)
    fast_prev, slow_prev = sma(closes[:-1], fast), sma(closes[:-1], slow)
    if fast_prev <= slow_prev and fast_now > slow_now:
        return +1
    if fast_prev >= slow_prev and fast_now < slow_now:
        return -1
    return 0


# --- The backtester ---

@dataclass
class BacktestResult:
    initial_cash: float
    final_equity: float
    n_trades: int
    equity_curve: list[float] = field(default_factory=list)

    @property
    def total_return_pct(self) -> float:
        return (self.final_equity / self.initial_cash - 1.0) * 100.0

    @property
    def max_drawdown_pct(self) -> float:
        """Worst peak-to-trough drop in equity over the whole run."""
        peak, mdd = float("-inf"), 0.0
        for equity in self.equity_curve:
            peak = max(peak, equity)
            if peak > 0.0:
                mdd = max(mdd, 1.0 - equity / peak)
        return mdd * 100.0


def run_backtest(
    closes: list[float],
    strategy,
    initial_cash: float = 10_000.0,
    fee_rate: float = 0.001,
    warmup: int = 50,
) -> BacktestResult:
    cash, units, n_trades = initial_cash, 0.0, 0
    equity_curve: list[float] = []

    for i, price in enumerate(closes):
        # Decide using ONLY data up to and including bar i.
        if i >= warmup:
            signal = strategy(closes[: i + 1])
            if signal == +1 and units == 0.0:
                fee = cash * fee_rate
                units = (cash - fee) / price
                cash, n_trades = 0.0, n_trades + 1
            elif signal == -1 and units > 0.0:
                proceeds = units * price
                cash = proceeds - proceeds * fee_rate
                units, n_trades = 0.0, n_trades + 1
        equity_curve.append(cash + units * price)

    return BacktestResult(initial_cash, equity_curve[-1], n_trades, equity_curve)

Two knobs deserve a sentence. fee_rate charges every fill from the very first run — that's Rule 2, not an upgrade. And warmup holds the strategy back for the first 50 bars: belt-and-braces, so results never depend on a strategy remembering to self-guard its own lookbacks. Set it at least as large as the longest lookback you ever intend to grid-search. (Yes, closes[: i + 1] copies the list every bar, which makes the loop O(n²). At 1,000 candles that's nothing, and I'll trade speed for structural safety all day; at millions of bars you'd pass an index into a read-only view instead.)

Feed it real candles and run it twice — once fee-free, once with a realistic 0.1% per trade:

import ccxt  # pip install ccxt

WARMUP = 50

candles = ccxt.binance().fetch_ohlcv("BTC/USDT", timeframe="1h", limit=1000)
closes = [float(c[4]) for c in candles]

for fee, label in [(0.0, "no fees"), (0.001, "0.10% per trade")]:
    r = run_backtest(closes, sma_crossover, fee_rate=fee, warmup=WARMUP)
    print(f"[{label:<15}]  trades: {r.n_trades:3d}   "
          f"return: {r.total_return_pct:+7.2f}%   "
          f"max DD: {r.max_drawdown_pct:.2f}%   final: ${r.final_equity:,.2f}")

# The benchmark starts where the strategy is first allowed to trade -
# comparing against bars it never saw would be its own little lie.
hold = (closes[-1] / closes[WARMUP] - 1.0) * 100.0
print(f"\nbuy & hold over the same window: {hold:+.2f}%")

What it said about the "hello world" strategy

Here's a run from my logs — SMA(9/21), 1,000 hourly BTC/USDT candles. (Your numbers will differ; the market moved on. And this particular log predates the drawdown column above — that number joins the story in the risk section.)

[no fees        ]  trades:  55   return:   -6.26%   final: $9,374.45
[0.10% per trade]  trades:  55   return:  -11.27%   final: $8,872.53

buy & hold over the same window:      -21.37%

Three verdicts hiding in four lines:

It loses before a single fee. −6.26% at zero cost. The crossover every tutorial starts with has no edge — and the backtest just delivered that news for free, instead of my exchange account delivering it for a fee.

Fees are five percentage points, not a rounding error. The only difference between −6.26% and −11.27% is 0.1% per trade: 55 trades ≈ 27 round trips × ~0.2% ≈ 5 points, gone. A strategy that trades a lot has to clear that bar before it earns a cent. Most backtest results you see online quietly skip this line item.

"Beat buy & hold" is not "made money." −11.27% beats −21.37% and both lose. And for the record, the fee run's worst peak-to-trough was 13.70% — Rule 3's third number, and the main character two sections down. Rule 3 — absolute return, drawdown, and benchmark printed together — exists precisely because relative-only reporting is halfway to lying.

The optimizer never grades its own homework

Rule 4 is where most self-built backtesters quietly die. Given enough parameter combinations against one slice of history, the winner is a property of your persistence, not of the market.

So the architecture separates the two roles with a hard line: the grid search runs on the first half of the data, and the winning parameters are then judged on the second half, which the search never saw. Given run_backtest, the whole optimizer is a dozen lines:

half = len(closes) // 2
first, second = closes[:half], closes[half:]

def score(data, fast, slow):
    return run_backtest(
        data, lambda c: sma_crossover(c, fast=fast, slow=slow), warmup=WARMUP
    ).total_return_pct

grid = [(f, s) for f in (5, 9, 12, 21) for s in (21, 50) if f < s]
fast, slow = max(grid, key=lambda p: score(first, *p))

print(f"best on in-sample (first half):  SMA({fast}/{slow})  ->  {score(first, fast, slow):+.2f}%")
print(f"same params out-of-sample:                     {score(second, fast, slow):+.2f}%")

From my logs:

best on in-sample (first half):  SMA(9/50)  ->  +2.38%
same params out-of-sample:                     -1.48%

Nothing about the market changed between those two numbers. The only difference is whether the search had been allowed to see the answers. That green-to-red flip is what overfitting looks like from the inside — and this two-slice split is the cheapest instrument that detects it. (The scaled-up version of the experiment — five coins, three time slices, one survivor — is in the last article.)

Architecturally the rule is enforced the boring way: the optimizer and the final judgment literally never touch the same slice. Not "try to remember not to peek" — can't peek.

The risk layer overrides the strategy

A live bot has a layer whose job is to keep you solvent when the strategy is wrong: position sizing, stop-losses, a max-drawdown kill-switch. If your backtester doesn't model that layer, you're testing a different bot than the one you'll run.

The whole layer is small enough to show — three rules, one class:

@dataclass
class RiskConfig:
    risk_per_trade: float = 0.02    # fraction of equity risked per trade
    stop_loss_pct: float = 0.05     # exit a long 5% below entry
    max_drawdown_pct: float = 0.20  # halt trading after a 20% equity drawdown


class RiskManager:
    def __init__(self, config: RiskConfig) -> None:
        self.config = config
        self.peak_equity = 0.0
        self.halted = False        # the kill-switch: once True, no new trades - ever

    def update_equity(self, equity: float) -> None:
        self.peak_equity = max(self.peak_equity, equity)
        # Compare money, not derived ratios - see the war story below.
        floor = self.peak_equity * (1.0 - self.config.max_drawdown_pct)
        if self.peak_equity > 0.0 and equity <= floor:
            self.halted = True

    def position_size(self, equity: float, price: float) -> float:
        # Risk R of equity with a stop S away -> position worth (R * equity) / S.
        # The loss on any single trade is bounded and known BEFORE you enter.
        position_value = equity * self.config.risk_per_trade / self.config.stop_loss_pct
        return min(position_value, equity) / price

    def stop_price(self, entry_price: float) -> float:
        return entry_price * (1.0 - self.config.stop_loss_pct)

In the bar loop it sits above the strategy — the equity update feeds the kill-switch, and hard exits get evaluated first, without asking the strategy's opinion:

# Inside the bar loop:
equity = cash + units * price
risk.update_equity(equity)                     # may trip the kill-switch

stop_hit = units > 0 and price <= risk.stop_price(entry_price)
killed   = units > 0 and risk.halted
if stop_hit or killed:
    ...  # force-exit the position (and remember entry_price on every buy)
else:
    signal = strategy(closes[: i + 1])
    if signal == +1 and units == 0.0 and not risk.halted:
        units = risk.position_size(equity, price)  # sized from the stop, not the gut
        ...

No repo to clone, deliberately: the core, the demo, and the optimizer blocks run exactly as pasted; wiring the risk layer into run_backtest is the ten-line exercise the snippet sketches, and the whole honest machine still lands under 150 lines.

Same losing strategy, same window, two configurations from my logs:

All-in on every signal: −11.3% return, 13.7% max drawdown
2% risk per trade, sized from the stop: −5.4% return, 6.5% max drawdown

The risk layer didn't turn a loser into a winner — nothing honest does that. It made the loser survivable, which is its actual job.

One war story about why this layer needs unit tests more than your strategy does. My kill-switch was set to trip at a 20% drawdown. I tested it by hand and fed it a drop of exactly 20% — equity from $11,000 down to $8,800. It didn't trip:

>>> 1 - 8800 / 11000
0.19999999999999996        # < 0.2 - floating point says you're fine

It finally tripped at 20.9% — not because floating point costs 0.9 points, but because that's simply where the next bar landed after the threshold had already been missed. The fix is the one the class above already carries: compare money, not derived ratios — equity <= peak * (1 - threshold) trips at exactly $8,800 — or compare with an explicit tolerance. Either way: a boring, "can't possibly be wrong" rule, wrong on exactly the day it exists for. Unit-test the guard rails.

What this architecture can't do

Honesty cuts both ways, so here are the limits, written down:

Fills are a fairy tale. This backtester decides at a bar's close and gets filled at that same close — no slippage, no spread, no order-book depth. The same-bar fill is also a timing optimism that Rule 1 does not cover: the stricter convention is to act on the next bar's open (a one-line change), and to treat close-fills as what they are — a deliberate simplification, not part of the lookahead guarantee. On a liquid market at small size all of that is tolerable. On a thin market it's the whole story: the gap between quoted price and realized fill is how a strategy once quoted +740% in my backtest and realized about $2.81 live.
It can't fix your data. Feed it a universe containing only coins that exist today and it will happily backtest survivors and report survivor results. The architecture enforces honest process; honest data is your job.

Writing the limits down isn't modesty — every limitation you don't list gets billed to you later, in money.

The checklist version

Feed the strategy prefixes. Never the whole series.
Fees from the very first run.
Absolute return + max drawdown + buy & hold, printed together, every time.
Tune on one slice, judge on another. No peeking, no "one more tweak" after seeing the holdout.
Risk rules override the strategy — and get their own unit tests.
Distrust beautiful curves. Hunt for the bug first; the honest ones are ugly.

I write about the honest engineering behind trading bots — including the losing numbers. If this was useful:

This backtester is one module of a complete build. My course walks through the whole bot the same honest way — exchange client, strategy, backtesting, risk & sizing, live execution, monitoring: *Build Your Own Crypto Trading Bot — An Honest Engineering Course (Python)*.
Free companion guide (PDF): The 4 Ways Your Backtest Is Lying To You
The book on why backtests deceive: Backtests Lie on Amazon
Day-to-day notes on X: @AuroraSystemsX

Educational only. Not financial advice. Trading carries real risk, including total loss.

Your Trading Bot's Backtest Is Lying to You - Here Are the 4 Ways

Aurora Systems — Sat, 11 Jul 2026 09:40:11 +0000

A strategy I once tested quoted +740% in the backtest. When it actually traded, it made about $2.81.

That gap - between what a backtest promises and what a live market pays - is where most trading bots quietly die. I've spent three years building bots in Python, and the most valuable thing I've learned isn't a strategy. It's that a backtest is the easiest thing in the world to rig in your own favor, usually by accident.

Here are the four ways it lies, each with a real number from my own testing, and how to catch each one before it costs you money.

The baseline: an honest test of the "hello world" strategy

First, a calibration. The classic moving-average crossover - SMA(9/21), the strategy every tutorial starts with - backtested honestly on 1,000 hours of real BTC/USDT, with decisions using only past, closed bars:

No fees: -6.3%
With a realistic 0.1% fee per trade: -11.3%
Buy & hold over the same window: -21.4%

Notice two things. It loses before a single fee. And it "beat buy & hold" while still losing money - those are not the same sentence, and any backtest report that only shows relative performance is already halfway to lying.

Lie #1: Lookahead - your strategy can see the future

The most common bug, and it's usually one careless line:

signal = strategy(prices)         # WRONG - sees the whole series, including the future
signal = strategy(prices[:i+1])   # RIGHT - only the past, one bar at a time

Hand the strategy the entire price series instead of only the data up to the current bar, and every historical "decision" secretly knows how the story ends. The equity curve goes vertical. The returns are fiction. Subtler versions hide in feature scaling (normalizing by a series max that includes future data) and in fill assumptions (deciding at the close of a bar and getting filled at that same close).

How to catch it: run the backtest walk-forward - feed the strategy a growing window, never the whole series. And treat any curve that looks too smooth as a bug until proven otherwise. Real edges are messy.

Lie #2: Overfitting - you tuned until it looked perfect

A real experiment from my logs. I grid-searched MA parameters on the first half of my data. The best combination returned +2.4%. Then I ran those exact same parameters on the second half - data the search had never seen: -1.5%.

The profit was a property of the search, not the market. Run enough parameter combinations against one slice of history and something will always look brilliant - you're not discovering an edge, you're memorizing noise. That green-in-sample, red-out-of-sample flip is the fingerprint of overfitting.

I later re-ran this lesson at a bigger scale: parameters tuned on one half-year, validated on the next, then scored on a final unseen period across five coins. One of five survived. The other four evaporated exactly the way the +2.4% did.

How to catch it: judge a strategy only on data it was never tuned on. Optimize on one slice, score on another. No exceptions, no peeking, no "just one more tweak" after seeing the holdout.

Lie #3: Ignored costs - you assume you trade for free at the price you see

Fees alone nearly doubled the baseline loss above (-6.3% -> -11.3%). And fees are the polite cost. Slippage is the rude one. Walking a market order through the real BTC order book:

$1,000: ~0.000% price impact
$1,000,000: ~0.035%
$10,000,000: ~0.132% - and it exhausts the entire visible book

That's on the deepest market in crypto. On a thin pair, the visible book collapses after the first few coins - which is exactly how a backtest can quote +740% on a freshly launched pool while the tradeable reality is about $2.81. The price you see is the price for the first few coins. Everything after that is slippage.

How to catch it: model fees and a slippage assumption from the very first backtest. The thinner the market and the bigger your size, the more the quote lies about the fill.

Lie #4: Survivorship - you only tested the winners

Backtest "all coins" using today's coin list and you've silently deleted every project that went to zero and got delisted. Your universe is pre-filtered to survivors, so of course the results look great. The same trap hits stock strategies tested on the current index members - the companies that failed out of the index are exactly the ones your strategy would have bought on the way down.

How to catch it: use data that includes the dead - delisted assets, failed projects, all of it. And be suspicious of any result that claims "tested on everything."

The honest checklist

Run this on every idea:

Express it as "prices in -> signal out." If you can't, it isn't testable yet.
Backtest with realistic fees and slippage from day one.
Score it only on data it was never tuned on.
Read absolute return and max drawdown - "beat the market" while losing money is not a win.
Distrust anything that looks great; hunt for the bug first.
If it survives all that, paper-trade it on a testnet. For a long time.
Size positions with a real risk layer and keep a kill-switch armed.
Never run money you can't afford to lose to zero.

The goal was never to find a magic strategy. It's to build a backtester honest enough to kill your bad ideas cheaply - so the rare good one is the only thing left standing.

I write about the honest engineering behind trading bots - including the losing numbers. If this was useful:

The full version is a book: Backtests Lie - why most bots lose, and what building one really teaches you.
Free companion guide (PDF): The 4 Ways Your Backtest Is Lying To You
Day-to-day notes on X: @AuroraSystemsX

Educational only. Not financial advice. Trading carries real risk, including total loss.