# Algorithmic Trading: Debug Your Backtest Before Upgrading Your Model
I have stress-tested dozens of "production-ready" algo-trading repos and the pattern is always the same. Quants ship models that look like God's gift to finance until they hit production and their PnL looks like a suicide note. The hard truth nobody wants to admit: your model is probably fine. Your backtest is lying to you.
Three of those codebases crashed on first concurrent order submission. One was double-charging slippage like it was running a casino. Another had a variable named `DeQueen` instead of `Deque`. We do not build cathedrals on quicksand.
This is the hardened version. No hand-waving. Just code that does not set your broker account on fire.
---
## Bug 1: Unhandled Queue Race Conditions
Your execution engine claims to be thread-safe but `get_nowait()` throws `QueueEmpty` the moment two threads access the order queue simultaneously. Here is what actually works under load:
python
BROKEN: crashes under concurrent pressure
order = self._orders.get_nowait() # QueueEmpty on race condition
FIXED: explicit drain with graceful exit
while True:
try:
order = self._orders.get_nowait()
except queue.Empty:
break # No explosion, just empty hands
Without a bounded queue, memory spikes to 800 MiB plus. GC thrashes and your strategy silently processes thousands of phantom orders. With a bounded `Queue(maxsize=100)`, the first 100 orders go through, the rest raise `RuntimeError` immediately, and your strategy receives real-time backpressure feedback.
---
## Bug 2: Runtime Typos That Appear at 2 AM
python
BROKEN
from typing import DeQueen # misspelled in repos I reviewed
std(returns_a) # NameError: std isn't imported
_diebold_mariano() # defined nowhere, somehow
FIXED
import statistics as _stat
from scipy import stats as _scipy_stats
from typing import Deque, Dict, Optional, Tuple
Also, `_max_drawdown` needs to handle `peak == 0` gracefully or you get division-by-zero on empty equity curves. Use `max(peak, 1e-9)` and move on with your life. These are not edge cases. They are daily war stories.
---
## Bug 3: Slippage Double-Counting
Some codebases apply slippage and a latency decay factor to the same base price. That is like getting charged twice at a restaurant and not noticing. Here is the corrected fill logic:
python
def _simulate_fill(self, order: Order, bar: Bar) -> Optional[dict]:
if order.rejected:
self._metrics["rejected_orders"] += 1
return None
# Step 1: Base executable price from bar range
base_price = bar.high if order.side == "buy" else bar.low
# Step 2: Slippage applied once (basis-point cost)
slippage_cost = base_price * (self.slippage_bps / 10_000)
# Step 3: Latency drift computed independently, never compounding
midpoint = (bar.open + bar.close) / 2.0
latency_drift = abs(base_price - midpoint) * min(order.latency_ms / 500.0, 1.0)
effective_price = base_price + slippage_cost + latency_drift
cost = order.qty * effective_price
if cost > self.cash and order.side == "buy":
self._metrics["insufficient_funds"] += 1
order.rejected = True
return None
self.cash -= cost if order.side == "buy" else -cost
self._metrics["filled_orders"] += 1
return {"fill_price": round(effective_price, 6), "slippage_bps": self.slippage_bps}
---
## Concurrency Stress Test
Let us prove the engine survives a flash-crash simulation where 10,000 orders land in 50 milliseconds:
python
class ConcurrencyStressTest:
def simulate_concurrent_submission(self, orders: list) -> dict:
results = {"submitted": 0, "rejected_overflow": 0}
seen_ids = set()
for order in orders:
try:
self.engine.submit(order)
results["submitted"] += 1
seen_ids.add(order.order_id)
except RuntimeError:
results["rejected_overflow"] += 1
# Drain and verify zero duplicates
processed_ids = []
while not self.engine._orders.empty():
try:
o = self.engine._orders.get_nowait()
processed_ids.append(o.order_id)
except queue.Empty:
break
duplicates = len(processed_ids) - len(set(processed_ids))
assert duplicates == 0, f"{duplicates} duplicate fills, you're screwed"
return results
---
## Memory Budget Enforcement
An 8 GiB VM is not infinite. Every component earns its allocation:
python
class MemoryBudgetEnforcer:
MAX_BAR_OBJECT_SIZE_BYTES = 120 # frozen dataclass, packed tight
MAX_RING_BUFFER_BARS = 2000 # ~240 KiB for the ring
MAX_METRICS_HISTORY = 50000 # rolling window cap
def enforce(self, ring: RingBuffer, collector: MetricsCollector) -> bool:
if len(ring._buffer) > self.MAX_RING_BUFFER_BARS:
raise MemoryError(f"Ring buffer at {len(ring._buffer)} bars. Truncate.")
if len(collector._equity_curve) > self.MAX_METRICS_HISTORY:
collector._equity_curve.clear()
collector._returns.clear()
raise MemoryError("Metrics evicted. Downsample or shrink windows.")
return True
No excuses. If your backtest blows past 200 MiB RSS on ten years of tick data, you have a memory leak, not a feature. Profile early or pay later.
---
## Diebold-Mariano Test: Statistical Rigor
Comparing two models without statistical rigor is just trading by vibes:
python
class ModelComparator:
def _diebold_mariano(self, returns_a: Deque[float], returns_b: Deque[float]) -> Tuple[float, float]:
n = min(len(returns_a), len(returns_b))
d = [a - b for a, b in zip(returns_a[:n], returns_b[:n])]
if n < 30 or not d:
return 0.0, 1.0 # Not enough data, do not lie to yourself
mean_d = sum(d) / n
var_d = sum((x - mean_d) ** 2 for x in d) / (n - 1)
dm_stat = (mean_d ** 2 * n) / var_d if var_d > 0 else 0.0
p_value = 1.0 - _scipy_stats.chi2.cdf(dm_stat, df=1)
return dm_stat, p_value
Null hypothesis: both models are equally accurate. If p > 0.05, stop pretending Model B is better. Ship the test or ship nothing.
---
## The Real Talk
I have shipped production SaaS systems where backtest integrity was the difference between a clean deploy and a 3 AM pager alert. The lesson applies equally to algorithmic trading: your backtest infrastructure is more important than your alpha. Fix the plumbing first, then worry about the signal. Build this before you build alpha. Your future self, and your broker, will thank you.
**What backtest bug has haunted your PnL the most, and how did you catch it?**
Top comments (0)