Propfirmkey

Posted on Mar 8

Backtesting Trading Strategies: Common Mistakes to Avoid

#trading #python #datascience #beginners

Backtesting is how traders validate strategies before risking real money. But flawed backtesting leads to false confidence and real losses.

Mistake 1: Survivorship Bias

Testing your stock strategy only on companies that exist today ignores all the companies that went bankrupt. Your strategy might have picked those losers too.

Fix: Use datasets that include delisted securities. Point-in-time data is essential.

Mistake 2: Look-Ahead Bias

Using information that wouldn't have been available at the time of the trade.

# WRONG - using future data
df['signal'] = df['close'].rolling(20).mean()  # This uses the current bar

# RIGHT - shift the signal
df['signal'] = df['close'].rolling(20).mean().shift(1)  # Uses only past data

Mistake 3: Overfitting

Adding parameters until your strategy perfectly fits historical data.

# Overfitted: 8 parameters, perfect backtest
def strategy_overfit(data, ma1=7, ma2=23, rsi_period=13,
                     rsi_upper=72, rsi_lower=28, atr_mult=2.3,
                     volume_threshold=1.4, time_filter=True):
    pass

# Better: 2-3 parameters, robust performance
def strategy_simple(data, ma_period=20, atr_mult=2.0):
    pass

Rule of thumb: If your strategy has more than 3-4 parameters, you're probably overfitting.

Mistake 4: Ignoring Transaction Costs

# Without costs - looks profitable
gross_pnl = sum(trade_pnls)  # +$5,000

# With realistic costs - might be negative
spread_cost = num_trades * avg_spread  # $3,200
commission = num_trades * commission_per_trade  # $1,500
slippage = num_trades * estimated_slippage  # $800
net_pnl = gross_pnl - spread_cost - commission - slippage  # -$500

Mistake 5: Not Testing Out-of-Sample

# Split your data
train_data = df[:'2024-06-30']  # Optimize on this
test_data = df['2024-07-01':]    # Validate on this

# Train
params = optimize_strategy(train_data)

# Test - if it works here too, the strategy is likely robust
test_results = run_strategy(test_data, params)

Mistake 6: Ignoring Regime Changes

A strategy that works in trending markets fails in ranging markets. Test across multiple market conditions:

Bull trends
Bear trends
High volatility
Low volatility
Ranging/choppy

A Proper Backtesting Framework

class Backtest:
    def __init__(self, data, strategy, initial_capital=100000):
        self.data = data
        self.strategy = strategy
        self.capital = initial_capital
        self.trades = []

    def run(self):
        for i in range(len(self.data)):
            bar = self.data.iloc[:i+1]  # Only past data
            signal = self.strategy.generate_signal(bar)

            if signal:
                self.execute(signal, bar.iloc[-1])

        return self.calculate_metrics()

    def execute(self, signal, bar):
        # Include realistic fills
        slippage = bar['atr'] * 0.1
        fill_price = bar['close'] + slippage * (1 if signal == 'buy' else -1)
        commission = fill_price * 0.001  # 0.1%

        self.trades.append({
            'entry': fill_price,
            'commission': commission,
            'direction': signal
        })

The Golden Rule

If your backtest shows 90%+ win rate or 500% annual returns, something is wrong. Realistic strategies typically show:

Win rate: 40-65%
Annual return: 15-50%
Max drawdown: 10-25%
Profit factor: 1.3-2.5

The best strategy is one that survives real market conditions. Whether you trade independently or through a prop firm — and propfirmkey.com has detailed comparisons to help you choose — a properly backtested strategy is non-negotiable.

What backtesting framework do you use?

DEV Community