Backtesting the Pullback Filter: When a Safety Check Kills Profit

The Setup: One Strategy, Two Filters

The base strategy is a long-only momentum entry on 15-minute candles. Signal logic, position sizing, and exit rules are fixed. The only thing that changes between variants is the pullback confirmation that must be true before an entry fires.

The strict filter requires all three of the following on the entry bar:

1. At least one down candle among the last three 15-minute candles.
2. Current price below the high of the last five candles.
3. The latest candle is not a large bullish candle (body above a threshold, e.g. 0.8 × ATR).

The relaxed filter drops the three-candle window and the big-bullish veto, keeping only: at least one down candle within the last five candles. It still demands a pullback, just a looser one.

Both variants run on the same 90-day window, 24 symbols, same fee model, same slippage assumption. That is the whole point of the experiment: a single boolean changes, everything else is held constant.

Why the Comparison Is Easy to Get Wrong

The tempting move is to compare the two versions by their win rates or average PnL per trade. That comparison is contaminated. The strict filter does not just change how trades are taken — it changes which trades exist. The strict set is a subset of the relaxed set, so you are comparing two different populations of trades, not the same trades under two treatments.

To say anything honest about the filter, you need to look at the trades the strict filter rejects. Those rejected entries are where the answer lives. If the rejected trades have positive expectancy on their own, the filter is destroying edge. If they are negative, the filter is earning its keep.

So the experiment has three outputs, not two:

• Trades taken by both variants (the intersection).
• Trades taken only by the relaxed variant (rejected by strict).
• Expectancy of each group.

The base strategy under the relaxed filter is the control population. The strict filter is a treatment applied to the signal stream. Treat it like an A/B test where the "B" arm is a subset of "A", and you want to know whether the excluded slice is better or worse than average.

Counting the Incremental Trades

The cleanest way to run this is to evaluate the signal stream once, tag every candidate entry with the filter booleans, and then slice. Here is a minimal harness that does exactly that.

from dataclasses import dataclass
from typing import Callable

@dataclass
class Trade:
    symbol: str
    ts: int
    entry: float
    exit: float
    pnl: float          # net of fees and slippage, in quote units

def strict_pullback(candles, i: int, atr: float) -> bool:
    # candles: list of (ts, o, h, l, c) ascending, i = entry bar index
    last3 = candles[i-2:i+1]
    has_down = any(c[4] < c[1] for c in last3)
    five_high = max(c[2] for c in candles[i-4:i+1])
    body = candles[i][4] - candles[i][1]
    not_big_bull = body < 0.8 * atr
    return has_down and candles[i][4] < five_high and not_big_bull

def relaxed_pullback(candles, i: int, atr: float) -> bool:
    last5 = candles[i-4:i+1]
    return any(c[4] < c[1] for c in last5)

The backtest engine stays untouched. You generate candidate entries from the momentum signal, then record both filter verdicts alongside the realized PnL. Note that the entry price and exit price are identical for a given candidate regardless of which filter accepted it — the filter does not move the entry bar, it only decides whether the bar is tradeable at all. That assumption matters: if your strict filter also shifts entry timing (e.g. waiting for confirmation on the next bar), you have changed two variables and the comparison is dead.

def run_backtest(candles_by_symbol, signal_fn, exit_fn):
    rows = []
    for sym, candles in candles_by_symbol.items():
        atr_series = compute_atr(candles, 14)
        for i in range(20, len(candles) - 1):
            if not signal_fn(candles, i):
                continue
            atr = atr_series[i]
            entry = candles[i][4]
            exit_price = exit_fn(candles, i)
            pnl = (exit_price - entry) - fee_and_slip(entry, exit_price)
            rows.append({
                "symbol": sym,
                "ts": candles[i][0],
                "pnl": pnl,
                "strict": strict_pullback(candles, i, atr),
                "relaxed": relaxed_pullback(candles, i, atr),
            })
    return rows

Now the analysis is just a groupby. No second backtest run, no risk of drift between the two code paths.

def slice_expectancy(rows):
    both = [r["pnl"] for r in rows if r["strict"] and r["relaxed"]]
    strict_only = [r["pnl"] for r in rows if r["strict"] and not r["relaxed"]]
    relaxed_only = [r["pnl"] for r in rows if r["relaxed"] and not r["strict"]]
    # note: strict implies relaxed by construction only if the
    # relaxed rule is a strict superset — verify this before trusting it
    return {
        "both": (len(both), sum(both) / len(both) if both else 0),
        "strict_rejected": (len(relaxed_only),
                            sum(relaxed_only) / len(relaxed_only) if relaxed_only else 0),
        "strict_only": (len(strict_only),
                        sum(strict_only) / len(strict_only) if strict_only else 0),
    }

One check before you trust any of this: confirm the strict rule is actually a subset of the relaxed rule. If your relaxed rule is "any down candle in the last five" and your strict rule also requires "price below the five-candle high," then strict does not imply relaxed — a bar can satisfy strict's down-candle condition and high condition while failing nothing in relaxed, but the reverse can also happen if the down candle is outside the five-bar window in strict's three-bar check. Run the set-difference both ways and print the counts. If strict_only is non-empty, your two rules are not nested, and you should compare them as two separate populations rather than a control and a treatment.

The Numbers

On the 90-day, 24-symbol run:

Variant	Trades	Expectancy (U/trade)
Strict pullback	~1,900	-0.013
Relaxed pullback	~2,600	+0.011

The strict filter rejects roughly 700 candidate entries. Those rejected trades carry positive expectancy — enough to flip the whole strategy from negative to positive. The filter did not remove noise; it removed the profitable tail.

The win rate on the strict set was actually slightly higher. That is the trap. A filter that raises win rate while lowering expectancy is selecting for small, frequent wins and occasional large losses — exactly what a late, tired move produces.

Why "Safer" Is Worse Here

The strict filter's three conditions all bias toward the same failure mode: entering after the move has already extended and then rolled over.

• Requiring a down candle in the last three bars means you enter on the first sign of weakness. In a momentum regime, that is often the start of a real reversal, not a healthy pullback.
• Requiring price below the five-candle high means you are buying into a local dip — but only after the high was already set. The move is mature.
• Vetoing a big bullish candle removes exactly the bars where momentum is re-accelerating. Those are the entries with the best follow-through.

The relaxed filter keeps the pullback requirement but gives the move room to breathe. It accepts entries where the pullback happened four or five bars ago and price has since resumed. Those entries are earlier in the next leg, not later in the current one.

Put differently: the strict filter is a proxy for "the setup looks clean right now." But clean-looking setups are, by construction, the ones where the crowd has already noticed. The relaxed filter accepts messier setups that the strategy's exit logic can still harvest. The filter was supposed to reduce drawdown. It reduced trade count and took the edge with it.

Practical Rules for Filter A/B Tests

A few habits that keep this kind of test honest:

Keep the exit fixed. If you tune the exit for each filter variant, you are no longer measuring the filter. Any difference in expectancy is confounded by the exit change.

Always report the rejected slice. A filter's value is the expectancy of what it excludes. If you do not compute that number, you do not know what the filter does.

Check nesting before slicing. Confirm which candidate population each rule accepts, and whether one is a subset of the other. Non-nested rules need separate baselines.

Watch win rate and expectancy diverge. When a filter raises win rate but lowers expectancy, you have found a filter that trades frequency for size. That trade is only good if the size reduction is larger than the frequency gain — usually it is not.

Run the test on out-of-sample data. A 90-day window with 24 symbols is enough to see a sign flip, not enough to trust the magnitude. Re-run on a held-out period and check whether the rejected-slice expectancy keeps the same sign.

Closing

Filters that feel safe are often filters that select for late entries. Measure what they reject, not just what they keep.

More notes like this ship every week on this site.

---

Daily Picks

The following pairs are selected from the multi-timeframe trend scanner (Gate.io futures) and are for technical-analysis study only — not investment advice.
Data updated: 2026-10-27 08:46:47

Short

Pair	Signal	Price	Take Profit	Stop Loss	R/R
Q	热点跟涨	$0.2008	$0.1848	$0.2079	1:2.3

1 picks selected. Scanner runs every 15 minutes.