Day 9 — Building a Python backtesting framework to simulate order execution, measure performance, and optimize market-making strategies.
Introduction
In the previous article, we built an adaptive market-making strategy that adjusts its spreads based on volatility, liquidity, and inventory.
But an important question remains:
How do we know whether our strategy actually works?
Imagine your Polymarket bot places:
- YES buy order at $0.48
- YES sell order at $0.52
- Order quantity: 100 tokens
The theoretical gross profit is:
(0.52 - 0.48) * 100 = $4.00
Looks profitable, right?
But what if:
- Your buy order never gets filled?
- Your order is only partially filled?
- The market moves against you immediately after execution?
- Other traders are ahead of you in the order queue?
- Trading costs eliminate your spread?
This is why we need backtesting.
In this article, we'll build the foundation of a Python backtesting framework for a Polymarket market maker bot.
We'll cover:
- Backtesting fundamentals
- Historical market data
- Market-making quote calculations
- Limit-order execution simulation
- Inventory and P&L tracking
- Performance metrics
- Parameter optimization
- Avoiding overfitting
- Preparing for paper trading
1. What Is Backtesting?
Backtesting means testing a trading strategy against historical market data to estimate how it would have performed in the past.
Instead of immediately connecting your bot to a real trading account, you replay historical market conditions and simulate the decisions your bot would have made.
Think of it like a flight simulator.
You can test different situations without risking a real aircraft. Backtesting provides a similar environment for trading strategies.
For example, we can compare three configurations:
| Strategy | Half-spread | Order size |
|---|---|---|
| A | $0.01 | 10 tokens |
| B | $0.02 | 10 tokens |
| C | $0.03 | 10 tokens |
Using the same historical data, we can compare:
- Number of executed orders
- Captured spread
- Inventory exposure
- Maximum drawdown
- Net profitability
Backtesting vs. Paper Trading vs. Live Trading
| Stage | Purpose | Real money |
|---|---|---|
| Backtesting | Replay historical data | No |
| Paper trading | Simulate current market conditions | No |
| Live trading | Execute real orders | Yes |
A successful backtest does not guarantee successful live trading. Each stage reveals different problems.
2. Understand the Backtesting Architecture
Before writing code, let's understand the main components.
A basic backtesting system needs six components:
Historical Market Data
|
v
Market Replay Engine
|
v
Strategy Engine
|
v
Execution Simulator
|
v
Portfolio Manager
|
v
Performance Analyzer
|
v
Backtest Results
Each component has a separate responsibility.
| Component | Responsibility |
|---|---|
| Historical Data | Stores market snapshots and trades |
| Replay Engine | Replays historical events chronologically |
| Strategy Engine | Calculates bid and ask prices |
| Execution Simulator | Simulates order fills |
| Portfolio Manager | Tracks inventory, cash, and P&L |
| Performance Analyzer | Calculates performance and risk metrics |
One important design principle:
The strategy should propose orders, but the execution simulator should decide whether those orders get filled.
This separation makes the system easier to test and improve.
3. Collect Historical Market Data
A backtesting system is only as reliable as its historical data.
For a Polymarket market maker, useful data includes:
| Data | Purpose |
|---|---|
| Timestamp | Reconstruct market events |
| YES best bid/ask | Track YES market prices |
| NO best bid/ask | Track NO market prices |
| Bid/ask depth | Measure available liquidity |
| Trade price | Track executed prices |
| Trade size | Track executed quantities |
| Market ID | Identify the market |
| Token ID | Identify YES or NO tokens |
| Tick size | Validate quote prices |
| Fee information | Calculate trading costs |
| Market status | Identify active or resolved markets |
Why Is Order Book Depth Important?
Consider two markets:
Market A
| Price | Available quantity |
|---|---|
| $0.49 | 1,000 |
| $0.48 | 800 |
| $0.47 | 500 |
Market B
| Price | Available quantity |
|---|---|
| $0.49 | 10 |
| $0.48 | 5 |
| $0.47 | 3 |
Both markets have the same best bid.
However, their liquidity is completely different.
If your bot wants to buy 100 tokens, Market A has much more available liquidity.
This is why using only the best bid and ask is insufficient.
Snapshots vs. Event-Level Data
There are two common approaches:
Order book snapshots
Record the order book at specific intervals.
Event-level data
Record individual changes, such as orders being added, removed, or executed.
Event-level data provides more detailed information and is preferable for realistic execution simulation when available.
4. Calculate Market Metrics and Quotes
Let's start with basic market calculations.
The midpoint is:
midpoint = (best_bid + best_ask) / 2
The spread is:
spread = best_ask - best_bid
For example:
best_bid = 0.48
best_ask = 0.52
midpoint = (best_bid + best_ask) / 2
spread = best_ask - best_bid
print(midpoint) # 0.50
print(spread) # 0.04
Now calculate the market-making quotes.
def calculate_quotes(midpoint, half_spread):
bid = midpoint - half_spread
ask = midpoint + half_spread
return {
"bid": round(bid, 4),
"ask": round(ask, 4)
}
Example:
quotes = calculate_quotes(
midpoint=0.50,
half_spread=0.02
)
print(quotes)
Output:
{
"bid": 0.48,
"ask": 0.52
}
This is our basic quoting model.
In production, we must also validate tick size, order quantity, available balance, market status, and inventory risk.
5. Simulate Realistic Order Execution
This is one of the most important parts of backtesting.
A market price touching your limit order does not automatically mean your order gets filled.
Imagine your bot places a YES buy order at $0.48.
There are already 300 tokens ahead of your order at that price.
Another trader sells 50 tokens at $0.48.
Your order receives nothing because the earlier orders have priority.
This is called queue position.
A simplified execution model:
def calculate_fill_quantity(
trade_volume,
quantity_ahead,
remaining_order_quantity
):
available = max(
0,
trade_volume - quantity_ahead
)
return min(
available,
remaining_order_quantity
)
Example:
fill = calculate_fill_quantity(
trade_volume=150,
quantity_ahead=100,
remaining_order_quantity=80
)
print(fill) # 50
The estimated fill is 50 tokens.
This is only an educational approximation. A realistic simulator must account for actual matching rules, queue changes, and available historical data.
Important Execution Scenarios
Your simulator should eventually support:
- No fills
- Full fills
- Partial fills
- Multiple partial fills
- Order cancellations
- Cancellation delays
- Order replacements
- Expired orders
- Network interruptions
A realistic execution simulator is often more valuable than adding another complicated trading indicator.
6. Track Order States and Inventory
Your backtesting engine needs to understand the lifecycle of each order.
A simplified order state machine:
NEW
|
v
OPEN
|
v
PARTIAL
/ \
v v
FILLED CANCELED
For example, your bot submits an order for 100 YES tokens.
- First execution: 30 tokens
- Second execution: 40 tokens
- Remaining quantity: 30 tokens
Your system must accurately track the executed quantity and remaining quantity.
You also need to track YES and NO inventory separately.
Example:
class Inventory:
def __init__(self):
self.yes_quantity = 0
self.no_quantity = 0
self.yes_cost = 0.0
self.no_cost = 0.0
def buy_yes(self, quantity, price):
self.yes_quantity += quantity
self.yes_cost += quantity * price
def buy_no(self, quantity, price):
self.no_quantity += quantity
self.no_cost += quantity * price
Usage:
inventory = Inventory()
inventory.buy_yes(100, 0.48)
inventory.buy_no(50, 0.47)
print(inventory.yes_quantity) # 100
print(inventory.no_quantity) # 50
A complete portfolio manager should also track:
- Average cost basis
- Sell quantities
- Realized P&L
- Unrealized P&L
- Trading fees
- Cash balance
- Resolution payouts
Remember: a market maker is not automatically market-neutral.
Accumulating too many YES or NO tokens can create significant directional exposure.
7. Calculate P&L and Performance Metrics
Let's calculate a simple completed trade.
Assume:
- Buy price: $0.48
- Sell price: $0.52
- Quantity: 100
- Trading costs: $0.30
Gross P&L:
gross_pnl = (0.52 - 0.48) * 100
print(gross_pnl) # 4.00
Net P&L:
net_pnl = gross_pnl - 0.30
print(net_pnl) # 3.70
But total profit is not enough to evaluate a strategy.
We also need to measure risk.
Important Performance Metrics
1. Net P&L
How much money did the strategy make after trading costs?
2. Fill Rate
What percentage of submitted order quantity was executed?
fill_rate = filled_quantity / submitted_quantity
A higher fill rate is not always better. Aggressive quotes can receive more fills but suffer from adverse selection.
3. Maximum Drawdown
Measures the largest decline from a previous portfolio equity peak.
For example:
| Time | Portfolio equity |
|---|---|
| 1 | $1,000 |
| 2 | $1,100 |
| 3 | $1,050 |
| 4 | $950 |
| 5 | $1,150 |
The drawdown from $1,100 to $950 is:
drawdown = (1100 - 950) / 1100
print(drawdown) # 0.1364
That's approximately 13.64%.
4. Adverse Selection
This happens when your order gets filled and the market immediately moves against you.
Example:
- Your bot buys YES at $0.50.
- The order gets filled.
- The midpoint falls to $0.47.
- Your position loses approximately $0.03 per token at the new midpoint.
A useful measurement is markout:
buy_markout = midpoint_after - fill_price
sell_markout = fill_price - midpoint_after
Measure markout over multiple time horizons, such as 1, 5, and 30 seconds.
5. Inventory Exposure
Track how much capital is tied up in YES and NO positions.
Also consider the possible resolution outcomes instead of relying only on token quantity differences.
8. Optimize Trading Parameters
Now we can start comparing different strategy configurations.
Some important parameters include:
| Parameter | Example values |
|---|---|
| Minimum half-spread | 0.01, 0.02, 0.03 |
| Maximum half-spread | 0.03, 0.05, 0.08 |
| Order size | 5, 10, 20 |
| Maximum inventory | 50, 100, 200 |
| Inventory skew | 0.0001, 0.0002 |
| Quote refresh interval | 1s, 2s, 5s |
We can define experiments in Python:
experiments = [
{
"name": "baseline",
"half_spread": 0.02,
"order_size": 10,
"max_inventory": 100
},
{
"name": "wider_spread",
"half_spread": 0.03,
"order_size": 10,
"max_inventory": 100
},
{
"name": "smaller_inventory",
"half_spread": 0.02,
"order_size": 5,
"max_inventory": 50
}
]
Run every configuration against the same historical dataset.
Then compare the results:
| Strategy | Net P&L | Fill rate | Max drawdown |
|---|---|---|---|
| Baseline | $50 | 45% | 8% |
| Wider spread | $42 | 30% | 4% |
| Smaller inventory | $38 | 35% | 2% |
These numbers are hypothetical examples.
The baseline generates the highest profit, but it also has the largest drawdown.
The smaller-inventory strategy earns less but has lower risk.
Don't optimize only for maximum historical profit.
Consider profitability, drawdown, inventory exposure, adverse selection, and consistency.
9. Avoid Overfitting
Overfitting is one of the biggest problems in quantitative trading.
It happens when you adjust a strategy so heavily to historical data that it performs well on past observations but poorly on new market conditions.
A practical solution is to divide historical data into three periods.
| Dataset | Purpose |
|---|---|
| Training | Develop the strategy |
| Validation | Compare configurations |
| Testing | Evaluate unseen historical data |
For example:
- January–March: Training
- April: Validation
- May: Testing
These dates are illustrative.
Keep the periods chronological. Never allow future market information to influence earlier trading decisions.
Walk-Forward Testing
Another useful approach is walk-forward testing.
- Tune parameters using an earlier period.
- Test the selected configuration on the next unseen period.
- Move the testing window forward.
- Repeat.
- Compare performance across all periods.
This helps determine whether your strategy can adapt to changing market conditions.
10. Account for Realistic Trading Conditions
A good backtest should not assume perfect execution.
Consider these factors:
| Factor | Why it matters |
|---|---|
| Trading fees | Reduce captured spread |
| Queue position | Orders may not receive fills |
| Partial fills | Orders may execute gradually |
| Latency | Prices can change before execution |
| Cancellation delays | Orders may fill before cancellation completes |
| Adverse selection | Filled orders may move against you |
| Resolution risk | Open positions have final settlement outcomes |
I recommend testing three execution scenarios:
Optimistic: Higher fill probability and lower latency.
Base case: Conservative queue and execution assumptions.
Stress case: Lower fill probability, higher costs, and delayed cancellations.
If your strategy only works under optimistic assumptions, it needs further investigation.
11. Organize the Python Project
As the backtesting system grows, avoid putting everything into one Python file.
A modular project structure could look like this:
polymarket-market-maker/
│
├── config/
│ ├── backtest.yaml
│ └── strategy.yaml
│
├── data/
│ ├── historical/
│ └── processed/
│
├── src/
│ ├── data/
│ │ ├── loader.py
│ │ └── validator.py
│ │
│ ├── strategy/
│ │ ├── market_maker.py
│ │ ├── spread_model.py
│ │ └── inventory_skew.py
│ │
│ ├── execution/
│ │ ├── simulator.py
│ │ ├── order.py
│ │ └── fill_model.py
│ │
│ ├── portfolio/
│ │ ├── inventory.py
│ │ └── pnl.py
│ │
│ ├── backtesting/
│ │ ├── engine.py
│ │ └── metrics.py
│ │
│ └── main.py
│
├── tests/
├── results/
├── requirements.txt
└── README.md
Each directory has a clear responsibility.
The main advantage is that the strategy can eventually be reused with different execution adapters:
- Backtesting execution
- Paper trading execution
- Live exchange execution
This makes the system easier to maintain and extend.
12. Common Backtesting Mistakes
Before finishing, remember these common mistakes:
- Assuming every order gets filled.
- Ignoring trading fees.
- Ignoring partial executions.
- Using future information.
- Optimizing only for historical profit.
- Ignoring remaining inventory.
- Testing on insufficient historical data.
- Treating paper trading as proof of live execution.
- Changing too many parameters simultaneously.
A backtest should help you understand where your strategy fails, not just where it succeeds.
Conclusion
Building a Polymarket market maker bot is not just about calculating attractive bid and ask prices.
You also need to understand:
- Whether your orders can realistically get filled.
- How much inventory they accumulate.
- How execution costs affect profitability.
- How the strategy behaves under different market conditions.
- Whether its performance remains consistent on unseen data.
Remember these three lessons:
- Realistic execution matters more than impressive backtest profits.
- Always test your strategy on historical data it has never seen before.
- Risk management and inventory exposure matter as much as profitability.
A strategy that survives realistic simulations and unseen historical periods is a stronger candidate for paper trading.
However, even a strong backtest cannot guarantee future profitability.
What's Next? Day 10
Polymarket Market Maker Bot: From Backtesting to Live Trading and Production Monitoring
In the next article, we'll explore:
- Connecting the strategy to an execution engine.
- Moving from backtesting to paper trading.
- Monitoring orders and partial fills.
- Handling API failures and network interruptions.
- Managing risk during live operation.
- Building production monitoring workflows.
Follow the Project
If you're interested in building a Polymarket trading bot with Python, check out the project:
-
GitHub:
Benj4minCup
/
Polymarket-Trading-Bot-python
polymarket Twap Bot polymarket Arbitrage Bot polymarket bot
Polymarket Trading bot system
An open-source and Strong Strategy collection of Polymarket trading bot and Polymarket arbitrage bot and Polymarket TWAP trading bot in Python for high-performance automated trading on polymarket crypto 5min and 15min markets.
This repository is primarily intended for educational and research purposes. It includes strategy concepts, implementation approaches, and selected performance screenshots to help developers understand how different automated trading strategies can be designed and tested.
The repository does not provide a complete production-ready trading bot source code. Instead, it provides strategy descriptions and research materials that you can use as a foundation for developing your own system.
If you are interested in building a Polymarket Trading Bot, you can follow my tutorials and use the concepts in this repository to develop your own implementation.
For users who prefer a ready-to-deploy solution or require custom strategy development, commercial bot development and customization are also available.
Features
… YouTube:
Telegram: https://telegram.me/BenjaminCup
Disclaimer: This article is for educational purposes only and does not constitute financial advice. Backtesting and simulated performance do not guarantee future results. Prediction-market trading involves financial risk, including the possibility of losing your entire trading capital.
Top comments (0)