When building a Polymarket bot, the first version can be surprisingly small:
market data
↓
strategy
↓
order
That's enough to demonstrate an idea.
It isn't enough to prove that the idea works.
Once you care about realistic execution, the architecture becomes more interesting.
Market Data
↓
Data Validation
↓
Signal Engine
↓
Risk Engine
↓
Execution Engine
↓
Trade Events
↓
Analytics
This separation is what allows me to test the strategy independently from the infrastructure.
1. Don't backtest the API call
One mistake I see in trading-bot development is mixing the strategy with execution.
For example:
if (signal) {
await placeOrder();
}
This is convenient for a prototype.
But how do you test the strategy without sending an order?
Instead:
const signal = strategy.evaluate(marketState);
const decision = riskEngine.check(signal, portfolio);
if (decision.allowed) {
await executionEngine.submit(signal);
}
Now each component can be tested independently.
2. Model execution separately
A backtest shouldn't assume:
signal price === fill price
Instead, the execution simulator should model things such as:
signal price
spread
slippage
available liquidity
fees
latency
Then:
expected PnL
↓
execution model
↓
realistic PnL estimate
The difference can be substantial.
Polymarket's CLOB exposes order-book data and executable prices, making the order book an important part of any execution-aware strategy.
3. Separate in-sample and out-of-sample data
Don't optimize and evaluate on the same dataset.
A simple structure:
Dataset
├── Train
└── Test
The strategy is developed using Train.
Parameters are frozen.
Then Test is used only for evaluation.
For time-series trading, I prefer chronological splits rather than random shuffling:
Past ───────────────────────> Future
[ Training ][ Validation ][ Test ]
This better represents the actual information flow of a trading system.
4. Measure more than win rate
Win rate is useful, but insufficient.
I want to measure:
trades
wins
losses
gross PnL
fees
slippage
net PnL
average trade
max drawdown
profit factor
For example:
Net PnL =
Gross PnL
- Trading Fees
- Slippage
A 65% win rate can still produce a bad strategy.
A lower win rate can be profitable if the payoff distribution is favorable.
5. Treat market data as untrusted input
Real-time market data can fail.
The system should explicitly handle:
CONNECTED
DISCONNECTED
RECONNECTING
STALE
RECOVERING
HEALTHY
Polymarket provides public WebSocket channels for near-real-time market, order-book and trade updates, while RTDS provides streaming crypto-price data.
The trading engine shouldn't assume that every received event is valid.
For example:
if (Date.now() - lastUpdate > MAX_DATA_AGE) {
return NO_TRADE;
}
A missing signal is better than a signal generated from stale information.
6. Make the strategy deterministic
One of my favorite properties for a trading strategy is:
Given the same market state, it should produce the same decision.
For example:
const decision = strategy.evaluate(state);
This makes it possible to replay historical events:
event 1
event 2
event 3
event 4
...
and reproduce the strategy's decisions.
That is extremely useful when debugging.
7. Record every decision
A useful event log might contain:
{
"timestamp": "...",
"market": "...",
"signal": "...",
"price": 0.48,
"expectedValue": 0.03,
"riskApproved": true,
"action": "BUY"
}
Later, you can ask:
Why did the bot enter this position?
without reconstructing the entire system manually.
8. Replay is one of the most useful tools
Once events are stored, you can replay them.
Historical Events
↓
Event Replay
↓
Strategy
↓
Execution Simulator
↓
Results
Now you can change the strategy without recollecting all the market data.
You can also compare:
Strategy A
vs
Strategy B
against exactly the same events.
That's much more useful than comparing two completely different live runs.
9. Test failure paths
Don't only test:
data arrives
signal works
order succeeds
Test:
WebSocket disconnects
API timeout
stale data
empty order book
partial fill
order rejection
duplicate event
duplicate order
process restart
A trading system becomes much more robust when failure behavior is designed explicitly.
10. The research loop
My preferred development loop is:
Hypothesis
↓
Data
↓
Backtest
↓
Out-of-sample
↓
Execution simulation
↓
Paper trading
↓
Small live test
↓
Measure
↓
Improve
This matters because a recent public high-frequency study using synchronized Polymarket/Binance data found that an out-of-sample model did not outperform Polymarket's own implied probabilities, while simulated trading was negative under its stated assumptions.
That's exactly why I don't consider a profitable backtest to be the finish line.
It's the beginning of validation.
Final architecture
The system I want to build eventually looks like:
┌───────────────┐
│ Market Data │
└───────┬───────┘
↓
┌───────────────┐
│ Data Validator│
└───────┬───────┘
↓
┌───────────────┐
│ State Manager │
└───────┬───────┘
↓
┌───────────────┐
│ Strategy │
└───────┬───────┘
↓
┌───────────────┐
│ Risk Engine │
└───────┬───────┘
↓
┌───────────────┐
│ Execution │
└───────┬───────┘
↓
┌───────────────┐
│ Event Store │
└───────┬───────┘
↓
┌───────────────┐
│ Analytics │
└───────────────┘
The goal isn't to build the biggest bot.
It's to build a system where I can answer:
What happened?
Why did it happen?
Would it have happened under different execution conditions?
Does the strategy still work on unseen data?
Can I reproduce the decision?
That's the difference between a trading script and an engineering system.
Backtests and simulations are research tools, not guarantees of future trading performance.
Top comments (0)