You asked for a crossover, but the code buys whenever one line is above another. Check the rules, timeframe, sizing and fees before trusting the returns.
You ask an AI for a strategy that buys when one moving average crosses above another. The backtest runs and the chart looks good. Then the logs reveal a different condition: buy whenever the fast average is above the slow one. The software solved a different problem successfully. How do you verify the rules before trusting the returns?
Does a working backtest prove the strategy is correct?
Successful execution shows that the program ran. Correct implementation requires comparing the requested rules with observed behavior. Compilation, a clean run and a performance report do not individually establish closed-candle confirmation, crossover timing or position limits.
Imagine ordering beef noodles and getting fried rice. It is cooked, but it is still the wrong order. Check which conditions the strategy actually traded before judging its returns.
How is “crosses above” different from “is above”?
A crossover is an event: the fast line was no higher than the slow line on the previous candle and is higher now. “Is above” is a state on the current candle. They can agree at one timestamp and diverge on the next.
Suppose the slow line stays at 100 across three candles while the fast line moves from 99 to 101 to 102. Put the two conditions side by side:
| Time | Fast | Slow | Fast is above slow | Crosses above now |
|---|---|---|---|---|
| A | 99 | 100 | No | No previous value supplied |
| B | 101 | 100 | Yes | Yes |
| C | 102 | 100 | Yes | No |
The fast line is above the slow line at both B and C, but the crossover happens only at B. If the program buys again at C, check which condition it used and whether repeat entries are allowed. Those records make the extra order much easier to explain.

What belongs in a trading-strategy specification?
- Data: market, timeframe, price field and closed versus unfinished candles.
- Entry: an event or a persistent state, equality behavior and repeated entries.
- Exit: stop conditions and priority when several exits trigger together.
- Size: allocation or risk-based sizing, maximum exposure and additional entries.
- Timing: signal time, order submission time and fill assumptions.
- Costs: fee and slippage settings, plus the evaluation start.
Replace “add a little after a large drop” with a reference price, threshold, amount and maximum count. Unspecified choices should become questions to resolve, not silent changes to the strategy.
Do timeframe, position size and fees need checking too?
Yes. If you requested confirmation on a closed four-hour candle but the program uses hourly data, the trading schedule has already changed even if the crossover is correct. A fixed order amount and a fraction of account equity also create different exposure.
A blank fee field does not automatically mean zero fees. Check the effective rates, starting funds and fill assumptions used by that run; platforms can apply their own defaults. Save them with the strategy version so you can explain why two results differ.
How do you validate AI-generated trading rules?
- Freeze the request. Save a dated version instead of overwriting the original.
- Map each rule to its implementation. Identify parameters, conditions and unresolved defaults.
- Write small cases with expected behavior. Include no entry, exact threshold, equality, an existing position and conflicting exits.
- Inspect events. Separate data timestamps, indicator values, conditions, orders and fills. Find the first disagreement.
- Repeat checks after a fix. Record the change and rerun previously correct cases.
Start with a few candles you can work through yourself and check whether the trades match your rules. Then extend the backtest. There is no need to wait through years of data to find a mistake visible in a handful of candles.
Can an AI debugging fix change the strategy?
Sometimes the code is wrong; sometimes a condition was never fully specified. A fix that swaps an indicator or loosens a threshold also changes the strategy. You can make that change, but record it and check the new rules. Otherwise, what looks like a repaired program may be a different trading approach.
The QuantHarness paper examines research trajectories beyond final code, including specifications, data, repairs and execution evidence. Its useful lesson here is to retain the path to the answer.
What comes after implementation checks?
Evaluate which AI strategies deserve further testing, parameter-selection bias, costs and out-of-sample behavior. Correct rules and a useful trading edge are separate conclusions.
When you already have rules and backtest material, AlgoRover Research can help organize further research questions. Once the rules are checked, take the same strategy version and configuration into execution and inspect whether the order sequence matches your intent.
Frequently asked questions
Is asking the same AI to review its code enough?
It can help, but retain independently specified expected behavior and actual run evidence. Another reviewer can examine important assumptions without inheriting the same interpretation.
Can I review code without checking trades?
Code describes intended behavior; logs show this run's behavior. Data, timing and order settings can change the outcome, so compare both.
Can I keep an AI parameter change if returns improve?
Treat it as a new research candidate, record the change and validate it. Improved returns do not establish compliance with the original request.
Does passing rule checks make a strategy live-ready?
Data quality, costs, out-of-sample evidence and execution checks still matter. Implementation consistency is one gate.
Check whether orders follow the rules
Make “buy on the crossover” and “no repeat entry while holding” explicit, then run the reviewed version in a supported exchange Demo with AlgoRover Live. Inspect orders and positions to check whether entries happen when intended and whether unwanted repeat entries occur.
Continue checking execution with LiveSources
- QuantHarness research paper
- Full paper: research trajectories and evidence
Not investment advice.
Top comments (0)