Evaluating Multi-Horizon Trading Models in Python: Forward Returns, Signal Decay, and Overlapping Labels

Disclosure: AI assistance was used in preparing this article. Code examples are illustrative and have not been tested against a live trading system.

A machine learning model can perform well on a 24-hour return prediction task while providing little useful information for a 30-minute trading decision.

This is not necessarily a modeling failure. It may be a mismatch between the prediction target and the intended use of the model.

For developers building financial machine learning systems, multi-horizon evaluation should cover three separate questions:

1. How accurately does the model predict returns over each horizon?
2. How long does the original signal retain predictive information?
3. Does the resulting trading policy generate value after transaction costs?

1. Construct Horizon-Specific Targets

For log prices (p_t=\ln(P_t)), a forward return target can be defined as:

[
r_{t,H}=p_{t+H}-p_t
]

A model predicting a 5-step return and one predicting a 60-step return are solving different supervised learning problems.

They should have separate target definitions and separate evaluation results.

A basic implementation using pandas is:

import numpy as np
import pandas as pd

def build_forward_targets(prices, horizons):
    log_prices = np.log(prices.astype(float))

    return pd.DataFrame({
        f"return_{h}": log_prices.shift(-h) - log_prices
        for h in horizons
    })

# Example: prices indexed by regular time intervals.
targets = build_forward_targets(
    prices,
    horizons=[5, 15, 60, 288]
)

The horizon values represent the number of observations, not fixed calendar durations. If observations are irregular, timestamp-based alignment is required.

These forward targets overlap when they are generated at frequent intervals.

That overlap must be considered during validation.

2. Measure Signal Decay Using Future Return Intervals

A cumulative return forecast does not reveal when the expected return is generated.

To investigate signal persistence, evaluate the original signal against successive, non-overlapping future return intervals.

For example, with five-minute intervals:

[
r_{0,5}=p_{t+5}-p_t
]

[
r_{5,10}=p_{t+10}-p_{t+5}
]

[
r_{10,15}=p_{t+15}-p_{t+10}
]

This separates the initial response from subsequent market movements.

An illustrative implementation is:

import pandas as pd
import numpy as np

def future_return_buckets(log_prices, step=5, buckets=6):
    result = {}

    for i in range(buckets):
        start = i * step
        end = (i + 1) * step

        result[f"{start}_{end}"] = (
            log_prices.shift(-end)
            - log_prices.shift(-start)
        )

    return pd.DataFrame(result)


# signal must be generated using information available at time t.
log_prices = np.log(prices.astype(float))

future_returns = future_return_buckets(
    log_prices,
    step=5,
    buckets=6
)

evaluation = future_returns.join(
    signal.rename("signal")
)

decay = evaluation.corr(
    method="spearman"
).loc["signal"].drop("signal")

print(decay)

The resulting correlations provide a simple view of how the original signal relates to successive future returns.

They are not a complete measure of signal decay.

A stronger analysis should also examine conditional returns by signal strength, statistical uncertainty, different market regimes, and the impact of execution costs.

3. Handle Overlapping Labels Correctly

Suppose a model generates a 60-minute forecast every five minutes.

Adjacent targets share 55 minutes of future price information.

This means the forecast errors can exhibit serial dependence.

A conventional random train-test split may also place overlapping label intervals in different subsets.

The evaluation pipeline should therefore preserve each observation's prediction timestamp and label-end timestamp.

Where training and testing labels share future information, affected training observations should be purged.

For suitable cross-validation designs, an embargo may also be needed.

For strictly chronological walk-forward validation, training should use only information available before the test period, and any training labels that extend into the test interval must be excluded.

Statistical comparisons across overlapping forecast windows should account for serial correlation rather than treating all forecast errors as independent.

The Diebold–Mariano framework and its multi-horizon extensions provide useful approaches for comparing predictive accuracy under dependent forecast errors.

4. Separate Forecast Quality From Trading Performance

A model's prediction error is not a direct measure of trading profitability.

For each horizon, researchers should report:

• MAE or RMSE for forecast error.
• IC or rank IC for predictive relationships.
• Calibration and prediction stability.
• Signal persistence across subsequent intervals.
• Turnover and transaction costs.
• Net returns and drawdown under a defined execution policy.

The holding period should be selected through strategy evaluation, not simply copied from the model's forecast horizon.

A longer-lived signal may justify slower position adjustment, while a short-lived signal may require faster execution.

Gârleanu and Pedersen's research shows how different signal mean-reversion speeds and transaction costs affect optimal dynamic portfolio adjustments.

5. Build a Multi-Horizon Model Architecture

A practical architecture can assign different responsibilities to different models:

• Short-horizon models: near-term market conditions and execution.
• Intermediate-horizon models: trend persistence and position management.
• Long-horizon models: broader market conditions and risk allocation.

The outputs should be combined through an explicit decision policy.

They should not simply vote on buy or sell decisions.

For production systems, each model should have a documented prediction target, information cutoff, update frequency, validation method, and intended use.

For KAEL AI's quantitative research, this separation is important because a predictive model should be evaluated not only by what it forecasts, but also by how its output behaves when passed into a real trading and risk-management system.

Conclusion

Multi-horizon evaluation is not simply a matter of training the same model with several different target windows.

It requires understanding overlapping labels, signal decay, prediction uncertainty, execution costs, and the relationship between forecasts and actual holding decisions.

A model should only be used for decisions that its validated predictive information can reasonably support.

The goal is not to maximize forecast accuracy at every horizon independently. It is to build a coherent system in which predictive information, portfolio adjustments, and risk constraints work together.

References

Gârleanu & Pedersen (2013). Dynamic Trading with Predictable Returns and Transaction Costs. The Journal of Finance.

Grant, Mrazik & Satchell (2026). Evaluating Forecasts at Multiple Horizons. Journal of Forecasting.

López de Prado (2018). Advances in Financial Machine Learning. Wiley.