DEV Community: Artem

How I Built an Automated Multi-Asset Divergence Engine in Python & Django

Artem — Wed, 12 Aug 2026 18:42:51 +0000

Screening 100+ financial assets across multiple timeframes for technical chart anomalies is an impossible task for a human eye. If you want to detect momentum divergence across Forex, Crypto, Metals, and Commodities in real time, you need an automated data pipeline.

In this post, I will break down the engineering architecture behind the AEMMtrader automated divergence engine: how we calculate real-time extrema, filter out false signals using Higher-Timeframe (MTF) trend confirmation, and dynamically plot ATR execution levels.

System Architecture Overview

The system runs on a modern Python/Django backend with Celery background workers managing scheduled websocket/REST market data streams.

[ Market Feeds: Crypto / FX / Metals ] 
                   │
                   ▼
       [ Celery Worker Queue ] 
                   │
                   ▼
  [ Technical Engine: Pandas / NumPy ]
    ├─ 1. Calculate Oscillators (RSI, Stoch, CCI, MACD, UO)
    ├─ 2. Peak & Trough Extrema Detection
    ├─ 3. Multi-Timeframe (MTF) Trend Verification
    └─ 4. Dynamic Volatility (ATR) Sizing
                   │
                   ▼
    [ PostgreSQL & Redis Cache ]
                   │
                   ▼
  [ Client UI: Django Templates + DataTables + Plotly.js ]

1. The Core Algorithm: Detecting Mathematical Divergence

To identify a divergence, the algorithm must find consecutive local peaks (or troughs) in both the price series and the oscillator series, then evaluate their relative slopes.

Here is a simplified Python representation using pandas and scipy.signal.argrelextrema:

import numpy as np
import pandas as pd
from scipy.signal import argrelextrema

def detect_regular_bearish_divergence(df, order=5):
    """
    df requires: 'high', 'close', 'indicator' (e.g. RSI, Stochastic)
    order: number of points on each side to use for peak comparison
    """
    # Find local price maxima
    price_peaks_idx = argrelextrema(df['high'].values, np.greater, order=order)[0]

    # Find local indicator maxima
    ind_peaks_idx = argrelextrema(df['indicator'].values, np.greater, order=order)[0]

    if len(price_peaks_idx) < 2 or len(ind_peaks_idx) < 2:
        return None

    # Get the last two structural peaks
    p_last, p_prev = price_peaks_idx[-1], price_peaks_idx[-2]
    i_last, i_prev = ind_peaks_idx[-1], ind_peaks_idx[-2]

    # Temporal alignment check (ensure peaks correspond to similar candles)
    if abs(p_last - i_last) <= 2 and abs(p_prev - i_prev) <= 2:
        price_hh = df['high'].iloc[p_last] > df['high'].iloc[p_prev]
        ind_lh = df['indicator'].iloc[i_last] < df['indicator'].iloc[i_prev]

        if price_hh and ind_lh:
            return {
                "type": "Bearish Divergence",
                "price_points": (p_prev, p_last),
                "ind_points": (i_prev, i_last),
                "bars_ago": len(df) - 1 - p_last
            }
    return None

2. Solving the False-Positive Problem: Multi-Timeframe (MTF) Filtering

A common failure mode in quantitative divergence detection is picking tops in strong trending markets. To solve this, our pipeline enforces an MTF Filter:

For an M5 / M15 signal, we query the H1 trend direction via an EMA 50/200 structural slope filter.
For an H1 signal, we query the D1 trend direction.

If a Bearish Divergence occurs while the higher timeframe is in a confirmed downtrend, the state is tagged as Confirmed (Trend Down) and prioritized in the database.

You can explore the live implementation of this table directly on the AEMMtrader Divergence Dashboard.

3. Dynamic ATR Execution Engine

Instead of outputting raw alerts, the system computes actionable entry, stop-loss, and take-profit targets based on volatility:

Stop Loss = Entry Price + (1.5 \times ATR_{14})

Take Profit = Entry Price - (3.0 \times ATR_{14})

This enforces a mandatory 1:2 Risk-to-Reward ratio tailored to the current market environment.

4. Client-Side Rendering with Plotly.js

When a user clicks "View" in the dashboard, an asynchronous AJAX endpoint returns the OHLC payload, indicator arrays, and calculated divergence coordinates.

The client renders an interactive dual-axis chart without page reloading:

// Sample snippet rendering the indicator trace with Plotly
let traceCandles = {
    x: res.datetime, open: res.open, high: res.high, low: res.low, close: res.close,
    type: 'candlestick', name: 'Price', yaxis: 'y'
};

let traceOscillator = {
    x: res.datetime, y: res.indicator,
    type: 'scatter', mode: 'lines', name: res.oscillator_name,
    line: { color: '#8e44ad', width: 2 }, yaxis: 'y2'
};

let layout = {
    grid: { rows: 2, columns: 1, pattern: 'independent' },
    yaxis: { domain: [0.3, 1], title: 'Price' },
    yaxis2: { domain: [0, 0.25], title: res.oscillator_name },
    xaxis: { rangeslider: { visible: false } }
};

Plotly.newPlot('plotlyContainer', [traceCandles, traceOscillator], layout);

Conclusion & What’s Next
By combining mathematical peak detection, higher-timeframe trend verification, and dynamic ATR volatility sizing, we transformed a discretionary charting concept into an automated quant engine.

Check out the live platform at AEMMtrader.com or explore the screener directly at AEMMtrader Live Divergences.

Have thoughts on refining peak detection algorithms or optimizing multi-timeframe caching in Django? Drop your ideas in the comments below!

Handling Non-Stationary Time Series: Building a Probabilistic Engine with XGBoost & Python

Artem — Wed, 20 May 2026 19:32:02 +0000

If you have ever tried to apply Machine Learning to financial time series, you know the heartbreak of the "perfect backtest." You build a model, train it on historical OHLC (Open, High, Low, Close) data, and it predicts the next sequence beautifully. Then you deploy it to production, the market regime shifts, and your model falls apart.
The core issue is that financial markets are highly non-stationary and chaotic. Deterministic models—those trying to predict a single, exact future price - are statistically fragile. They assume the future will exactly mirror the past.
At AEMMtrader, we spent the last year entirely rethinking our architecture. We stopped trying to predict one path and built a Python-based forecasting engine that treats the market as a probabilistic "multiverse," combining the non-linear regression power of XGBoost with the stress-testing capabilities of Monte Carlo simulations.
Here is a deep dive into the architecture and the Python code powering our engine.

1. The Core Engine: Why XGBoost?

While Deep Learning (LSTMs, Transformers) gets a lot of hype, tree-based models like Gradient Boosted Decision Trees (XGBoost) consistently outperform them on structured, tabular data.
However, feeding raw price data into an XGBoost model is a recipe for overfitting. Our feature vector engineers metrics that describe the state of the market rather than absolute prices:

Volatility-Adjusted Returns: Measuring the energy of a move.
Momentum and Relative Volume: Capturing sudden institutional liquidity shifts.
RSI dynamically calculated via Pandas: To gauge mean-reversion probabilities.

2. Injecting Chaos: The Monte Carlo Layer

Even with great features, XGBoost can overfit to recent market noise. To prevent "fake breakouts," we wrap our predictive model in a Monte Carlo simulation loop.

Instead of running the model once, we run it 30 independent times. During each simulation, we inject calibrated stochastic noise into the input features (both price and volume). If the underlying signal is robust, the model will fight through the noise and converge on the same destination. If it's just market noise, the 30 paths will scatter randomly, and our engine flags the asset state as "Neutral."

3. The Architectural Blueprint (Python)

To protect our proprietary logic, I won't share the exact production code, but here is the foundational architecture of how we built the engine. Instead of a monolithic script, we broke the problem down into three independent logical blocks.

Block A: State-Based Feature Engineering

We never feed raw prices into the XGBoost model. Instead, we transform the price action into a vector that describes the "energy" and "state" of the market.

import pandas as pd
import numpy as np
import xgboost as xgb

class ProbabilisticEngine:
    def __init__(self, timeframe):
        self.timeframe = timeframe
        self.model = xgb.XGBRegressor(max_depth=5, n_estimators=80) # Base concept

    def _engineer_features(self, df):
        """
        Calculates the state of the market rather than absolute price.
        """
        features = pd.DataFrame(index=df.index)

        # 1. Energy: Logarithmic returns
        features["log_ret"] = np.log(df["close"] / df["close"].shift(1))

        # 2. Institutional Activity: Relative volume spikes
        features["vol_rel"] = df['tick_volume'] / df['tick_volume'].rolling(20).mean()

        # 3. Market Structure: Custom momentum and mean-reversion metrics
        # (Proprietary calculations omitted)
        features["momentum"] = self._calculate_momentum(df)

        return features.dropna()

Block B: The Monte Carlo Multiverse Loop

This is the heart of the engine. Once the model predicts the base mathematical expectation for the next candle, we inject stochastic noise (based on recent market volatility) and simulate the future 30 times.

def generate_multiverse(self, current_state, n_steps=20, simulations=30):
        """
        Generates a matrix of possible future price paths.
        """
        all_paths = []
        historical_volatility = current_state["close"].pct_change().std()

        for _ in range(simulations):
            path = []
            simulated_state = current_state.copy()

            for step in range(n_steps):
                # 1. XGBoost predicts the expected baseline move
                features = self._engineer_features(simulated_state)
                expected_move = self.model.predict(features.tail(1))[0]

                # 2. Inject stochastic noise to simulate market chaos
                price_noise = np.random.normal(0, historical_volatility)
                next_price = simulated_state["close"].iloc[-1] * np.exp(expected_move + price_noise)

                path.append(next_price)

                # 3. Step forward in time (update the simulated state)
                simulated_state = self._update_state(simulated_state, next_price)

            all_paths.append(path)

        return np.array(all_paths) # Returns a shape of (30, 20)

Block C: Synthesizing the Consensus

Having an array of 30 different paths is useless for execution. We must compress this "multiverse" into a clear, actionable signal with a mathematical confidence score.

def extract_signal(self, current_price, all_paths):
        """
        Translates the Monte Carlo matrix into clean probabilities.
        """
        # Calculate the mathematical average of all 30 paths
        mean_path = np.mean(all_paths, axis=0)

        # Calculate Confidence Score based on terminal path locations
        bullish_paths = sum(path[-1] > current_price for path in all_paths)
        buy_probability = bullish_paths / len(all_paths)

        # Calculate dynamic wicks (High/Low) using historical ATR

        return {
            "mean_trajectory": mean_path,
            "confidence_score": buy_probability * 100,
            "direction": "BUY" if buy_probability > 0.5 else "SELL"
        }

4. Synthesizing the "Multiverse" into Actionable Visuals

Most Monte Carlo implementations output a "spaghetti chart" displaying dozens of overlapping lines. This is visually overwhelming.

Instead, look at the np.mean calculation in the code above. The engine calculates the Mean Probability Path across all 30 noise-injected simulations. From this path, it mathematically reconstructs future OHLC candles. The body of the future candle is the mean trajectory, while the wicks (high/low variance) are bound dynamically by a fraction of the Average True Range.

In the chart above, you can see the sequence of forecasted candles extending to the right. This is not a single deterministic guess. If 21 out of our 30 simulations closed above the starting price despite the injected noise, the underlying mathematical confidence heavily favors the upside." width="800" height="333">
Case Study: EUR/USD on the D1 Timeframe
In the chart above, you can see the sequence of forecasted candles extending to the right. This is not a single deterministic guess. If 21 out of our 30 simulations closed above the starting price despite the injected noise, the underlying mathematical confidence heavily favors the upside.

5. Smart Caching for Real-Time Performance

To make this viable in production across hundreds of assets and multiple timeframes, we couldn't afford to retrain the XGBoost model on every single tick.

We built an orchestrator layer that implements a smart caching logic based on the timeframe limits:

if model.tf_min >= 1440:  # D1, W1
    retrain_limit = 10
elif model.tf_min == 240:  # H4
    retrain_limit = 18
else:  # H1, M30
    retrain_limit = 24

if new_candles_count < retrain_limit:
    needs_training = False
    model.update_data(df) # Hot reload via state update

If the threshold of new candles isn't met, the model bypasses the heavy .fit() phase, updates the feature array via a hot reload, and recalculates the Monte Carlo matrix in milliseconds.

Final Thoughts

Transitioning from deterministic logic (if RSI < 30 then Buy) to probabilistic machine learning requires a mental shift. It means accepting that markets are chaotic, and our job as quantitative developers is not to predict the exact future, but to trap the price within a mathematical range of probabilities.

If you're interested in seeing how this engine outputs these Monte Carlo OHLC structures in real-time across various timeframes and assets, you can monitor the live dashboard at AEMMtrader.com.

I would love to hear how other Python developers and Data Scientists handle non-stationarity in their ML models. What is your go-to method for preventing overfitting on financial data? Drop your thoughts in the comments!