Why I stopped trading and started tendering with AI

Why I Stopped Trading and Started Tendering with AI

For years, I chased the dragon of algorithmic trading. Backtesting frameworks, complex indicators, optimization loops… I sunk countless hours (and a not-insignificant amount of capital) into trying to predict the market. The siren song of automated profits was strong, but ultimately, I found myself increasingly frustrated. The market, it turns out, is remarkably good at not being predictable.

Then, I had a shift in perspective. Instead of trying to predict where the market will be, what if I focused on predicting what someone else is willing to pay right now? This led me down the path of automated tendering, specifically leveraging AI to bid on digital advertising inventory. And honestly? It's been a game changer.

This isn't about overnight riches (though it has been profitable). It's about a fundamentally different approach to financial automation, one that feels less like gambling and more like intelligent resource allocation.

The Problem with Prediction: Trading is Zero-Sum

Traditional algorithmic trading relies on identifying patterns and exploiting inefficiencies. The problem? Everyone else is trying to do the same thing. Any profitable pattern gets arbitraged away almost instantly. It’s a zero-sum game. For every dollar you make, someone else is losing a dollar.

Even sophisticated strategies like mean reversion or momentum trading are constantly battling against increasingly sophisticated counter-strategies. The edge degrades rapidly. Plus, the latency race is brutal. You need lightning-fast execution to capitalize on fleeting opportunities.

I found myself caught in an endless cycle of optimization, only to see profits evaporate as soon as a new player entered the field or market conditions shifted. The effort-to-reward ratio felt… unsustainable.

Enter Tendering: A Different Kind of Game

Tendering, in the context of digital advertising, is a process where advertisers bid on ad impressions in real-time. Think programmatic advertising, Real-Time Bidding (RTB). It's a highly competitive landscape, driven by complex algorithms on both the buy-side (advertisers) and sell-side (publishers).

The key difference from trading is that it’s not zero-sum. A successful bid benefits both the advertiser (by acquiring valuable impressions) and the publisher (by maximizing revenue). There's a genuine value exchange, and the opportunity to create value through intelligent bidding.

The core idea? Build an AI that can accurately predict the value of an impression to an advertiser and bid accordingly, without necessarily knowing the future price of the impression.

My Tech Stack & Approach: From Python to PyTorch

My initial attempts at tendering were using simple rule-based systems. They were… underwhelming. That’s when I pivoted to machine learning. Here's the core of what I built:

• Language: Python (obviously). It's the workhorse of data science and has excellent libraries for ML.
• ML Framework: PyTorch. I opted for PyTorch over TensorFlow due to its more dynamic graph and Pythonic feel, which felt more conducive to rapid experimentation.
• Data Sources: I'm consuming data from a Demand-Side Platform (DSP) via their API. This includes features like:
  • User demographics (age, gender, location)
  • Website/App category
  • Device type (mobile, desktop, tablet)
  • Ad size
  • Time of day/day of week
  • Historical click-through rates (CTR) and conversion rates (CVR)
• Model Architecture: I started with a relatively simple Feedforward Neural Network (FNN) but quickly moved to a more sophisticated architecture – a combination of an Embedding layer, followed by a series of FNN layers, and finally, a regression output.

The Code Snippet: A Simplified Bid Prediction Model

Let's look at a simplified version of my PyTorch model (using PyTorch Lightning for training):

import torch
import torch.nn as nn
import pytorch_lightning as pl

class BidPredictor(pl.LightningModule):
    def __init__(self, input_dim, embedding_dim=8):
        super().__init__()
        self.embedding = nn.Embedding(input_dim, embedding_dim)
        self.fc1 = nn.Linear(embedding_dim, 64)
        self.fc2 = nn.Linear(64, 32)
        self.output = nn.Linear(32, 1)
        self.relu = nn.ReLU()

    def forward(self, x):
        embedded = self.embedding(x)
        x = self.relu(self.fc1(embedded))
        x = self.relu(self.fc2(x))
        x = self.output(x)
        return x

# Example Usage
input_dim = 100  # Number of possible features (e.g., website categories)
model = BidPredictor(input_dim)
print(model)

Explanation:

• nn.Embedding: Crucially, many features are categorical (e.g., website category). The Embedding layer learns a vector representation for each category, allowing the model to understand relationships between them. This is far more effective than one-hot encoding.
• nn.Linear: Standard fully connected layers to process the embedded data.
• nn.ReLU: Activation function for non-linearity.
• forward(x): Defines the forward pass of the network, taking input x (categorical feature indices) and predicting a single bid value.

Training and Optimization: Continuous Learning

The model is trained on historical bid request data. I use a Mean Squared Error (MSE) loss function, aiming to minimize the difference between predicted bids and actual won bid prices.

However, unlike static backtesting in trading, this is a continual learning system. The market (and user behavior) changes constantly. I implemented a system to retrain the model daily with the latest data.

Furthermore, I incorporated an exploration-exploitation strategy. I don’t always bid the model’s predicted value. Occasionally, I add a small random factor to encourage exploration of different bid prices and uncover new optimal strategies.

From Predictions to Bids: The Bidding Engine

The heart of the system is a bidding engine that receives bid requests from the DSP, processes them through the AI model, and submits bids in real-time.

python
def submit_bid(bid_request, predicted_bid):
  """
  Submits a bid to the DSP.
  """
  bid_price = min(predicted_bid, bid_request.max_bid) # Cap bid at max_bid
  # API call to DSP to submit bid
  dsp_response = dsp_api.