Why I stopped trading and started tendering with AI

#ai #automation #opensource

Why I Stopped Trading and Started Tendering with AI

For years, I chased the dragon of algorithmic trading. Countless hours poured into backtesting, optimizing, and tweaking strategies, fueled by the dream of automated profits. I experimented with everything - mean reversion, momentum, statistical arbitrage, even dabbling in deep learning for price prediction. While I had some success – a profitable strategy here, a lucky trade there – it always felt like fighting a losing battle against the market’s inherent noise and the speed of institutional players.

Then, I had a realization: I was optimizing for reaction when I should have been optimizing for anticipation. Trading is playing catch-up. Tendering – proactively identifying and bidding on opportunities before they’re widely known – is getting ahead of the curve. And AI, specifically Large Language Models (LLMs), turned out to be the key.

This isn't about predicting the stock market. It’s about predicting opportunities. Opportunities for supply, for niche requirements, for projects that haven't even been formally announced yet. Think government contracts, RFPs (Requests for Proposals), specialized material sourcing, even obscure data sets needed for research.

This article outlines my journey from trading algorithms to a system I call “IronVision Nexus,” built on AI to identify and assess tendering opportunities. It’s technical, and I’ll share some of the code (Python, naturally) to demonstrate the core concepts.

The Problem with Trading (for Me)

Let’s be honest. Successful high-frequency trading requires incredibly low latency, proximity to exchanges, and frankly, a level of capital most individuals don't have access to. My strategies, while profitable on paper, were consistently degraded in live markets by slippage, execution delays, and the sheer volume of participants.

The biggest issue, though, wasn’t execution. It was information. I was reacting to public data. Everyone else was too. Trying to extract an edge from already-processed information is a zero-sum game.

I needed a way to access information that wasn't readily available. I needed early signals.

The "Tendering" Paradigm Shift

My background isn't just in finance. I've also worked with procurement and supply chain management. I started noticing the vast amount of publicly available information regarding upcoming projects – government websites, industry news, legal notices, even obscure online forums. But sifting through it manually is incredibly time-consuming.

This is where LLMs come in. Their ability to understand and extract meaning from unstructured text is unparalleled. Instead of trying to predict price movements, I could use AI to:

Identify potential tender opportunities buried in noise.
Analyze the requirements and assess our suitability.
Prioritize opportunities based on profitability and risk.

Building IronVision Nexus: A Technical Deep Dive

IronVision Nexus is built on a combination of technologies. Here's a simplified breakdown of the core components:

Data Sources: We pull data from a variety of sources:
- Government Tender Portals: Using web scraping libraries like BeautifulSoup and requests (Python).
- Industry News APIs: Accessing APIs from news aggregators and specialized industry publications.
- Legal Notice Databases: Scraping databases of legal notices and public announcements.
- Forum Crawling: Monitoring relevant online forums for early discussions of upcoming projects.
LLM Integration: We primarily use OpenAI's GPT-3.5 Turbo (though experimenting with open-source models like Llama 2) via the openai Python library.
Vector Database: We use Pinecone to store embeddings of tender descriptions and project requirements. This allows for semantic search and similarity matching.
Scoring Engine: A custom scoring engine that calculates a suitability score based on the LLM’s analysis and pre-defined business criteria.

Example: Identifying Potential Opportunities (Python)

import openai
import requests
from bs4 import BeautifulSoup

# Replace with your OpenAI API key
openai.api_key = "YOUR_OPENAI_API_KEY"

def extract_tender_details(url):
    """Scrapes a tender listing URL and extracts key details."""
    try:
        response = requests.get(url)
        response.raise_for_status()  # Raise HTTPError for bad responses (4xx or 5xx)
        soup = BeautifulSoup(response.content, 'html.parser')
        title = soup.find('h1').text.strip() if soup.find('h1') else "No Title Found"
        description = ' '.join([p.text.strip() for p in soup.find_all('p')]) if soup.find_all('p') else "No Description Found"
        return title, description
    except requests.exceptions.RequestException as e:
        print(f"Error fetching URL: {e}")
        return "Error", "Error"

def classify_tender(description):
    """Uses GPT-3.5 to classify a tender description."""
    prompt = f"Classify the following tender description into one of these categories: 'IT Services', 'Construction', 'Supply Chain', 'Consulting', 'Research & Development', 'Other'.\n\nDescription: {description}\n\nClassification:"
    response = openai.ChatCompletion.create(
        model="gpt-3.5-turbo",
        messages=[{"role": "user", "content": prompt}],
        temperature=0.2, # Lower temperature for more deterministic results
        max_tokens=50
    )
    return response['choices'][0]['message']['content'].strip()

# Example Usage
tender_url = "https://example.com/tender-listing" # Replace with an actual URL
title, description = extract_tender_details(tender_url)
classification = classify_tender(description)

print(f"Title: {title}")
print(f"Classification: {classification}")

Explanation:

extract_tender_details() scrapes the content from a given URL using requests and BeautifulSoup. Error handling is included to gracefully handle failed requests.
classify_tender() leverages OpenAI’s GPT-3.5 to categorize the tender based on its description. The prompt is carefully crafted to guide the LLM towards specific categories. temperature=0.2 ensures more predictable classifications.

Assessing Suitability & Scoring

Once a potential opportunity is identified and classified, the real magic happens. We use the LLM to analyze the detailed requirements of the tender. This involves:

Requirement Extraction: Extracting key requirements from the tender document (e.g., "Must have ISO 9001 certification," "Experience in AWS cloud migration," "Deliverables within 6 months").
Capability Matching: Comparing the extracted requirements against our company's documented capabilities

DEV Community

Why I stopped trading and started tendering with AI