Why I stopped trading and started tendering with AI

Why I Stopped Trading and Started Tendering with AI

For years, I chased the dragon of algorithmic trading. I devoured books on technical analysis, backtested countless strategies, and spent late nights optimizing code, all in pursuit of that elusive, consistent profit. I dabbled in everything: momentum trading, mean reversion, arbitrage, even sentiment analysis. And for a while, it worked. Small wins, occasional bigger ones. But ultimately, it felt like I was perpetually running on a treadmill, constantly fighting market noise and the inherent randomness of the system.

The truth is, trading, even with sophisticated algorithms, is fundamentally a zero-sum game. One person’s gain is another’s loss. It’s a high-friction environment filled with predatory behavior, flash crashes, and a constant need to anticipate the irrational. I began to feel less like an innovator and more like a sophisticated gambler.

Then, I stumbled upon a different path: AI-powered tendering. Specifically, using AI to identify and win government and corporate tenders – requests for proposals (RFPs). It's a market fundamentally different from trading. Instead of taking something from someone, you're providing a solution to a clearly defined need. It's a positive-sum game, and the margins, when you win, are significantly higher.

This wasn't about predicting price movements; it was about understanding language, analyzing requirements, and creating value. And AI, surprisingly, is exceptionally good at that. This article details my journey, the tools I built, and the shift in mindset from a trader to a tendering specialist leveraging the power of AI.

The Problem with Traditional Tendering

Traditionally, finding and responding to tenders is a hugely manual process. You have to:

• Monitor multiple platforms: GovWin, BidNet, state/local procurement websites, industry-specific portals – the list is endless.
• Sift through hundreds of RFPs: Most are irrelevant, poorly written, or already awarded.
• Analyze complex documents: Understanding the requirements, scoring criteria, and compliance rules is time-consuming and prone to error.
• Craft compelling responses: This requires deep understanding of the client's needs and tailoring your offering accordingly.

The sheer volume of data makes it nearly impossible for small to medium-sized businesses (SMBs) to compete effectively against larger, well-resourced firms. They often miss opportunities or submit rushed, incomplete responses.

The AI Solution: IronVision Nexus

I decided to build a system, IronVision Nexus, to automate and optimize this process. It’s built on a foundation of Python, utilizing libraries like requests, BeautifulSoup4 for web scraping, spaCy and transformers for Natural Language Processing (NLP), and a vector database (ChromaDB in my case) for semantic search.

Here's a breakdown of the key components:

1. RFP Scraping & Aggregation:

The first step is to gather RFPs from various sources. This involves writing web scrapers to extract relevant information like title, description, closing date, and issuing organization.

import requests
from bs4 import BeautifulSoup

def scrape_govwin(url):
    response = requests.get(url)
    soup = BeautifulSoup(response.content, 'html.parser')
    # Extract relevant data using soup.find() and soup.find_all() 
    # (Implementation details omitted for brevity - GovWin's structure changes frequently)
    return rfp_data

# Example usage (assuming you have a list of GovWin URLs)
# rfps = [scrape_govwin(url) for url in govwin_urls]

This is arguably the most fragile part of the system, as website structures change frequently. Maintaining the scrapers is an ongoing task.

2. NLP-Powered Opportunity Filtering:

Once the RFPs are collected, I use NLP to filter out irrelevant opportunities. This is where spaCy comes in.

import spacy

nlp = spacy.load("en_core_web_lg")

def filter_rfps(rfps, keywords):
    filtered_rfps = []
    for rfp in rfps:
        doc = nlp(rfp['description'])
        # Check for keyword presence in the description
        if any(keyword.lower() in doc.text.lower() for keyword in keywords):
            filtered_rfps.append(rfp)
    return filtered_rfps

# Example usage
keywords = ["artificial intelligence", "machine learning", "data analysis"]
relevant_rfps = filter_rfps(rfps, keywords)

This snippet demonstrates a simple keyword-based filtering approach. However, the power comes in leveraging semantic similarity.

3. Semantic Search & Requirement Analysis:

This is where the transformers library (specifically, models like Sentence Transformers) shines. I embed the RFP descriptions into vector representations using a pre-trained model. Then, I can query the vector database (ChromaDB) with a specific skill or service to find relevant tenders.

from sentence_transformers import SentenceTransformer
import chromadb

# Load a pre-trained Sentence Transformer model
model = SentenceTransformer('all-mpnet-base-v2')

# Embed the RFP descriptions
rfp_embeddings = model.encode([rfp['description'] for rfp in relevant_rfps])

# Create a ChromaDB collection
client = chromadb.Client()
collection = client.create_collection("rfps")

# Add the embeddings and metadata to the collection
collection.add(
    embeddings=rfp_embeddings,
    metadatas=[{"rfp_id": rfp['id']} for rfp in relevant_rfps] # Replace 'id' with your actual rfp identifier
)

# Perform a semantic search
query = "Software development using Python and Django"
query_embedding = model.encode(query)
results = collection.query(
    query_embeddings=[query_embedding],
    n_results=5 # Retrieve top 5 results
)

print(results)

This allows me to find tenders even if they don’t explicitly mention the keywords I’m looking for. For example, searching for "Python web development" will surface RFPs that ask for "a web application built with a modern framework" even without explicitly mentioning Python.

4. Response Generation (In Progress):

The ultimate goal is to automate the response generation process. Currently, I’m using a large language model (LLM) – a fine-tuned version of Llama 2 – to draft initial responses based on the RFP requirements and a knowledge base of my company’s capabilities. This is the most complex component and still under development. The challenge is ensuring accuracy, compliance, and a compelling narrative.

From Trading to Tendering: A World of Difference

The shift from trading to tendering has been transformative. Here