DEV Community

Cover image for How I Built a System to Track Insider Trading Data with Python
Huzaifa Zahoor
Huzaifa Zahoor

Posted on

How I Built a System to Track Insider Trading Data with Python

#python #web3 #webdev #ai

Learn how to track insider trading data with Python. See how executives buy and sell their own stock and use it to make smarter investment decisions.

Every quarter, executives at public companies file reports with the SEC when they buy or sell their own stock. This is called insider trading data — and it is 100% legal and public.

When a CEO buys $2 million worth of their own company's stock, that is a signal worth paying attention to. They know things the average investor does not. Tracking this data gives you a real edge.

In this article, we will walk through how we built a Python system that pulls insider trading data every day, cleans it, and stores it in a database — automatically.

What Is Insider Trading Data?

Insider trading data refers to the buying and selling activity of company insiders. This includes CEOs, CFOs, board members, and major shareholders. In the US, they are required by law to report these transactions to the SEC within two business days.

These filings are public. Anyone can access them. But reading raw SEC filings is painful. That is why we built a system to pull, clean, and store this data automatically using Python.

Key terms to know:

  • Acquisition (A): The insider bought shares
  • Disposition (D): The insider sold shares
  • Form 4: The SEC form used to report insider transactions
  • Reporting CIK: The unique identifier for the person filing

Source: SEC EDGAR Form 4 Filings

Why Insider Trading Data Matters for Investors

Insiders have the best knowledge of their own company. When they buy shares with their own money, it often signals confidence in the company's future. When they sell in large volumes, it can be an early warning sign.

Here are some facts worth knowing:

  • Studies show stocks with heavy insider buying outperform the market by 6-8% annually on average
  • Form 4 filings must be submitted within 2 business days of the transaction
  • The SEC makes all filings publicly available on EDGAR
  • Tracking insider transactions is completely legal for retail investors

Source: Journal of Financial Economics - Insider Trading Returns

The problem is most retail investors never look at this data. It is buried in complex SEC documents. That is exactly the gap we wanted to fill.

The System We Built

We built a Python script that runs daily and does four things:

  1. Loads all valid stock symbols from our database
  2. Fetches insider trading data from the FMP API for the last 2 days
  3. Cleans and deduplicates the data
  4. Saves everything to PostgreSQL with no duplicates

Let us walk through each part.

Step 1: Load Valid Stock Symbols

Before fetching any data, we load all stock symbols from our database. This lets us filter out transactions for stocks we do not track.

with DBConnection() as connection:
    with connection.cursor() as cursor:
        symbols = get_stock_symbols(cursor, only_symbols=True)

valid_symbols = set(symbols)
logging.info(f"Loaded {len(valid_symbols)} valid symbols")
Enter fullscreen mode Exit fullscreen mode

We use a set for O(1) lookup speed. When processing thousands of trades, this matters.

Step 2: Fetch Insider Trading Data From the API

We use the Financial Modeling Prep (FMP) API to pull insider trading data. It gives us clean, structured data from SEC Form 4 filings without having to parse raw XML ourselves.

def fetch_insider_trading(date, page=0, limit=1000):
    url = "https://financialmodelingprep.com/stable/insider-trading/latest"
    params = {
        "date": date,
        "page": page,
        "limit": limit,
        "apikey": os.getenv("FMP_API_KEY"),
    }
    return fetch_api_data(url, params=params, default=[])
Enter fullscreen mode Exit fullscreen mode

We fetch up to 1000 records per page and loop through all pages until we get an empty response. This handles days with high filing volume.

def fetch_all_pages(date):
    all_trades = []
    for page in range(0, 101):  # Max 100 pages safety limit
        trades = fetch_insider_trading(date, page, limit=1000)
        if not trades:
            break
        all_trades.extend(trades)
        if len(trades) < 1000:  # Last page
            break
    return all_trades
Enter fullscreen mode Exit fullscreen mode

We also skip weekends automatically since the SEC does not process filings on weekends.

while current_date <= end_date:
    if current_date.weekday() >= 5:  # Skip Saturday and Sunday
        current_date += timedelta(days=1)
        continue
Enter fullscreen mode Exit fullscreen mode

Step 3: Clean and Deduplicate the Data

Raw SEC data is messy. The same transaction can appear multiple times due to amended filings. We handle this with a strict deduplication system.

def process_insider_data(trades, valid_symbols):
    seen_combinations = set()

    for trade in trades:
        symbol = trade.get("symbol")

        # Skip if not in our database
        if symbol not in valid_symbols:
            continue

        # Skip incomplete records
        if not all([transaction_date, reporting_cik,
                    transaction_type, acquisition_or_disposition]):
            continue

        # Build a unique key from all fields
        unique_key = (
            symbol, filing_date, transaction_date,
            reporting_cik, transaction_type,
            securities_transacted, price, ...
        )

        if unique_key in seen_combinations:
            continue  # Already processed

        seen_combinations.add(unique_key)

    return list(seen_combinations)
Enter fullscreen mode Exit fullscreen mode

The unique key includes every meaningful field. This ensures we never store the same transaction twice, even if the API returns it on multiple pages.

Step 4: Save to PostgreSQL With No Duplicates

We use PostgreSQL's ON CONFLICT DO NOTHING to handle duplicates at the database level as well. This is a second layer of protection.

INSERT INTO insider_transaction (
    symbol, "filingDate", "transactionDate", "reportingCik",
    "transactionType", "securitiesTransacted", price, ...
)
VALUES %s
ON CONFLICT (
    symbol, "filingDate", "transactionDate", "reportingCik", ...
)
DO NOTHING;
Enter fullscreen mode Exit fullscreen mode

This means even if the script runs twice on the same day, the database stays clean. No duplicates. No errors. Just reliable data.

What the Data Looks Like

Each insider transaction record contains the following fields:

Field Description
symbol Stock ticker (e.g. AAPL)
reportingName Name of the insider
typeOfOwner CEO, CFO, Director, etc.
transactionType Type of transaction
acquisitionOrDisposition A (bought) or D (sold)
securitiesTransacted Number of shares
price Price per share
securitiesOwned Total shares owned after trade
filingDate When the SEC filing was submitted
transactionDate When the actual trade happened

How to Use This Data

Once you have insider trading data flowing into your database, here is how to use it:

Look for cluster buying. When multiple insiders at the same company buy shares in the same week, that is a strong signal. One person buying could be personal. Five people buying is a pattern.

Focus on open market purchases. Insiders receive stock as compensation all the time. That is not a signal. What matters is when they go to the open market and buy shares with their own cash.

Watch the size of the transaction. A CEO buying $10,000 worth of stock is noise. A CEO buying $2 million is a signal worth investigating.

Combine it with other data. Insider buying is most powerful when combined with strong fundamentals, increasing institutional ownership, and positive analyst ratings. Alone it is useful. Combined it is powerful.

The API We Used

We use the Financial Modeling Prep (FMP) API for all our financial data including insider transactions, institutional holdings, analyst ratings, and more. It is one of the most complete financial data APIs available for developers.

If you want to build something similar, FMP has a discount available. You can get 34% off using the coupon code huzaifa at signup.

The Premium plan at $39.20/month (billed annually) covers everything you need including real-time data, insider transactions, and up to 750 API calls per minute.

Sign up here: financialmodelingprep.com/pricing-plans?couponCode=huzaifa

Want This Built for Your Platform?

We built this system as part of a larger stock market platform. It runs daily, processes thousands of insider transactions, and feeds clean data into a searchable API.

If you run a finance app, trading platform, or stock research tool and want something like this built, feel free to reach out.

Also Check Out Meyka

If you are building stock research tools or just want better data for your own trading, check out Meyka. It is an AI-powered stock research platform that combines alternative data, insider transactions, institutional holdings, and smart analysis in one place.

It gives retail investors and developers access to the kind of research tools that used to be available only to hedge funds.

Check it out at meyka.com

Frequently Asked Questions

Is tracking insider trading data legal?
Yes. SEC Form 4 filings are public records. Anyone can access and use this data. It is completely legal to track and act on it.

How often do insiders file with the SEC?
They must file within 2 business days of the transaction. So the data is almost real time.

What is the best Python library for pulling SEC data?
You can use the SEC EDGAR API directly or use a financial data API like FMP which structures the data for you. FMP is much faster to work with.

How do I know if an insider transaction is a buy or sell?
Look at the acquisitionOrDisposition field. A means the insider acquired shares. D means they disposed of shares.

What is the difference between direct and indirect ownership?
Direct ownership means the insider personally owns the shares. Indirect ownership means shares are held through a trust, family member, or company they control.

Can I use insider trading data Python scripts in production?
Yes. The script we shared is production-ready. It handles pagination, deduplication, weekend skipping, and error alerting out of the box.

Found this useful? Share it with a developer or trader who would benefit. And if you want a custom data pipeline built for your platform, reach out on LinkedIn.

Top comments (0)