Getting a company's recent 10-Ks from SEC EDGAR without 403s (Python, free)

If you've hit 403 Forbidden from sec.gov, or got an HTML page back when you expected JSON, it's almost always the User-Agent. SEC's fair-access rules want every request to say who you are, with a contact email. Python's default python-requests/x.y gets blocked.

Here's the free way, no API key.

1. Ticker to CIK

import requests, time

HEADERS = {"User-Agent": "YourProject [e-mail]"}

tickers = requests.get("https://www.sec.gov/files/company_tickers.json", headers=HEADERS).json()
cik_by_ticker = {row["ticker"]: str(row["cik_str"]).zfill(10) for row in tickers.values()}
cik = cik_by_ticker["AAPL"]   # "0000320193", zero-padded to 10 digits

2. Recent filings for that CIK

sub = requests.get(f"https://data.sec.gov/submissions/CIK{cik}.json", headers=HEADERS).json()
recent = sub["filings"]["recent"]

rows = []
for i, form in enumerate(recent["form"]):
    if form != "10-K":
        continue
    acc = recent["accessionNumber"][i]
    doc = recent["primaryDocument"][i]
    url = f"https://www.sec.gov/Archives/edgar/data/{int(cik)}/{acc.replace('-', '')}/{doc}"
    rows.append((recent["filingDate"][i], acc, url))

for r in rows:
    print(r)

filings.recent covers the most recent filings (up to about 1,000). For older ones, the same JSON lists extra files under filings.files that you fetch the same way.

3. Stay polite

SEC's limit is 10 requests per second, counted across everything you run. For a list of tickers, sleep between companies:

time.sleep(0.25)

and back off on 429/403 instead of retrying instantly.

4. If you'd rather not run it yourself

Disclosure: we publish an Apify actor that does steps 1-3 for you (Brindlewave Data, "SEC EDGAR Filings and Company Facts"). You give it a ticker or CIK, an optional form filter like ["10-K"], a since-date and your contact email, and it returns one row per filing with the archive links, plus a few recent XBRL values (revenue, assets, net income...) from the companyfacts endpoint. It's pay per result: $0.40 per 1,000 filing rows plus a tiny per-run start charge ($0.00005 per GB of memory), one company per run, and it only reads filings.recent, so for full history the script above with filings.files is the better route.

Link: https://apify.com/hirethomas/sec-edgar-filings-and-company-facts

The script above is all you need for most cases, though. The User-Agent line is the part that fixes the 403.

Comments

[Arhan Canli]

Two things that bit me on top of the User-Agent line, both in step 2. First, form != "10-K" drops 10-K/A (amendments), 10-KT (transition periods) and the older 10-K405 forms, and 10-K/A is where restated numbers often show up, so if the goal is "what did they report", match on form.startswith("10-K") and keep the form column so you can tell original from amended. Second, filingDate is a date, not a moment. The submissions JSON also has acceptanceDateTime, but I found it is off by twice the Eastern offset in the parts I checked: JPMorgan's 06:30 ET filing shows 14:30Z. If you use it to order filings against price data, check it against a filing you know the time of first.

On the free route: I built canli-mcp, which wraps EDGAR (filings, XBRL facts, the 13D/13G XML) plus about 300 other finance tools behind three MCP tools, MIT licensed, no key, runs locally: npx -y canli-mcp, source at github.com/arhancanli/canlicapital.

[Saad | Data APIs]

Good write-up, the User-Agent line really is 90% of the 403s. Two things that bit me on top of that: the 10 req/s limit counts across sec.gov and data.sec.gov together, and once you trip it the block lasts for a while, so an instant retry loop just keeps you blocked. Back off for minutes, not seconds.

Also, if what you want is "new 8-Ks/10-Ks from anyone" rather than per company, the current-filings Atom feed (https://www.sec.gov/cgi-bin/browse-edgar?action=getcurrent&type=8-K&output=atom) or the daily index under /Archives/edgar/daily-index/ saves you from looping over thousands of CIKs.