DEV Community

Estrecho IA
Estrecho IA

Posted on

Parsing SEC ownership filings (Form 4 / 13F) is awful. Here is what we learned building a normalized API for it.

#webdev #api #data

If you have ever needed insider-trade or institutional-holdings data, you have
probably met EDGAR. It is the SEC's public filing system, it is free, and it
contains everything — which is exactly the problem. The raw surface is built
for compliance, not for software.

This post is a short field guide to what we learned while building a small API
that normalizes SEC ownership filings. Most of it is general knowledge you can
use even if you never touch our tool.

What Form 4 and 13F actually are

  • Form 4 is filed by insiders (officers, directors, >10% owners) when they buy or sell company stock. Each filing has the insider, the transaction type, share counts, prices, and ownership after the trade.
  • 13F-HR is filed quarterly by institutional investment managers with >$100M in AUM. It lists their equity holdings. This is how people approximate what funds like Berkshire hold.

Both are "ownership filings", but they are structurally different documents.

The pain points we hit (and that anyone will hit)

  1. The official API is rate-limited, has no CORS, and no notifications. data.sec.gov is deliberately conservative about automated access, and there is no webhook when a new filing lands. Engineers have resorted to reverse-engineering an undocumented full-text search endpoint just to find filings. (Here is a write-up of that: dev.to.)
  2. 13F parsing is genuinely messy. The raw text layouts vary; open-source parsers track their own failure rate as an issue — edgartools#476 documents improving 13F-HR parsing from ~80% to ~95%+. That last 5% is where the real-world filings live.
  3. Normalization is the whole job. The same economic event can appear in different forms, different units, different spellings of insider names. Consumers want one stable JSON shape, not a tree of XBRL tags.

What we built (and its honest limits)

We are building EdgarFeed, a small API that serves normalized ownership
filing data as clean JSON. It is a beta:

  • The live demo serves real SEC EDGAR filing metadata — currently a sample of 23 major US tickers, the last ~120 days, metadata-level records (filing, form, date, accession, source URL). It is not full market coverage and not yet parsed line items.
  • Webhooks, full history, bulk export, and normalized transaction line items are planned for launch, not shipped today.
  • Pricing is proposed ($0 / $29 / $79) and unverified against the market.
  • The official SEC API stays free; we are selling the developer experience and reliability, not the data.

Try the live demo with real data: https://edgarfeed.onrender.com

What we learned that applies to any EDGAR project

  • Start from the bulk files / official APIs, not scraped HTML — SEC fair access rules matter, and a descriptive User-Agent with a real contact is required.
  • Plan for incremental ingestion with idempotency: accession numbers are the natural dedup key.
  • Assume variation: file formats drift over time; keep a coverage/parser health metric and be public about it.
  • If you only need metadata, the submissions API gets you a long way without touching XBRL at all.

We are happy to answer questions about the approach. Feedback on the API shape
is welcome — that is exactly what the beta is for.

Top comments (0)