DEV Community: itsRaxzey

8-K corporate events hide in SGML headers: how to classify them

itsRaxzey — Mon, 10 Aug 2026 13:36:04 +0000

If you want to know the moment a public company reports earnings, fires a CEO, or signs a merger, the source of truth is a Form 8-K on SEC EDGAR. There's a catch that surprises everyone who tries to consume 8-Ks programmatically: the thing you actually want — which "items" the 8-K reports — is not in the document. It's in the filing's SGML header.

I run FilingPulse, an API that normalizes EDGAR filings, so I've had to make this robust across every 8-K the SEC has accepted. Here's what the problem looks like and how the classifier works.

What an 8-K "item" is

8-K disclosures are keyed to a fixed list of item numbers. A few you'll recognize:

2.02 — Results of Operations (earnings)
5.02 — Departure/Election of Directors or Officers
1.01 — Entry into a Material Definitive Agreement
7.01 — Regulation FD Disclosure
8.01 — Other Events

One filing can carry several items at once. A single 8-K might be [2.02, 7.01, 9.01] — an earnings release, a Reg-FD statement, and the exhibits index. So the useful signal is which item codes are present, structured, not prose.

The trap: the body doesn't reliably contain the codes

The primary 8-K document is HTML meant for humans. Sometimes it lists "Item 2.02" as a heading; sometimes it just says "Results of Operations and Financial Condition"; sometimes the whole thing is a press release attached as an exhibit and the body barely mentions the item at all. Parsing the body for item codes is a losing game.

The fix: the index-headers file

Every filing folder on EDGAR publishes a small header file — the SGML submission header wrapped in HTML — at a predictable URL:

.../Archives/edgar/data/<CIK>/<ACCESSION_NODASH>/<ACCESSION>-index-headers.html

Inside, each reported item shows up as a line like:

ITEM INFORMATION: Results of Operations and Financial Condition
ITEM INFORMATION: Financial Statements and Exhibits

So you never parse the document at all. You pull one small header file, read the ITEM INFORMATION: lines, and map each caption back to its code:

import re, html

def item_codes(header_text):

    caps = re.findall(r"ITEM INFORMATION:\s*([^\r\n<]+)", header_text)

    return [CAPTION_TO_CODE.get(norm(html.unescape(c))) for c in caps]

Two quirks that will bite you

Building the caption-to-code map from the SEC's official item list almost works. Two things break it against real filings:

1. EDGAR's captions don't always match the form instructions. The official text for item 3.03 is "Material Modification to Rights of Security Holders" (singular). A large share of real headers say "Material Modifications" (plural). Match captions exactly and you silently drop 3.03 events. You need an alias table built from what EDGAR actually emits, not from the rulebook.

2. Captions are HTML-escaped. Apostrophes arrive as ', ampersands as &. "Changes in Registrant's Certifying Accountant" won't match unless you unescape first.

I found both the only way you ever really find these: by classifying a live feed and watching which items came back as unmapped.

Why headers, not full-text search

People reach for EDGAR full-text search or scraping the document because it's the obvious path. The header approach wins on three axes: it's one small request per filing instead of downloading a multi-MB exhibit bundle; it's structured (the SEC already classified it at filing time); and it's complete (every item the filer declared is there, even the ones the body forgets to mention).

If you just want the JSON

This is one endpoint in FilingPulse — /v1/events?item=2.02 returns every earnings 8-K, newest first, as normalized JSON, and webhooks push new ones within minutes of filing. Free tier, no card. But honestly the header trick above is the whole secret and you can run it against EDGAR yourself in an afternoon — the fair-access rules just want a declared User-Agent and a sane request rate.

It's data infrastructure, not investment advice — it reports what was filed, nothing else. Happy to answer EDGAR questions in the comments.

Parsing SEC EDGAR's Form 4 feed: the edge cases nobody warns you about

itsRaxzey — Sat, 01 Aug 2026 18:46:14 +0000

Every US public-company insider trade is public data. When an executive buys or sells their own stock, they have two business days to file a Form 4 with the SEC, and it lands on EDGAR — free, no login, no API key. I spent the last month building a pipeline that ingests every one of them in near-real-time, and the gap between "the data is public" and "the data is usable" turned out to be the entire project.

Here's the field guide I wish I'd had.

1. The feed's form-type filter is a lie (it's a prefix match)

EDGAR's current-filings feed takes a type parameter:

https://www.sec.gov/cgi-bin/browse-edgar?action=getcurrent&type=4&output=atom

You'd expect type=4 to return Form 4s. It returns everything that starts with 4 — including 424B2 prospectuses and 425 merger communications, which have nothing to do with insider trading. You must re-filter on the exact form type parsed from each entry's title. This isn't documented anywhere I could find; you discover it when a bond prospectus shows up in your insider-trades table.

2. Every filing appears twice

The same filing is listed once under the reporting owner's CIK and once under the issuer's CIK — same accession number, different index URLs. If you dedupe on URL, you store everything twice. The accession number (0001628280-26-049369) is the real identity of a filing; treat it as your primary key and make inserts idempotent:

INSERT OR IGNORE INTO filings (accession, ...) VALUES (?, ...)

3. Not every XML in the filing folder is the filing

A Form 4 filing folder contains several files, and the tempting move — "grab the .xml" — breaks quietly, because filing-fee exhibit XMLs live in the same folder. Validate the root element (<ownershipDocument>) before parsing, not the file extension.

4. Some Form 4s contain no transactions at all

Holdings-only filings exist: no transaction table, no holdings table, just remarks and a signature. They're legal, they're rare, and they will null-pointer your parser at 2am. Multi-owner filings also exist — one filing reporting for three retirement-plan trustees simultaneously — so "the reporting owner" is actually a list.

Amendments (form type 4/A) reference the original filing and carry an amendment date. If you don't handle them, your dataset double-counts corrected transactions.

5. The interesting data hides in footnotes

Share counts and prices carry footnote references (<footnoteId id="F1"/>), and the footnote text often changes the meaning of the number — "price is a weighted average of executions between $18.11 and $18.29" — so a parser that drops footnotes is destroying information. Keep them attached to the fields they annotate.

6. Decide your philosophy: values as filed, or values as "fixed"

The biggest design decision in the whole pipeline: numbers in filings are typed by humans at law firms. They contain typos, weird formats, and occasionally nonsense. You can either coerce everything to floats (and silently corrupt the record when coercion guesses wrong) or deliver strings exactly as filed and let consumers coerce at their edge.

I chose strings-as-filed, and it's the choice I'd defend hardest: this is federal filing data. If an insider fat-fingers a share count, you should see what the SEC saw — plus two invariants that make the schema livable: every documented field is always present, and null always means "not stated in the filing."

7. HTML entities are everywhere

Issuer names arrive HTML-escaped in some feed views and not others: Organon & Co. in one place, Organon & Co. in another. Normalize with html.unescape() on ingest or enjoy duplicate-company bugs forever. (The related 8-K feed also HTML-escapes apostrophes in item captions, which breaks caption-matching — same fix.)

8. The 403 that isn't a throttle

The nastiest production incident so far: EDGAR serves 403 Forbidden — not 404 — for daily-index files that were never published (weekends, holidays). A 403 is also exactly what EDGAR's rate limiter serves when you're being throttled. My backfill hit a Sunday index file, interpreted the 403 as throttling, and politely retried the same URL on a backoff loop for 17 hours.

The fix is a canary request: on a 403 for an index file, immediately fetch a URL you know exists. If the canary returns 200, you're not throttled — the index is genuinely absent, and that day can be checkpointed as empty.

9. Be polite — it's cheap and it's the rules

EDGAR's fair-access guidance asks for a declared User-Agent with contact info and a cap of 10 req/s. I run a process-wide rate limiter pinned to 2 req/s with exponential backoff on 403/429 — one enforcement point that every request goes through, so adding a new worker can't accidentally exceed the budget. The entire pipeline — feed polling, two years of backfill, ~90,000 Form 4s and ~30,000 8-Ks so far — runs comfortably inside that on a €5 VPS with SQLite in WAL mode.

The punchline

None of these problems is hard in isolation. The product is surviving all of them at once, continuously, unattended — which is why "the data is free" and "the data is usable" are separated by a parser with a regression corpus of real filings: the amendments, the holdings-only ones, the multi-owner ones, the footnote-heavy ones.

I packaged all of this as FilingPulse — normalized Form 4 + 8-K JSON over REST and HMAC-signed webhooks, free tier with the full schema (2,500 req/mo, email signup, no card). If you'd rather build your own pipeline, honestly: take the edge-case list above, it's yours. That's the part I couldn't Google a month ago.

Standard disclaimer that matters in this domain: this is data infrastructure. Nothing here is investment advice — the API reports what was filed, nothing more.