DEV Community: itsRaxzey

How late is Form 4 'real-time' data? Measuring filing delay on 11,241 insider filings

itsRaxzey — Mon, 24 Aug 2026 01:13:14 +0000

If you backtest anything involving insider trades, there's a quiet bug waiting for you: Form 4 data is timestamped twice. Every filing carries a transaction date (when the insider actually traded) and a filing date (when EDGAR disseminated the form). If your backtest joins on the transaction date, you're trading on information that did not exist yet.

How big is that gap in practice? I measured it for every Form 4 filed in July 2026 — 11,241 filings — and the answer is: usually 2 days, sometimes 8 years.

(Disclosure up front: I run FilingPulse, the SEC filings API this study is built on. This is a methodology piece, not a pitch — the whole thing reproduces on the free tier, and the raw numbers below are what they are regardless of whose API you use.)

Method

Window: all Form 4s with a filed date in 2026-07-01 … 2026-07-31 (11,052 originals; 189 Form 4/A amendments excluded — an amendment's delay measures the correction, not the original report).
For each filing, delay = filed date − earliest transaction date on the form. Holdings-only forms (70) and forms with no dated transaction (31) are excluded; that leaves 10,951 analyzed.
Coverage was verified against EDGAR's own daily indexes for all 22 trading days in the window (22/22 exact match) — so "every Form 4 in July" is a checked claim, not a scrape-and-hope.
One framing rule I held throughout: a long delay is an observed reporting lag, not a compliance violation. Section 16 has deemed-execution-date rules (10b5-1 plans, small acquisitions, etc.) that legitimately extend the clock, and you cannot adjudicate those from the filing alone.

The headline distribution

Calendar days from transaction to dissemination:

Percentile	Delay
p50	2 days
p75	4 days
p90	5 days
p95	7 days
p99	91 days
max	3,140 days (8.6 years)

92.9% of filings landed within the SEC's 2-business-day deadline. The median is well-behaved. The tail is not: 190 filings arrived 11–30 days after the trade, 131 arrived 31–90 days after, and 110 arrived more than 90 days late. That 8.6-year max is a real filing disseminated in July 2026 for a transaction from 2017.

The interesting part: delay depends on what the insider did

Breaking the same numbers out by transaction code:

Code	Meaning	Legs	Within 2 bus. days
S	Open-market sale	6,694	98.2%
D	Disposition to issuer	1,163	98.8%
M	Option exercise	4,552	95.4%
A	Grant/award	6,897	92.8%
F	Tax withholding	1,850	93.8%
P	Open-market purchase	1,198	80.9%
J	"Other"	822	79.2%

Sales are filed on time almost universally — they're usually broker-assisted and the paperwork pipeline is well-oiled. Open-market purchases — the transaction type most insider-trading research cares about most — are the least punctual common code: 1 in 5 arrives after the deadline window. If your strategy keys on insider buys specifically, your look-ahead bias is concentrated exactly where it hurts.

What this means for a backtest

Join on filing date, not transaction date. The filing date is when the information became public. Using the transaction date leaks 2+ days of future knowledge on the median filing and months on the tail.
The tail is not ignorable. ~4% of filings arrive more than 5 calendar days late. In a monthly-rebalance backtest those stragglers silently teleport information backwards across rebalance boundaries.
Point-in-time data matters more for purchases than sales. See the table above.

Reproduce it

The full write-up with charts is here: How late is Form 4 data? — and the study script is a single Python file that runs against a free API key (the since=/until= parameters give you a fixed, reproducible population):

params = {"since": "2026-07-01", "until": "2026-07-31", "limit": 100}
# page through /v1/insider-trades, then for each filing:
# delay = filed_date - min(leg["transaction_date"] for leg in legs)

Script + methodology notes are linked from the study page. If you rerun it for a different month I'd genuinely like to see whether the purchase-vs-sale punctuality gap holds — it's the one result I didn't expect.

8-K corporate events hide in SGML headers: how to classify them

itsRaxzey — Mon, 10 Aug 2026 13:36:04 +0000

If you want to know the moment a public company reports earnings, fires a CEO, or signs a merger, the source of truth is a Form 8-K on SEC EDGAR. There's a catch that surprises everyone who tries to consume 8-Ks programmatically: the thing you actually want — which "items" the 8-K reports — is not in the document. It's in the filing's SGML header.

I run FilingPulse, an API that normalizes EDGAR filings, so I've had to make this robust across every 8-K the SEC has accepted. Here's what the problem looks like and how the classifier works.

What an 8-K "item" is

8-K disclosures are keyed to a fixed list of item numbers. A few you'll recognize:

2.02 — Results of Operations (earnings)
5.02 — Departure/Election of Directors or Officers
1.01 — Entry into a Material Definitive Agreement
7.01 — Regulation FD Disclosure
8.01 — Other Events

One filing can carry several items at once. A single 8-K might be [2.02, 7.01, 9.01] — an earnings release, a Reg-FD statement, and the exhibits index. So the useful signal is which item codes are present, structured, not prose.

The trap: the body doesn't reliably contain the codes

The primary 8-K document is HTML meant for humans. Sometimes it lists "Item 2.02" as a heading; sometimes it just says "Results of Operations and Financial Condition"; sometimes the whole thing is a press release attached as an exhibit and the body barely mentions the item at all. Parsing the body for item codes is a losing game.

The fix: the index-headers file

Every filing folder on EDGAR publishes a small header file — the SGML submission header wrapped in HTML — at a predictable URL:

.../Archives/edgar/data/<CIK>/<ACCESSION_NODASH>/<ACCESSION>-index-headers.html

Inside, each reported item shows up as a line like:

ITEM INFORMATION: Results of Operations and Financial Condition
ITEM INFORMATION: Financial Statements and Exhibits

So you never parse the document at all. You pull one small header file, read the ITEM INFORMATION: lines, and map each caption back to its code:

import re, html

def item_codes(header_text):

    caps = re.findall(r"ITEM INFORMATION:\s*([^\r\n<]+)", header_text)

    return [CAPTION_TO_CODE.get(norm(html.unescape(c))) for c in caps]

Two quirks that will bite you

Building the caption-to-code map from the SEC's official item list almost works. Two things break it against real filings:

1. EDGAR's captions don't always match the form instructions. The official text for item 3.03 is "Material Modification to Rights of Security Holders" (singular). A large share of real headers say "Material Modifications" (plural). Match captions exactly and you silently drop 3.03 events. You need an alias table built from what EDGAR actually emits, not from the rulebook.

2. Captions are HTML-escaped. Apostrophes arrive as ', ampersands as &. "Changes in Registrant's Certifying Accountant" won't match unless you unescape first.

I found both the only way you ever really find these: by classifying a live feed and watching which items came back as unmapped.

Why headers, not full-text search

People reach for EDGAR full-text search or scraping the document because it's the obvious path. The header approach wins on three axes: it's one small request per filing instead of downloading a multi-MB exhibit bundle; it's structured (the SEC already classified it at filing time); and it's complete (every item the filer declared is there, even the ones the body forgets to mention).

If you just want the JSON

This is one endpoint in FilingPulse — /v1/events?item=2.02 returns every earnings 8-K, newest first, as normalized JSON, and webhooks push new ones within minutes of filing. Free tier, no card. But honestly the header trick above is the whole secret and you can run it against EDGAR yourself in an afternoon — the fair-access rules just want a declared User-Agent and a sane request rate.

It's data infrastructure, not investment advice — it reports what was filed, nothing else. Happy to answer EDGAR questions in the comments.

Parsing SEC EDGAR's Form 4 feed: the edge cases nobody warns you about

itsRaxzey — Sat, 01 Aug 2026 18:46:14 +0000

Every US public-company insider trade is public data. When an executive buys or sells their own stock, they have two business days to file a Form 4 with the SEC, and it lands on EDGAR — free, no login, no API key. I spent the last month building a pipeline that ingests every one of them in near-real-time, and the gap between "the data is public" and "the data is usable" turned out to be the entire project.

Here's the field guide I wish I'd had.

1. The feed's form-type filter is a lie (it's a prefix match)

EDGAR's current-filings feed takes a type parameter:

https://www.sec.gov/cgi-bin/browse-edgar?action=getcurrent&type=4&output=atom

You'd expect type=4 to return Form 4s. It returns everything that starts with 4 — including 424B2 prospectuses and 425 merger communications, which have nothing to do with insider trading. You must re-filter on the exact form type parsed from each entry's title. This isn't documented anywhere I could find; you discover it when a bond prospectus shows up in your insider-trades table.

2. Every filing appears twice

The same filing is listed once under the reporting owner's CIK and once under the issuer's CIK — same accession number, different index URLs. If you dedupe on URL, you store everything twice. The accession number (0001628280-26-049369) is the real identity of a filing; treat it as your primary key and make inserts idempotent:

INSERT OR IGNORE INTO filings (accession, ...) VALUES (?, ...)

3. Not every XML in the filing folder is the filing

A Form 4 filing folder contains several files, and the tempting move — "grab the .xml" — breaks quietly, because filing-fee exhibit XMLs live in the same folder. Validate the root element (<ownershipDocument>) before parsing, not the file extension.

4. Some Form 4s contain no transactions at all

Holdings-only filings exist: no transaction table, no holdings table, just remarks and a signature. They're legal, they're rare, and they will null-pointer your parser at 2am. Multi-owner filings also exist — one filing reporting for three retirement-plan trustees simultaneously — so "the reporting owner" is actually a list.

Amendments (form type 4/A) reference the original filing and carry an amendment date. If you don't handle them, your dataset double-counts corrected transactions.

5. The interesting data hides in footnotes

Share counts and prices carry footnote references (<footnoteId id="F1"/>), and the footnote text often changes the meaning of the number — "price is a weighted average of executions between $18.11 and $18.29" — so a parser that drops footnotes is destroying information. Keep them attached to the fields they annotate.

6. Decide your philosophy: values as filed, or values as "fixed"

The biggest design decision in the whole pipeline: numbers in filings are typed by humans at law firms. They contain typos, weird formats, and occasionally nonsense. You can either coerce everything to floats (and silently corrupt the record when coercion guesses wrong) or deliver strings exactly as filed and let consumers coerce at their edge.

I chose strings-as-filed, and it's the choice I'd defend hardest: this is federal filing data. If an insider fat-fingers a share count, you should see what the SEC saw — plus two invariants that make the schema livable: every documented field is always present, and null always means "not stated in the filing."

7. HTML entities are everywhere

Issuer names arrive HTML-escaped in some feed views and not others: Organon & Co. in one place, Organon & Co. in another. Normalize with html.unescape() on ingest or enjoy duplicate-company bugs forever. (The related 8-K feed also HTML-escapes apostrophes in item captions, which breaks caption-matching — same fix.)

8. The 403 that isn't a throttle

The nastiest production incident so far: EDGAR serves 403 Forbidden — not 404 — for daily-index files that were never published (weekends, holidays). A 403 is also exactly what EDGAR's rate limiter serves when you're being throttled. My backfill hit a Sunday index file, interpreted the 403 as throttling, and politely retried the same URL on a backoff loop for 17 hours.

The fix is a canary request: on a 403 for an index file, immediately fetch a URL you know exists. If the canary returns 200, you're not throttled — the index is genuinely absent, and that day can be checkpointed as empty.

9. Be polite — it's cheap and it's the rules

EDGAR's fair-access guidance asks for a declared User-Agent with contact info and a cap of 10 req/s. I run a process-wide rate limiter pinned to 2 req/s with exponential backoff on 403/429 — one enforcement point that every request goes through, so adding a new worker can't accidentally exceed the budget. The entire pipeline — feed polling, two years of backfill, ~90,000 Form 4s and ~30,000 8-Ks so far — runs comfortably inside that on a €5 VPS with SQLite in WAL mode.

The punchline

None of these problems is hard in isolation. The product is surviving all of them at once, continuously, unattended — which is why "the data is free" and "the data is usable" are separated by a parser with a regression corpus of real filings: the amendments, the holdings-only ones, the multi-owner ones, the footnote-heavy ones.

I packaged all of this as FilingPulse — normalized Form 4 + 8-K JSON over REST and HMAC-signed webhooks, free tier with the full schema (2,500 req/mo, email signup, no card). If you'd rather build your own pipeline, honestly: take the edge-case list above, it's yours. That's the part I couldn't Google a month ago.

Standard disclaimer that matters in this domain: this is data infrastructure. Nothing here is investment advice — the API reports what was filed, nothing more.