Spain publishes every commercial-registry act — incorporations, director changes, insolvencies — in an official gazette called BORME (Boletín Oficial del Registro Mercantil). Most tools we found scrape its PDFs. It turns out you don't need to: since 2009 there is an official open-data API on boe.es that almost nobody seems to use. We just finished backfilling all of it — 9.5 million company events, 2009 → today — and these are the notes we wish we'd had at the start.
The API nobody talks about
- Daily summary:
https://www.boe.es/datosabiertos/api/borme/sumario/{YYYYMMDD}withAccept: application/json→ lists ~50 provincial XML files per business day, each with a directurl_xml. No auth, no keys, no rate-limit headers. - The XML is clean, structured, and 1:1 with the PDFs. If you are parsing BORME PDFs today: stop. The XML makes an entire class of problems (page breaks mid-record!) disappear.
Six traps that cost us real time
-
The 404 lies about its content type. Non-publication days (weekends, holidays) return 404 — with an XML body, even when you asked for
application/json. Check the status code before parsing. -
Missing documents return HTTP 200. A province that didn't publish gives you
200 OKwith<error><descripcion>No se encontró el documento original</descripcion></error>inside. If you only check status codes, you will cache garbage. -
The trailing dot is a minefield.
312077 - DAMI DELUSION S.L.— is the final dot a sentence terminator or part of "S.L."? A naiverstrip('.')corrupts thousands of company names. -
One paragraph = a chain of acts. A single entry routinely packs 3–6 acts:
Ceses/Dimisiones. Adm. Unico: X. Nombramientos. Liquidador: X. Disolución. Extinción. Datos registrales…You need an ordered-vocabulary scanner, not a per-line regex. -
Provincial dialects are real. Barcelona SHOUTS IN UPPERCASE, cooperatives have a
Consejo Rectorinstead of directors, associations aJunta Directiva, and province names carry accents that never match what users type (ARABA/ÁLAVA). -
Dates arrive in at least five formats (
26.06.26,4.05.09,15/11/16,2-10-2009,21 DE FEBRERO DE 2006) — and pre-euro records still quotePtas.
What worked
A deterministic parser over a closed vocabulary — no LLM in the hot path. The gazette's shorthand is a regular language over ~50 act-type labels and ~80 role labels. A scanner plus per-act regexes parses the entire 17-year corpus in minutes and is reproducible byte-for-byte. (LLMs were still useful — for drafting test fixtures.)
Measure what you didn't parse. The single best QA metric we found: consumed characters ÷ total characters, per file. Anything unconsumed ships in the output verbatim as unparsed_spans, so format drift shows up as a number going down — not as silent data loss. Long-run average: 99.9%.
Politeness scales fine. ≤1 request/second with an identifying User-Agent and retries with backoff. The full 2009→today backfill is ~160k requests — about two days of wall-clock, fully resumable (idempotent by file, month checkpoints).
Capacity-planning numbers: ~2,000–2,500 events per business day currently; ~6 GB of raw XML for the full history; 9.5M parsed events across 3.2M companies in Postgres.
The subtle part: three format epochs
The XML layout differs subtly between 2009, 2015 and 2026. Two examples we hit in production: a handful of files carry a number-less articulo holding an erratum note (- or - COMPANY NAME before a "Fe de erratas" / "Corrección de errores" paragraph) — reject the pair, not the file; and one 2024 day shipped registral coordinates with the T / F / S markers missing entirely. A strict parser that refuses to guess, plus a metric for what it skipped, catches every one of these as a number — 17 years of gazette yielded only a handful of such cases. The format is remarkably stable; BOE deserves more credit for it.
If you'd rather not build all this yourself: we packaged the parsed feed as a typed English JSON dataset — 52 act-type enums, officer roles with names, registry coordinates — with a REST API, webhooks and an MCP server for AI agents. The Apify actor is $1 per 1,000 events.
Everything above applies if you build your own instead — the source is official, open, and genuinely pleasant to work with once you know the traps.
Top comments (2)
Great find. It's easy to assume scraping is the only option, but checking for an official API first can save a lot of maintenance and legal headaches. Open data APIs like this are a win for both developers and data quality—more reliable, easier to integrate, and built for long-term use. Thanks for highlighting this resource.
The consumed-chars over total-chars metric is a smart coverage check; most parsers never know what they silently dropped. The 200-with-error-body case is the nastiest, since status-only caching poisons your store. Deterministic parse over a closed vocab is the right call.