Quick answer
TED — the EU's official public-procurement journal — publishes every tender notice across the bloc through a free, keyless API that answers a GET with HTTP 405. It is POST-only, which reads like a dead endpoint and is not. And once you are past that, its multilingual fields do not all have the same shape, which is how our first build produced a dataset where every tender's title was the single letter F. The EU TED Tenders Scraper handles both, at $4.20 per 1,000 notices.
Trap one: the 405 that looks like a 404 🚪
GET https://api.ted.europa.eu/v3/notices/search
-> 405 {"message":"Request method 'GET' is not supported","error":null}
Search is a POST with a JSON body carrying the expert query, the page, the limit and the field list. Nothing about a 405 on a documented search URL suggests "you used the wrong verb" to someone scanning statuses in bulk — it sits in the same mental bucket as 404 and 410, the ones that mean stop, this is gone. We very nearly filed TED as retired on exactly that reading during the morning probe.
Trap two: the field that silently returns its first character 🔤
This is the one worth the article.
TED's fields are multilingual — each returns an object keyed by language code. But the value type is not consistent across fields:
"buyer-name": { "eng": ["DB Station&Service AG"] } // dict[str, list[str]]
"buyer-city": { "eng": ["Berlin"] } // dict[str, list[str]]
"description-lot":{ "eng": ["..."] } // dict[str, list[str]]
"winner-name": { "eng": ["..."] } // dict[str, list[str]]
"notice-title": { "eng": "Germany-Berlin: IT services" } // dict[str, str] <--!
Four fields wrap their value in a list. The title does not.
Our first implementation wrote one helper for "multilingual field", picked the language, and did values[0]. On a list that yields the string. On a string it yields the first character. So the dataset came out with titles of "F", "G", "A" — not empty, not an error, just wrong in a way that looks like a truncation bug somewhere else entirely.
The part that matters: all 62 tests passed. They were written from the same misreading of the wire contract as the code, so the fixtures encoded notice-title as a list too, and the suite happily confirmed the code did what the code did. A test written from a misread API cannot detect a misread API. Only re-probing the live endpoint caught it, and the fix now handles both shapes per key, with regression tests built from a captured live response rather than a hand-written one.
There is a smaller sibling trap in the same helper: a present-but-empty key. {"eng": [], "pol": ["Tytuł"]} must fall through to the Polish value rather than returning empty because English "existed". Language preference is English, then mul, then the first key that actually has content.
What you get, and what paging costs 💶
Each row carries the publication number, title, buyer name and country, CPV classification codes, publication date and the notice's document links — filterable by CPV code, buyer country and notice type, which is how a bid team narrows the EU's entire procurement firehose to the handful of notices they can actually bid on.
Underneath, the ordinary defences a paid Actor needs against a public endpoint: 429 and 5xx retried with capped exponential backoff honouring Retry-After, a malformed notice validated, logged and skipped rather than sinking its page, and a filter that legitimately matches nothing finishing as a clean successful run instead of a false failure.
Is TED hard to scrape? 🛡️
Not in the anti-bot sense — it is a public EU API with no challenge page, and we probed it before writing a client. The difficulty is that its two traps both produce plausible output: a status code that means "wrong verb" but reads as "gone", and a type inconsistency that yields a one-character string instead of an error. Neither one raises. Both ship.
FAQ
What is TED?
Tenders Electronic Daily — the EU's official journal of public procurement notices, covering member states plus EEA.
Can I filter by sector?
Yes, by CPV code — the EU's procurement classification vocabulary — plus buyer country and notice type.
Does it cover contract awards as well as calls for tender?
Notice type is a filter, so you can pull calls, awards, or both.
How is it billed?
$0.20 per run start plus $0.004 per notice row, so 1,000 notices cost $4.20.
Built by Devil Scrapes. We publish the traps we hit, because the ones that return 200 OK are the expensive ones.
Top comments (0)