Spain's BOE runs a public auction portal. Every day it lists court-ordered repossessions, tax-agency seizures and notarial sales: flats, garages, warehouses, plots, vehicles. Each listing states an auction value, a minimum bid, a deposit, whether the property is occupied and what charges survive the sale. All of it is public, and none of it is designed to be read by someone deciding whether to bid.
I built Subastas BOE around it: a Spanish-language site that turns those listings into a searchable catalogue. The hard part was not the frontend, it was making a government HTML portal behave like a dataset.
See it in Spanish: subastas.devetty.es has a filterable search, one page per province, per municipality and per asset type, and a guide to bidding.
Two pages per auction
The portal exposes a search page and a detail page. The search page returns result blocks; each block has a title, an authority, a file number and a link to the auction. The record itself is split across tabs, and each tab is a separate request with a different ver parameter.
So the scrape is one list request with pagination, then two detail requests per auction — one for the auction terms, one for the asset. The list config is short:
"list": {
"selector": "li.resultado-busqueda",
"dedupeBy": "id",
"required": ["id", "url_ficha"],
"fields": {
"id": {
"selector": "h3",
"transform": ["collapse", { "regex": "(SUB-[A-Z0-9-]+)" }]
}
}
}
The identifier is the only thing I trust from the list. It looks like SUB-JA-2026-..., and it is what the detail URL needs:
https://subastas.boe.es/detalleSubasta.php?idSub={id}&ver=1
https://subastas.boe.es/detalleSubasta.php?idSub={id}&ver=3
Pagination follows a link labelled siguiente, capped at 30 pages and paced 1.5–2.5 seconds apart. Detail requests run one at a time with a 2–4 second delay, and the whole thing is a single daily run, not a tight loop. This is a public service. The cost of being polite is a slower crawl; the cost of not being polite is being blocked, deservedly.
Parsing with configuration instead of a parser
I did not write selectors in code. The scraper takes a JSON config where each field declares a CSS selector and a chain of transforms. Extracting the minimum bid becomes a line in a file, not a function to debug:
"puja_minima": {
"selector": "th:contains('Puja mínima') + td",
"transform": ["collapse", "number"]
}
The portal renders label/value tables, so th:contains(...) + td does most of the work. Dates are the exception: the conclusion date is human text with an ISO timestamp inside parentheses, so I regex the ISO: part for fecha_fin and keep the display text separately.
The province problem is a slug problem
Asset details carry a province in the source spelling, and Spain's provinces are officially bilingual in several regions. The portal is inconsistent: Araba/Álava, Bizkaia, Gipuzkoa, Illes Balears, Valencia/València, Palmas (Las). Slug those directly and you get three URLs for one place, plus a hub page that splits its own inventory.
I resolve it at ingestion, as a chain:
"provincia_es": { "op": "clean_regex", "from": "provincia", "pattern": "/^Araba\\s*\\/\\s*/u" },
"provincia_slug_base": { "op": "clean_regex", "from": "provincia_es", "pattern": "/\\s*\\/.*$/u" },
"provincia_alias": { "op": "map", "from": "provincia_slug_base",
"map": { "Bizkaia": "Vizcaya", "Gipuzkoa": "Guipúzcoa", "Illes Balears": "Baleares" } },
"provincia_canon": { "op": "coalesce", "fields": ["provincia_alias", "provincia_slug_base"] },
"provincia_slug": { "op": "slug", "from": "provincia_canon" }
Two regexes and one alias map reduce every variant to one canonical name and one slug. Araba/Álava becomes alava, Valencia/València becomes valencia. The display name stays in provincia; only the URL is canonicalised.
That single decision is why the site can cross-link. The same slug works on my sister sites, so an auction in madrid can point at public tenders in Madrid, subsidies in Madrid and civil-service exams in Madrid without a per-site translation table.
Money, occupancy and charges
Four numbers decide whether an auction is worth reading: auction value, appraisal, minimum bid and deposit. All four are labels in the same table, so the config treats them identically — collapse, then parse as a number. Spanish formatting arrives as text and is rebuilt for display:
"deposito_num_texto": {
"op": "number_format", "from": "deposito",
"decimals": 2, "decimalSeparator": ",", "thousandsSeparator": ".", "suffix": " €"
},
"deposito_texto": { "op": "coalesce", "fields": ["deposito_num_texto", "no_consta_texto"] }
Missing values get an explicit No consta instead of an empty cell. "We do not know" and "zero" are different facts, and a page that silently renders nothing invites the reader to assume the wrong one.
Occupancy and charges are worse than missing: they are prose. The situación posesoria field might say the property is free of occupants, or rented, or burdened by a usufruct. I turn that into booleans with regexes — one for "no occupants", one for any of con ocupantes, ocupado, arrendatario, inquilino, usufructo, posesión de terceros. Charges work the same way: an amount in euros with a non-zero digit means there are charges; no, ninguna, sin cargas or zero means there are none. The raw text is always kept and shown. The booleans exist so search can offer a "only without occupants and without charges" toggle, which is the filter I would want myself.
Expiry as data, not as a cleanup job
Auctions end. I did not write a job to delete closed ones. The collection declares expireField: "fecha_fin" with a two-day grace period, so records leave the live collection when their closing date passes. A detail page for a closed auction still resolves; it renders an archived layout saying the auction is no longer open, with links to live inventory. Nothing 404s because a date moved.
Relative urgency is derived the same way: "ends today", "last 7 days" or "open auction". That label is stored, so the cards and the detail page cannot disagree.
Hub pages from collection views
There are pages for provinces, municipalities, asset types and province × type combinations. None is hand-written, and none counts at request time. The collection layer builds two aggregate views during ingestion: one grouped by asset type, one grouped by province and type, each with a count and title templates.
"provincia_tipo_pages": {
"key": "pair_key",
"stages": [
{ "type": "filter", "field": "provincia_slug", "op": "gt", "value": "" },
{ "type": "group_by", "fields": ["provincia_slug", "provincia", "tipo_bien"] },
{ "type": "count_if", "field": "id", "op": "ne", "value": "", "as": "total_subastas" }
]
}
The manifest then declares routes that read those views: /provincia/{provincia_slug}, /provincia/{provincia_slug}/{localidad_slug}, /tipo/{tipo_bien}, /provincia/{provincia_slug}/tipo/{tipo_bien} and /subasta/{slug}. Each hub combines a directory block, a price aggregate ("from X €"), the most urgent live auctions and a subscription form. A province page with no inventory is not rendered, so the sitemap never claims pages that would be empty.
Asset type is its own normalisation problem. Headings are free text: vivienda, piso, ático, plaza de garaje, nave industrial, finca rústica. I match the longest useful token from the heading plus the short description, then map it into a small closed set with a label and a plural, so titles read naturally: "Viviendas en subasta en Madrid".
A detail page that helps decide
The auction page is not a dump of the record. It is a table of the numbers that matter — value, appraisal, minimum bid, deposit, deadline, occupancy, charges, whether it is visitable, cadastral reference — followed by the bidding steps in order, with the deposit amount written into the sentence that tells you to pay it.
Below that come two related blocks: other live auctions in the same province, sorted by closing date, and other auctions of the same asset type. Then the cross-links to the same province elsewhere: public tenders at licitacionesalerta.es and grants at subvenciones.devetty.es. Someone looking at a repossessed flat in a province is plausibly also interested in what that province is buying and subsidising.
Every page links to the official record on subastas.boe.es. My page is a reading layer, not the source of truth, and the deposit is paid on the government portal.
Thin pages are noindexed
A page earns indexing only if it has the identifier, the description, the province and an auction value. Records missing the value are published but not indexed. Programmatic sites fail in a predictable way: the template works, so you generate every combination, and half are a heading above an empty table. If the data cannot support the page, the page should not compete in search.
Guides stay separate from listings. Articles are a normal posts collection with a language field, served under /blog, including the step-by-step guide to bidding in a BOE auction. It answers the question the listing raises and links back into the search, instead of pretending a listing page can teach the process.
What I would do differently
I would start with the canonical place names. I wrote the listing parser first and the province chain second, then had to reingest to fix slugs. It would have been cheaper to decide the identity of a place before storing the first record.
I would also treat every prose field as a potential boolean from the start. Occupancy and charges became useful filters only after I stopped storing them as text and started asking what decision they support.
Try it: Subastas BOE — search by province, type and maximum value, or start from Madrid or viviendas. The site is in Spanish.
What public portal in your country publishes listings structured enough to build on, and which field always ruins the schema?
Top comments (0)