Scraping 50k Spanish Public Tenders (PLACSP) into Clean JSON in <26s with Apify
Public procurement in Spain represents over €100 billion annually. The Spanish central government, 17 autonomous regions, and thousands of municipal councils publish every tender, contract award, and minor RFP on the official procurement platform: PLACSP (Plataforma de Contratación del Sector Público).
Under Spanish Law 37/2007 (Reutilización de la Información del Sector Público / RISP), this data is legally public and open. However, if you have ever tried to consume PLACSP data programmatically, you already know the engineering horror behind the curtain.
In this technical showcase, we break down why traditional web scraping fails against PLACSP, how we engineered an Apify Actor that bypasses slow government XML feeds in sub-26 seconds using a dual-tier cache strategy, and how you can query active tenders in Python or cURL for $0.02 per result.
The Engineering Problem: Why Scraping PLACSP is a Nightmare
PLACSP provides an open syndication feed (licitacionesPerfilesContratanteCompleto3.atom). On paper, it sounds simple: download Atom feeds, parse entries, load to your database. In production, five major bottlenecks break standard pipelines:
1. Legacy CODICE XML & Deep Namespaces
PLACSP feeds do not return simple REST payloads. Each entry is wrapped in complex European CODICE XML standards:
<entry xmlns="http://www.w3.org/2005/Atom"
xmlns:cbc="urn:dgpe:names:draft:codice:schema:xsd:CommonBasicComponents-2"
xmlns:cac="urn:dgpe:names:draft:codice:schema:xsd:CommonAggregateComponents-2"
xmlns:cbc-place-ext="urn:dgpe:names:draft:codice-place-ext:schema:xsd:CommonBasicComponents-2">
<cac-place-ext:ContractFolderStatus>
<cbc:ContractFolderID>2026/00142</cbc:ContractFolderID>
<cbc-place-ext:ContractFolderStatusCode>PUB</cbc-place-ext:ContractFolderStatusCode>
<cac:ProcurementProject>
<cbc:Name>Mantenimiento de infraestructura cloud</cbc:Name>
<cac:RequiredCommodityClassification>
<cbc:ItemClassificationCode listURI="http://simap.europa.eu/cpv">72200000</cbc:ItemClassificationCode>
</cac:RequiredCommodityClassification>
</cac:ProcurementProject>
</cac-place-ext:ContractFolderStatus>
</entry>
Parsing these deep, polymorphic trees requires strict namespace management, fallback logic across multiple schema versions (cbc vs cbc-place-ext), and safe extraction of split date/time fields.
2. High Latency & HTTP 504 Timeouts
The official servers at contrataciondelestado.es are notoriously fragile under load. A single paginated Atom document containing 500 records often takes 15 to 30 seconds to transfer. Traversing 10 to 15 pages in real-time will trigger scraper timeouts or gateway errors (504 Gateway Timeout).
3. State Tracking and Elapsed Deadlines
More than 70% of entries in raw feeds are status changes (EV for Evaluation, RES for Awarded, ADJ for Adjudicated) or tenders whose bid deadline elapsed weeks ago. To get actionable leads, you must:
- Filter exclusively for active status (
ContractFolderStatusCode: PUB). - Parse and validate submission deadlines against UTC
now(). - Filter out tenders that close before your team can act.
4. GDPR & Data Sanitization (Spanish Organic Law 3/2018)
Public procurement tenders include names and personal tax IDs (DNI/NIE) of individual natural persons (sole traders and physical person representatives) inside contact tags. Publishing or processing these without explicit consent violates GDPR. A production-grade extractor must apply automated regex filters to suppress personal identifiers while preserving corporate tax IDs (CIF) and public contracting body names.
Architecture: How the Apify Actor Achieves Sub-26s Runs
To solve the speed and timeout constraints, the PLACSP Spain Public Tenders Scraper implements a dual-tier caching architecture:
[ User Request / API Call ]
│
▼
[ Apify Actor: placsp-spain-tenders-scraper ]
│
├──► 1. Fetch Fast Hourly Cache (< 1.5s)
│ (50k+ normalized active dossiers from CDN / tack.devetty.es)
│
├──► 2. Live Feed Head Check (< 15s)
│ (Pulls page 1 of PLACSP Atom feed to capture tenders published in the last 60 min)
│
├──► 3. In-Memory Filter Engine (< 0.2s)
│ - CPV Division match (e.g., '72' IT, '45' Construction, '50' Maintenance)
│ - NUTS code & Territorial matcher (e.g., 'ES30' Madrid, 'ES51' Cataluña)
│ - Budget ceiling without VAT (e.g., < 60.000 € for minor/simplified contracts)
│ - Deadline validation (status == 'PUB' && deadline > now())
│ - GDPR sanitizer (strips DNI/NIE)
│
▼
[ Output: Clean JSON Dataset & Apify Key-Value Store ]
Why this works:
- Zero compute waste: Instead of crawling 15 legacy XML pages from government servers on every run, the Actor loads an hourly pre-crawled and normalized dataset in compressed gzip format.
- Real-time accuracy: It queries only the head of the live Atom stream (page 1) to ingest contracts published minutes ago.
- Execution time: Complete extraction and filtering finishes in under 26 seconds, well below Apify's default timeout limits.
Consuming the Actor: Code Examples
The Actor is deployed as a Pay-Per-Event (PPE) actor on the Apify Store at $0.02 per matching tender. You only pay for exact tenders matching your criteria—never for compute time, pagination skips, or empty results.
1. Quick cURL Execution
You can trigger a synchronous run and get JSON results directly via the Apify API:
curl -X POST "https://api.apify.com/v2/acts/devetty~placsp-spain-tenders-scraper/run-sync-get-dataset-items?token=YOUR_APIFY_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"cpv_prefix": "72",
"province_or_nuts": "Madrid",
"nuts_prefix": "ES30",
"max_budget": 100000,
"status_filter": "PUB",
"only_future_deadlines": true,
"max_results": 20
}'
2. Production Python Integration (Apify Client)
Here is a full Python script to stream active IT and software contracts (CPV: 72) under €60,000 in Madrid and notify your CRM or messaging webhook:
import os
from apify_client import ApifyClient
# Initialize the Apify client with your API token
client = ApifyClient(os.getenv("APIFY_TOKEN", "YOUR_APIFY_TOKEN"))
# Define search parameters for Spanish public contracts
run_input = {
# CPV 72: IT services: consulting, software development, Internet and support
"cpv_prefix": "72",
# Target region: Madrid (NUTS ES30)
"province_or_nuts": "Madrid",
"nuts_prefix": "ES30",
# SME focus: Contracts under €60,000 (Simplified open procedure / Art. 159.6 LCSP)
"max_budget": 60000,
# Only open tenders currently accepting bids
"status_filter": "PUB",
"only_future_deadlines": True,
"max_results": 25,
}
print("[*] Running PLACSP scraper on Apify...")
run = client.actor("devetty/placsp-spain-tenders-scraper").call(run_input=run_input)
# Fetch results from the run's default dataset
dataset_items = client.dataset(run["defaultDatasetId"]).list_items().items
print(f"[*] Retrieved {len(dataset_items)} active tenders matching criteria:\n")
for tender in dataset_items:
print(f"📄 Dossier ID: {tender['expediente_id']}")
print(f"🏢 Authority: {tender['organo_contratacion']}")
print(f"📌 Title: {tender['titulo']}")
print(f"💰 Net Budget: {tender['presupuesto_sin_iva']:,.2f} € (Total: {tender['importe_total_con_iva']:,.2f} €)")
print(f"⏰ Bid Deadline: {tender['fecha_limite']}")
print(f"🔗 Direct Link: {tender['enlace_oficial']}")
print(f"⚖️ Attribution: {tender['cita_legal']}")
print("-" * 80)
Clean JSON Output Schema
Every tender returned by the Actor conforms to a predictable schema ready for ingestion:
{
"expediente_id": "2026/AR21U/00001560E",
"organo_contratacion": "Subdirección General de Gestión Económica y Asuntos Generales",
"titulo": "Mantenimiento y conservación de redes de comunicaciones y servidores",
"presupuesto_sin_iva": 33057.85,
"importe_total_con_iva": 40000.00,
"fecha_limite": "2026-11-18 14:00:00",
"estado": "PUB",
"enlace_oficial": "https://contrataciondelestado.es/wps/poc?uri=deeplink:detalle_licitacion&idEvl=w1K9...",
"cpvs": ["72500000", "72220000"],
"provincias": ["Madrid"],
"nuts": ["ES300"],
"ciudades": ["Madrid"],
"codigos_postales": ["28071"],
"cita_legal": "Origen de los datos: Plataforma de Contratación del Sector Público, Ministerio de Hacienda"
}
Key Fields:
-
presupuesto_sin_iva&importe_total_con_iva: Clean floating-point values for budget threshold alerts. -
enlace_oficial: Verified deep-link resolving straight to the bidding dossier oncontrataciondelestado.es. -
cita_legal: Mandatory attribution notice required by Spanish Law 37/2007 (RISP) pre-populated on every record for compliance.
Common Use Cases
- GovTech SaaS & Alert Bots: Build daily Slack/Telegram alerts for B2B contractors without operating your own XML infrastructure.
- SME & Freelance Bidding (Art. 159.6 LCSP): Monitor simplified procedures (Procedimiento Abierto Supersimplificado) under €60k where public bodies do not require provisional guarantees or heavy financial auditing.
- Market Intelligence & Price Benchmarking: Track public procurement pricing trends and monitor competitor bids across specific CPV codes and autonomous regions.
Get Started
The scraper is live and maintained on the Apify Store:
👉 Try the PLACSP Spain Public Tenders Scraper on Apify Store
Pricing: $0.02 / result event (Pay-Per-Event). Zero infrastructure overhead.
Appendix: Ready-to-Publish Community Posts
1. LinkedIn B2B Engineering Post
🚀 How we automated 50k Spanish Public Procurement Tenders (PLACSP) into clean JSON in <26s.
Public procurement in Spain exceeds €100B/year, but scraping the official Plataforma de Contratación del Sector Público (PLACSP) is notoriously painful:
- Legacy CODICE XML schemas with 5+ nested namespaces
- Slow Atom servers with frequent 504 timeouts (15-30s per page)
- 70% outdated or closed tenders
- Strict GDPR restrictions on personal DNIs
We built and published the official PLACSP scraper on Apify Store using a dual-tier cache architecture:
1. Hourly pre-crawled normalized cache (<1.5s response)
2. Live feed head check for tenders published in the last 60 minutes
3. Instant CPV (e.g. 72 IT, 50 Maintenance), NUTS territory, and budget filters (<60k€ for SMEs)
4. Full GDPR and Law 37/2007 Open Data compliance
Now available as a Pay-Per-Event Actor ($0.02/result) with direct JSON/CSV export and API integration:
https://apify.com/devetty/placsp-spain-tenders-scraper?utm_source=linkedin&utm_medium=post&utm_campaign=placsp_showcase
Full technical breakdown and Python integration code on Dev.to.
#GovTech #OpenData #WebScraping #Python #Apify #B2B
2. Twitter / X Thread
1/4 Scraping Spanish Public Tenders (PLACSP) is an engineering nightmare: massive CODICE XML feeds, 504 server timeouts, and nested namespaces. 🇪🇸💻
Here is how we solved it to extract 50k+ active contracts in <26s with Python & @apify 👇
2/4 The bottleneck: PLACSP's syndication Atom feed serves 500 records/page, taking 20s+ per page.
Our solution: Dual-tier caching. An hourly pre-crawled normalized cache + a 1-page live diff for the last 60 mins. Response time drops from 5 minutes to 24 seconds.
3/4 Filter directly by CPV (72 IT, 45 Construction, 50 Maintenance), budget (<60k€ for simplified SME contracts), and territorial NUTS (ES30 Madrid, ES51 BCN) with 100% GDPR sanitization.
4/4 Try the Actor on Apify Store (Pay-Per-Event, $0.02/tender):
https://apify.com/devetty/placsp-spain-tenders-scraper?utm_source=twitter&utm_medium=post&utm_campaign=placsp_showcase
Code examples & schema in our Dev.to showcase! #webscraping #opendata #python
3. Tech Forum / Indie Hackers Post
Title: Building a sub-26s GovTech Scraper for Spanish Public Tenders ($0.02 Pay-Per-Event)
Hey hackers,
If you've ever explored European GovTech, you know public procurement is huge, but government APIs are stuck in 2008.
In Spain, all public contracts are published on PLACSP via Atom syndication feeds using CODICE XML schemas. We wanted to build automated alerts for SMEs targeting simplified contracts (<60k€ under Art. 159.6 LCSP), but hitting government servers live was causing constant timeouts.
We deployed an Apify Actor with a dual-cache model:
- Daily/hourly bulk crawl cached as normalized JSON.gz
- Live diff against page 1 of the Atom feed during actor startup
- Built-in filters for CPV categories, NUTS regions, and active bid deadlines
- Pay-Per-Event pricing on Apify Store ($0.02 per matching dossier)
Check out the actor here:
https://apify.com/devetty/placsp-spain-tenders-scraper?utm_source=indiehackers&utm_medium=community&utm_campaign=placsp_showcase
Would love feedback on our dual-cache approach from anyone else scraping heavy public data feeds!
Top comments (0)