DEV Community

Lindenwerk Data
Lindenwerk Data

Posted on

Build a daily French public-tender alert in Python with the free BOAMP open data API

#python #opendata #api #tutorial

French public buyers advertise their contracts in the BOAMP, the Bulletin officiel des annonces des marchés publics. It's run by the DILA, the French government's legal-information directorate. According to the DILA, notices above the EU thresholds have to be published there, and below them it's one of the allowed channels. Every notice is also released as open data on an Opendatasoft portal, refreshed twice a day, seven days a week, under the Licence Ouverte / Open Licence 2.0. There's no account and no API key.

boamp.fr has its own search. If you want the notices in your own tools (a spreadsheet, a morning email filtered your way), the API is easier. Here's what the script in this tutorial does:

  1. pulls every notice published on a given day, within the API's paging limits,
  2. tells new tenders, corrections and award notices apart,
  3. digs CPV codes out of three different notice formats,
  4. filters by CPV, French keywords, département, procedure and market type,
  5. follows corrections (avis rectificatifs) to notices you've already seen,
  6. writes CSV and an email-ready text file, remembers what it has reported, and is scheduled daily.

Dependencies: Python 3 (tested on 3.13) plus requests. The full script is at the end.

The endpoint 

GET https://boamp-datadila.opendatasoft.com/api/explore/v2.1/catalog/datasets/boamp/records
Enter fullscreen mode Exit fullscreen mode

It's the standard Opendatasoft Explore API v2.1 (dataset id boamp), with filters written in ODSQL, a small SQL-like language:

Parameter Meaning
where ODSQL filter, e.g. dateparution = date'2026-10-16'
order_by Sort order. Set it, or offset paging isn't reliable
limit Records per page, at most 100
offset Index of the first record
select, group_by Pick fields, or aggregate (count(*))

Try it:

curl -s -G "https://boamp-datadila.opendatasoft.com/api/explore/v2.1/catalog/datasets/boamp/records" \
  --data-urlencode "where=dateparution = date'2026-10-16'" \
  --data-urlencode "select=idweb, objet, nomacheteur, famille, nature_libelle" \
  --data-urlencode "limit=3" | python -m json.tool
Enter fullscreen mode Exit fullscreen mode

The answer is {"total_count": ..., "results": [...]}. When we looked on 17 October 2026, the dataset held 1,717,457 notices.

Two limits shape the whole design:

  • offset + limit can't go above 10,000. We asked for offset=9950&limit=100 and got HTTP 400: "Invalid value for sum of offset + limit API parameter: 10050 was found but <= 10000 is expected." For bigger pulls the docs point to the /exports endpoint.
  • A daily quota. The docs list HTTP 429 for "too many requests", and responses carry X-RateLimit-* headers (reset at midnight UTC). We saw a limit of 2,000,000 on /records (10,000 on the metadata call), and the remaining count fell faster than our own calls, so it looks like a shared anonymous pool. A daily alert needs about ten requests, but back off on 429 anyway.

How much is a day? Between 3 and 16 October 2026, weekdays had 223 to 584 notices, and BOAMP published at weekends too (124 and 143 on the Saturdays, 317 and 323 on the Sundays). Two weeks came to 5,276, so a month would hit the offset cap. The script asks for one publication day at a time.

What one record looks like

Here's a real national-form notice from 16 October 2026, trimmed:

{
  "idweb": "26-100546",
  "dateparution": "2026-10-16",
  "famille": "FNS",
  "nature_libelle": "Avis de marché",
  "etat": "INITIAL",
  "objet": "Maintenance et assistance du logiciel Businessobjects",
  "nomacheteur": "Département de la Lozère",
  "code_departement": ["48"],
  "type_procedure": "OUVERT",
  "type_marche": ["SERVICES"],
  "descripteur_libelle": ["Logiciel", "Maintenance", "Informatique (assistance)"],
  "datelimitereponse": "2026-11-09T12:00:00+00:00",
  "annonce_lie": null,
  "url_avis": "https://www.boamp.fr/pages/avis/?q=idweb:26-100546",
  "gestion": "{\"REFERENCE\": {...}, \"INDEXATION\": {...}}",
  "donnees": "{\"FNSimple\": {\"organisme\": {...}, \"initial\": {...}}}"
}
Enter fullscreen mode Exit fullscreen mode
  • idweb is the notice id, unique per published notice.
  • famille tells you the format. JOUE is a European notice in eForms, FNS is the national simplified form (the dataset labels it "Marchés entre 90 k€ et seuils européens"), MAPA is an adapted-procedure notice ("Marchés <90 k€"), and there's the occasional DSP (public-service delegation).
  • donnees is a JSON document stored as a string, and its schema depends on famille. It holds most of the detail, CPV codes included (there's no CPV column).
  • descripteur_libelle is BOAMP's own subject vocabulary (more on it below).
  • Personal data. donnees often names a contact person, with a direct e-mail address and phone number. The DILA reminds reusers that French data-protection law still applies, so our monitor keeps organisation names only.

Step 1: fetch one day, paginate politely 

API = "https://boamp-datadila.opendatasoft.com/api/explore/v2.1/catalog/datasets/boamp/records"
USER_AGENT = "LindenwerkData-boamp-monitor/1.0 (+https://dev.to/lindenwerkdata; open-data tutorial)"

def fetch_day(s, day, natures):
    where = f"dateparution = date'{day.isoformat()}'"
    if natures:
        where += " and (" + " or ".join(f"nature_libelle = {odsql_quote(n)}" for n in natures) + ")"
    records, offset, pages = [], 0, 0
    while True:
        params = {"where": where, "order_by": "idweb", "limit": 100, "offset": offset}
        data = get_with_retry(s, params).json()     # timeout, backoff on 429/5xx
        records.extend(data.get("results") or [])
        pages += 1
        offset += 100
        if offset >= data.get("total_count", 0):
            break
        if offset + 100 > 10_000:                    # can't happen for one day, but be explicit
            print(f"warning: {day} is too big for /records, use /exports", file=sys.stderr)
            break
        time.sleep(1)
    return records, pages
Enter fullscreen mode Exit fullscreen mode

Set order_by. Without a fixed order, nothing guarantees that consecutive pages line up, so sort on the unique idweb. Accented values like "Avis de marché" work in where; let requests do the URL encoding.

On 17 October 2026 the full feed for 16 October was 562 records on 6 pages (about 9 seconds with a 1-second pause). With the default filter (tenders and corrections only), the script fetched 865 records for 15 and 16 October in 10 requests, in about 16 seconds.

A small trap if you use group_by to count things: fields such as type_avis and code_departement are multi-valued, and grouping counts a notice once per value. Our first per-type count for 16 October added up to 1,100, for a day with 562 notices.

Step 2: know your notice types, and the two shapes of a correction

The 562 notices of 16 October were 410 Avis de marché (calls for competition), 127 Résultat de marché (awards), 24 Rectificatif and 1 Modification. An alert wants the first group plus corrections, and corrections come in two shapes:

  • European (eForms) corrections are full notices. They carry nature_libelle = "Avis de marché", etat = "RECTIFICATIF", and the original notice's id in annonce_lie. On 16 October, 22 of the 410 "Avis de marché" were these. If you only filter on nature_libelle, they look like brand-new tenders.
  • National and MAPA corrections are separate Rectificatif notices. They carry the original id and a free-text infosRectif, and datelimitereponse is empty. That was true for all 55 of them in our two-day window. On 16 October, 22 of the 24 moved the response deadline, and the new date was written in at least six ways: "Au lieu de 04/11/2026 à 12h00, lire 18/11/2026 à 12h00", "reportée au 17 novembre 2026 à 12H00mn", "déplacée au 13.11.2026 à 18h00", a weekday plus a date, or a small JSON object with two ISO timestamps.

Don't parse that text. Show it to a human:

is_correction = rec["nature_libelle"] == "Rectificatif" or rec["etat"] == "RECTIFICATIF"
corrects = (rec.get("annonce_lie") or [""])[0] if is_correction else ""
Enter fullscreen mode Exit fullscreen mode

In the two-day window, 99 of the 102 corrections named their original, but only 4 of those originals were in the same window. Most corrections refer to older notices, which is why the state file below matters.

Step 3: CPV codes are buried in donnees

National forms keep CPV codes under .../codeCPV/objetPrincipal/classPrincipale, sometimes per lot. eForms keep them in cbc:ItemClassificationCode elements, with the code under #text. Rather than hard-code a dozen paths, walk the document:

def walk(obj, path=""):
    if isinstance(obj, dict):
        for k, v in obj.items():
            yield from walk(v, f"{path}/{k}")
    elif isinstance(obj, list):
        for v in obj:
            yield from walk(v, path)
    else:
        yield path, obj

def cpv_codes(donnees):
    found = []
    for path, value in walk(donnees):
        if ("CPV" in path or "ItemClassificationCode" in path) and re.fullmatch(r"\d{8}(-\d)?", str(value)):
            code = str(value)[:8]
            if code not in found:
                found.append(code)
    return found
Enter fullscreen mode Exit fullscreen mode

Check the path, not just the value: eight-digit numbers also appear as amounts and ids.

Now the gaps. Of the 763 new tenders in the window, all 66 MAPA notices had no CPV code at all, and 14 of 347 national-form notices had none either. The eForms notices all had one. A CPV-only filter would never show you a MAPA notice. That's where descripteur_libelle helps: every one of the 763 had BOAMP descriptors (239 different labels in the window), so the script matches keywords against the title and the descriptors.

Also: objet is a shortened title (never over 200 characters in our data, and once re-cased from "SIRH" to "Sirh"); the full title is inside donnees. And text in donnees can be HTML-escaped: 149 of the 865 records contained entities such as &amp;, a few escaped twice (&amp;lt;br/&amp;gt;). Unescape before you display.

Step 4: départements and accents

code_departement is a list, and it has two quirks. BOAMP drops leading zeros ("7" for Ardèche, not "07"), and it writes Corsica as "20A" and "20B" instead of the official 2A and 2B. A filter for 2A would have found nothing, though 14 notices in our window were for Corsica. Also, 170 of the 763 tenders listed more than one département (up to 9), so treat the field as "any of".

def norm_dept(code):
    code = str(code).strip().upper()
    if code in ("20A", "20B"):
        return "2" + code[-1]
    return code.zfill(2) if code.isdigit() and len(code) < 2 else code

def fold(text):
    text = text.replace("’", "'").replace("œ", "oe").replace("Œ", "OE")
    text = unicodedata.normalize("NFKD", text)
    return "".join(c for c in text if not unicodedata.combining(c)).lower()
Enter fullscreen mode Exit fullscreen mode

Accent folding isn't optional. BOAMP's own descriptor is spelled Electricité (travaux), with no accent on the capital E. In the window, a plain case-insensitive search for "électricité" found 3 notices, and the folded search found 55. Apostrophes vary too: 117 of the 865 titles used the typographic ’, so système d’information and système d'information need to become the same string. The keyword regex anchors only at the start of a word, so logiciel also finds logiciels.

Step 5: don't do arithmetic on datelimitereponse

Every datelimitereponse ends in +00:00, but that suffix doesn't mean the same thing across notices:

  • In all 414 non-eForms notices with a deadline (national, MAPA and one DSP), the value is simply the clock time from the notice with +00:00 stuck on. "12:00" means noon local time, not noon UTC.
  • In 327 of the 373 eForms notices, it really was converted to UTC (12:00+01:00 became 11:00+00:00). In the other 46, the clock time was copied unchanged, even when the notice said +02:00.
  • The original values in the gestion JSON carry their own offsets, including +03:00, +04:00 and -04:00 for buyers in Mayotte, La Réunion, Guadeloupe and Saint-Martin.

For an alert, show the deadline as written, offset included:

def deadline_as_written(rec):
    gestion = json.loads(rec.get("gestion") or "{}")
    raw = (gestion.get("INDEXATION") or {}).get("DATE_LIMITE_REPONSE") or rec.get("datelimitereponse")
    if not raw:
        return ""
    offset = re.search(r"(Z|[+-]\d\d:\d\d)$", raw)
    return raw[:16].replace("T", " ") + (f" {offset.group(1)}" if offset else "")
Enter fullscreen mode Exit fullscreen mode

Also, 21 of the 763 tenders had no deadline at all.

Step 6: filter, follow corrections, remember

A notice matches on a CPV prefix or a keyword; hard filters (département, procedure, market type, excluded words) can then remove it. The state file is a small JSON dict of idweb to publication date, pruned after 180 days:

seen = load_state(state_path)
for r in rows:
    if r["idweb"] in seen:
        continue
    reasons = matches(r, args)
    if not reasons and r["kind"] == "correction" and r["corrects"] in seen:
        reasons = [f"corrects {r['corrects']}, which you were sent before"]
    if reasons:
        r["reasons"] = "; ".join(reasons)
        out.append(r)
seen.update({r["idweb"]: r["published"] for r in out})
Enter fullscreen mode Exit fullscreen mode

The second rule is the useful one: a correction gets through if it matches your profile or fixes a notice you were sent, even when it's too thin to match on its own.

On the two-day window, an IT profile (--cpv 72 48 --keyword informatique logiciel cybersécurité "système d'information") reported 33 notices: 26 new tenders and 7 corrections. Nine of the 33 had no matching CPV code and came in through keywords, including 3 MAPA notices. Descriptors also bring noise: a laser projector and ticketing system for a small cinema matched "informatique" through its descriptors alone. Read the why column before you trust a match. Other profiles on the same data:

  • Brittany works (--cpv 45 --dept 22 29 35 56 --procedure OUVERT PROCEDURE_ADAPTE): 15
  • Corsica, anything (--dept 2A 2B): 14
  • Open-procedure services in Paris and the inner suburbs (--dept 75 92 93 94 --market SERVICES --procedure OUVERT): 62

Replaying the same saved records with the same state file produced 0. While tuning, save the raw records once with --save-raw raw.json and iterate offline with --from-file raw.json.

Step 7: output and attribution

The CSV is encoded as utf-8-sig. Microsoft's support page says Excel opens a UTF-8 CSV normally only if it was saved with a byte-order mark; otherwise you have to go through the import dialog. The text file is ready to paste into an email:

BOAMP alert for 2026-10-15 to 2026-10-16: 26 new notice(s), 7 correction(s)

* Maintenance et assistance du logiciel Businessobjects
  Département de la Lozère | dept 48 | OUVERT | deadline 2026-11-09 12:00
  why: CPV 72; 'informatique'; 'logiciel'
  https://www.boamp.fr/pages/avis/?q=idweb:26-100546

Corrections (avis rectificatifs):

* Acquisition et déploiement d'un Système d'Information de Gestion des Ressources Humaines (Sirh) ... (corrects 26-95318)
  Rubrique : Date et heure limites de réception des plis, Au lieu de 04/11/2026 à 17h00, lire 16/11/2026 à 17h00 ...
  https://www.boamp.fr/pages/avis/?q=idweb:26-100060
Enter fullscreen mode Exit fullscreen mode

The Licence Ouverte allows free reuse, commercial too, as long as you credit the source. The DILA's reuse conditions ask you to name the DILA, the download URL, and the file and its date, so every output ends with the DILA credit, the dataset and API URLs, and the retrieval timestamp. The script also scrubs e-mail addresses and phone numbers from correction text.

Step 8: run it every day

BOAMP publishes at weekends, so run it every day. Fetching the last two days and letting the state file drop repeats also catches late additions.

Cron (it uses the machine's clock; write % as \%):

17 7 * * *  cd $HOME/boamp-monitor && TZ=Europe/Paris ./venv/bin/python fr_boamp_monitor.py --days 2 --cpv 72 48 --keyword informatique logiciel --state seen.json --csv "out/$(date +\%F).csv" --text "out/$(date +\%F).txt" >> monitor.log 2>&1
Enter fullscreen mode Exit fullscreen mode

GitHub Actions (.github/workflows/boamp-monitor.yml). It commits seen.json back, so the state survives between runs:

name: BOAMP tender monitor
on:
  schedule:
    - cron: "17 7 * * *"
      timezone: "Europe/Paris"
  workflow_dispatch: {}
permissions:
  contents: write
jobs:
  monitor:
    runs-on: ubuntu-latest
    timeout-minutes: 15
    env:
      TZ: Europe/Paris
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: {python-version: "3.12"}
      - run: pip install -r requirements.txt
      - run: python fr_boamp_monitor.py --days 2 --cpv 72 48 --keyword informatique logiciel --state seen.json --csv boamp.csv --text boamp.txt
      - uses: actions/upload-artifact@v4
        with: {name: "boamp-${{ github.run_id }}", path: "boamp.*", retention-days: 30}
      - run: |
          git config user.name "github-actions[bot]"
          git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
          git add seen.json
          git diff --cached --quiet || git commit -m "BOAMP state $(date +%F)"
          git push
Enter fullscreen mode Exit fullscreen mode

From GitHub's docs: the default schedule time zone is UTC (override it with timezone); runs may start late when GitHub is busy, the top of the hour above all, and some are dropped under heavy load; and in a public repository, schedules are disabled after 60 days without activity. A private repository suits your filters and state file better anyway.

Where to take it next

  • Add a "closing soon" view. One eForms correction in our window carried a deadline that had already passed by the time we ran.
  • Switch --nature to "Résultat de marché" to watch award notices. titulaire holds the winners' names, so check for personal data before you store it.
  • Always open the official notice on boamp.fr before you respond.

Official documentation

Full script: fr_boamp_monitor.py (334 lines, requests only) 
#!/usr/bin/env python3
"""Daily French public-tender alert built on the BOAMP open-data API.

BOAMP (Bulletin officiel des annonces des marchés publics) is published by the DILA as open data
on an Opendatasoft portal (Explore API v2.1). Free, no account, no API key.
Data: DILA, BOAMP, reused under the Licence Ouverte / Open Licence v2.0.

Usage examples:
    python fr_boamp_monitor.py --from 2026-10-15 --to 2026-10-17 --cpv 72 48 --keyword informatique logiciel
    python fr_boamp_monitor.py --yesterday --cpv 45 --dept 29 35 56 22 --procedure OUVERT PROCEDURE_ADAPTE
    python fr_boamp_monitor.py --days 3 --keyword nettoyage --state seen.json   # only what you haven't seen
"""
from __future__ import annotations

import argparse
import csv
import html
import json
import re
import sys
import time
import unicodedata
from datetime import date, datetime, timedelta, timezone
from pathlib import Path
from zoneinfo import ZoneInfo

import requests

DATASET_PAGE = "https://boamp-datadila.opendatasoft.com/explore/dataset/boamp/"
API = "https://boamp-datadila.opendatasoft.com/api/explore/v2.1/catalog/datasets/boamp/records"
USER_AGENT = "LindenwerkData-boamp-monitor/1.0 (+https://dev.to/lindenwerkdata; open-data tutorial)"
PAGE_SIZE = 100          # maximum for /records without group_by
OFFSET_CAP = 10_000      # offset + limit may not go above this, so we query one publication day at a time
PAUSE_BETWEEN_PAGES = 1.0
PARIS = ZoneInfo("Europe/Paris")
# Tender notices plus the national correction notices. European corrections arrive as "Avis de marché"
# records with etat = RECTIFICATIF, so they are included by the first value.
DEFAULT_NATURES = ["Avis de marché", "Rectificatif"]
KINDS = {"Avis de marché": "new", "Résultat de marché": "result"}


# ---------------------------------------------------------------- 1. fetch one day, paginate politely
def odsql_quote(value: str) -> str:
    return '"' + value.replace("\\", "\\\\").replace('"', '\\"') + '"'


def fetch_day(s: requests.Session, day: date, natures: list[str]) -> tuple[list[dict], int]:
    where = f"dateparution = date'{day.isoformat()}'"
    if natures:
        where += " and (" + " or ".join(f"nature_libelle = {odsql_quote(n)}" for n in natures) + ")"
    records, offset, pages = [], 0, 0
    while True:
        params = {"where": where, "order_by": "idweb", "limit": PAGE_SIZE, "offset": offset}
        data = get_with_retry(s, params).json()
        records.extend(data.get("results") or [])
        pages += 1
        total = data.get("total_count", 0)
        offset += PAGE_SIZE
        if offset >= total:
            break
        if offset + PAGE_SIZE > OFFSET_CAP:
            print(f"warning: {day} has {total} records, more than /records can page through; "
                  "use the /exports endpoint for this day", file=sys.stderr)
            break
        time.sleep(PAUSE_BETWEEN_PAGES)
    return records, pages


def get_with_retry(s: requests.Session, params: dict, tries: int = 5) -> requests.Response:
    for attempt in range(tries):
        try:
            resp = s.get(API, params=params, timeout=(10, 60))
        except (requests.ConnectionError, requests.Timeout) as exc:
            wait = 2 ** attempt * 5
            print(f"network error ({exc.__class__.__name__}), retrying in {wait}s", file=sys.stderr)
            time.sleep(wait)
            continue
        if resp.status_code == 200:
            return resp
        if resp.status_code == 429 or resp.status_code >= 500:
            wait = 2 ** attempt * 15
            print(f"HTTP {resp.status_code}, waiting {wait}s "
                  f"(X-RateLimit-Remaining={resp.headers.get('X-RateLimit-Remaining')})", file=sys.stderr)
            time.sleep(wait)
            continue
        print(resp.text[:300], file=sys.stderr)  # 400 = bad ODSQL or offset over the cap: a bug, fail loudly
        resp.raise_for_status()
    raise RuntimeError(f"giving up after {tries} attempts")


# ---------------------------------------------------------------- 2. flatten one record
def walk(obj, path=""):
    """Yield (path, value) for every leaf of the nested `donnees` JSON."""
    if isinstance(obj, dict):
        for k, v in obj.items():
            yield from walk(v, f"{path}/{k}")
    elif isinstance(obj, list):
        for v in obj:
            yield from walk(v, path)
    else:
        yield path, obj


def cpv_codes(donnees: dict) -> list[str]:
    """National forms keep CPV under .../codeCPV/... or .../CPV/...; eForms under cbc:ItemClassificationCode."""
    found = []
    for path, value in walk(donnees):
        if ("CPV" in path or "ItemClassificationCode" in path) and re.fullmatch(r"\d{8}(-\d)?", str(value)):
            code = str(value)[:8]
            if code not in found:
                found.append(code)
    return found


def norm_dept(code: str) -> str:
    """BOAMP drops leading zeros ('7' for Ardèche) and writes Corsica as 20A/20B."""
    code = str(code).strip().upper()
    if code in ("20A", "20B"):
        return "2" + code[-1]
    return code.zfill(2) if code.isdigit() and len(code) < 2 else code


def deadline_as_written(rec: dict) -> str:
    """Response deadline as written in the notice, e.g. '2026-11-16 12:00' or '2026-11-16 12:00 +04:00'.

    Don't do arithmetic on `datelimitereponse`: it always ends in +00:00, but national and MAPA notices
    copy the local clock time into it unchanged, while most eForms notices are converted to UTC.
    The original value sits in the `gestion` JSON. Overseas buyers use their own offsets (+03:00, -04:00, ...)."""
    gestion = rec.get("gestion") or "{}"
    gestion = json.loads(gestion) if isinstance(gestion, str) else gestion
    raw = (gestion.get("INDEXATION") or {}).get("DATE_LIMITE_REPONSE")
    raw = raw or rec.get("datelimitereponse")
    if not raw:
        return ""
    offset = re.search(r"(Z|[+-]\d\d:\d\d)$", raw)
    return raw[:16].replace("T", " ") + (f" {offset.group(1)}" if offset else "")


EMAIL_RE = re.compile(r"[\w.+-]+@[\w-]+(\.[\w-]+)+")
PHONE_RE = re.compile(r"(?:(?:\+|00)\d{2,3}[\s.]?\d[\d\s.]{6,}\d|\b0[1-9](?:[\s.-]?\d{2}){4}\b)")


def clean(text: str) -> str:
    """Text inside `donnees` can be HTML-escaped, sometimes twice ('&amp;lt;br/&amp;gt;')."""
    for _ in range(2):
        text = html.unescape(text)
    return " ".join(re.sub(r"<[^>]+>", " ", text).split())


def correction_text(donnees: dict) -> str:
    parts = [str(v) for p, v in walk(donnees) if "/infosRectif" in p and v not in (None, "")]
    text = clean(" ".join(parts))
    text = PHONE_RE.sub("[phone]", EMAIL_RE.sub("[email]", text))  # free text: keep no contact details
    return text[:300]


def flatten(rec: dict) -> dict:
    donnees = rec.get("donnees") or "{}"
    donnees = json.loads(donnees) if isinstance(donnees, str) else donnees
    linked = rec.get("annonce_lie") or []
    is_correction = rec.get("nature_libelle") == "Rectificatif" or rec.get("etat") == "RECTIFICATIF"
    kind = "correction" if is_correction else KINDS.get(rec.get("nature_libelle"), "other")
    return {
        "idweb": rec["idweb"],
        "kind": kind,
        "corrects": linked[0] if is_correction and linked else "",
        "published": rec.get("dateparution"),
        "title": clean(rec.get("objet") or ""),
        "buyer": rec.get("nomacheteur") or "",          # organisation only; contact persons are left out
        "depts": "|".join(sorted({norm_dept(d) for d in rec.get("code_departement") or []})),
        "cpv": "|".join(cpv_codes(donnees)),
        "descriptors": "; ".join(rec.get("descripteur_libelle") or []),
        "market": "|".join(rec.get("type_marche") or []),
        "procedure": rec.get("type_procedure") or "",
        "family": rec.get("famille") or "",
        "deadline": deadline_as_written(rec),
        "change": correction_text(donnees) if is_correction else "",
        "url": rec.get("url_avis") or f"https://www.boamp.fr/pages/avis/?q=idweb:{rec['idweb']}",
    }


# ---------------------------------------------------------------- 3. filter
def fold(text: str) -> str:
    """Lower-case, strip accents, unify apostrophes: 'Système d’information' -> "systeme d'information"."""
    text = text.replace("’", "'").replace("œ", "oe").replace("Œ", "OE")
    text = unicodedata.normalize("NFKD", text)
    return "".join(c for c in text if not unicodedata.combining(c)).lower()


def matches(r: dict, args: argparse.Namespace) -> list[str]:
    """Return the reasons a row matches (empty list = no match). CPV OR keyword, then hard filters."""
    reasons = []
    cpvs = [c for c in r["cpv"].split("|") if c]
    hits = [p for p in args.cpv if any(c.startswith(p) for c in cpvs)]
    if hits:
        reasons.append("CPV " + ",".join(hits))
    haystack = fold(f"{r['title']} {r['descriptors']}")
    for kw in args.keyword:
        if re.search(r"\b" + re.escape(fold(kw)), haystack):  # word start only: 'logiciel' finds 'logiciels'
            reasons.append(f"'{kw}'")
    if (args.cpv or args.keyword) and not reasons:
        return []
    if args.dept and not set(r["depts"].split("|")) & {norm_dept(d) for d in args.dept}:
        return []
    if args.procedure and r["procedure"] not in args.procedure:
        return []
    if args.market and not set(r["market"].split("|")) & {m.upper() for m in args.market}:
        return []
    if any(fold(x) in haystack for x in args.exclude):
        return []
    return reasons or ["no filter"]


# ---------------------------------------------------------------- 4. state + output
def load_state(path: Path) -> dict:
    return json.loads(path.read_text(encoding="utf-8")) if path.exists() else {}


def save_state(path: Path, seen: dict, keep_days: int = 180) -> None:
    cutoff = (date.today() - timedelta(days=keep_days)).isoformat()
    seen = {k: v for k, v in seen.items() if v >= cutoff}       # keep the file small
    path.write_text(json.dumps(seen, indent=0, sort_keys=True), encoding="utf-8")


def attribution(fetched_at: str) -> str:
    return (f"Source: DILA, BOAMP open data ({DATASET_PAGE}, API {API}), retrieved {fetched_at}. "
            "Reused under the Licence Ouverte / Open Licence v2.0. Not an official service.")


def email_text(rows: list[dict], window: str, fetched_at: str) -> str:
    new = [r for r in rows if r["kind"] == "new"]
    fixes = [r for r in rows if r["kind"] == "correction"]
    lines = [f"BOAMP alert for {window}: {len(new)} new notice(s), {len(fixes)} correction(s)", ""]
    for r in new:
        lines += [f"* {r['title']}", f"  {r['buyer']} | dept {r['depts'] or '?'} | {r['procedure'] or '?'}"
                  f" | deadline {r['deadline'] or 'not given'}",
                  f"  why: {r['reasons']}", f"  {r['url']}", ""]
    if fixes:
        lines += ["Corrections (avis rectificatifs):", ""]
        for r in fixes:
            lines += [f"* {r['title']} (corrects {r['corrects'] or '?'})",
                      f"  {r['change'] or 'see the updated notice'}", f"  {r['url']}", ""]
    others = [r for r in rows if r["kind"] not in ("new", "correction")]
    if others:
        lines += ["Other notices (results etc.):", ""] + [f"* [{r['kind']}] {r['title']}  {r['url']}" for r in others] + [""]
    lines.append(attribution(fetched_at))
    return "\n".join(lines)


def main() -> None:
    ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
    ap.add_argument("--from", dest="date_from", help="first publication day, YYYY-MM-DD")
    ap.add_argument("--to", dest="date_to", help="end of window, YYYY-MM-DD (exclusive)")
    ap.add_argument("--yesterday", action="store_true", help="window = yesterday (Paris calendar day)")
    ap.add_argument("--days", type=int, help="window = the last N full days before today (Paris)")
    ap.add_argument("--nature", nargs="*", default=DEFAULT_NATURES,
                    help="server-side filter on nature_libelle (default: tenders + corrections)")
    ap.add_argument("--cpv", nargs="*", default=[], help="CPV prefixes, e.g. 72 48 (IT), 45 (works)")
    ap.add_argument("--keyword", nargs="*", default=[], help="French keywords, accent- and case-insensitive")
    ap.add_argument("--exclude", nargs="*", default=[], help="drop notices whose title contains any of these")
    ap.add_argument("--dept", nargs="*", default=[], help="département codes, e.g. 75 92 2A 971")
    ap.add_argument("--procedure", nargs="*", default=[], help="type_procedure, e.g. OUVERT PROCEDURE_ADAPTE")
    ap.add_argument("--market", nargs="*", default=[], help="SERVICES, FOURNITURES and/or TRAVAUX")
    ap.add_argument("--state", help="JSON file of notices already reported; only new ones are output")
    ap.add_argument("--save-raw", help="also save the raw API records to this JSON file")
    ap.add_argument("--from-file", help="replay records from a --save-raw file instead of calling the API")
    ap.add_argument("--csv", default="boamp.csv")
    ap.add_argument("--text", default="boamp.txt", help="plain-text summary, ready to paste into an email")
    args = ap.parse_args()

    today = datetime.now(PARIS).date()
    if args.yesterday:
        start, end = today - timedelta(days=1), today
    elif args.days:
        start, end = today - timedelta(days=args.days), today
    elif args.date_from:
        start = date.fromisoformat(args.date_from)
        end = date.fromisoformat(args.date_to) if args.date_to else start + timedelta(days=1)
    elif not args.from_file:
        ap.error("give --yesterday, --days N, --from/--to or --from-file")
    window = f"{start} to {end - timedelta(days=1)}" if not args.from_file else args.from_file
    fetched_at = datetime.now(timezone.utc).isoformat(timespec="seconds")

    if args.from_file:
        records, pages = json.loads(Path(args.from_file).read_text(encoding="utf-8")), 0
    else:
        s = requests.Session()
        s.headers.update({"User-Agent": USER_AGENT, "Accept": "application/json"})
        records, pages, day = [], 0, start
        while day < end:
            got, n = fetch_day(s, day, args.nature)
            print(f"{day}: {len(got)} records on {n} page(s)")
            records += got
            pages += n
            day += timedelta(days=1)
            if day < end:
                time.sleep(PAUSE_BETWEEN_PAGES)
    if args.save_raw:
        Path(args.save_raw).write_text(json.dumps(records, ensure_ascii=False), encoding="utf-8")

    rows = {r["idweb"]: r for r in map(flatten, records)}          # idweb is unique; guards overlapping windows
    print(f"{len(records)} records, {pages} request(s) -> {len(rows)} distinct notices "
          f"({sum(r['kind'] == 'correction' for r in rows.values())} corrections)")

    seen = load_state(Path(args.state)) if args.state else {}
    out = []
    for r in rows.values():
        if r["idweb"] in seen:
            continue
        reasons = matches(r, args)
        if not reasons and r["kind"] == "correction" and r["corrects"] in seen:
            reasons = [f"corrects {r['corrects']}, which you were sent before"]
        if reasons:
            r["reasons"] = "; ".join(reasons)
            out.append(r)
    out.sort(key=lambda r: (r["kind"] != "new", r["deadline"] or "9999"))
    print(f"{len(out)} to report")

    if args.state:
        seen.update({r["idweb"]: r["published"] for r in out})
        save_state(Path(args.state), seen)
    fields = list(out[0].keys()) if out else list(flatten({"idweb": ""}).keys()) + ["reasons"]
    with open(args.csv, "w", newline="", encoding="utf-8-sig") as f:   # BOM: Excel then shows accents
        w = csv.DictWriter(f, fieldnames=fields)
        w.writeheader()
        w.writerows(out)
    Path(args.text).write_text(email_text(out, window, fetched_at), encoding="utf-8")
    for r in out[:10]:
        print(f"{r['kind']:<10} {r['deadline'][:10] or '-':<10}  {r['title'][:70]}  {r['url']}")
    print(attribution(fetched_at))


if __name__ == "__main__":
    main()
Enter fullscreen mode Exit fullscreen mode

Data: BOAMP, Direction de l'information légale et administrative (DILA), from https://boamp-datadila.opendatasoft.com/explore/dataset/boamp/ (retrieved 17 October 2026), reused under the Licence Ouverte / Open Licence v2.0. Independent tutorial, not affiliated with the DILA or BOAMP.

AI helped write this article. All figures come from our own live API runs on 17 October 2026 (publication days 15 and 16 October); the API, licence and scheduling details were checked against the official docs listed above.

Top comments (0)