DEV Community: Odilon HUGONNOT

Offline PWA: the pitfalls of an app that never asks for the network

Odilon HUGONNOT — Thu, 17 Sep 2026 09:00:02 +0000

In the previous article, I told the story of why I built Mes mots, a pictogram communication app for a kid who doesn't talk yet. One of the project's constraints is that it has to work in airplane mode, on a tablet that might not see wifi for days. This article is only about that: how you build something truly offline, and above all how it still gets updated without a permanent connection.

"Offline-first", in most projects that carry the label, means something different from what it means here. Worth clarifying before getting into the code.

The misunderstanding around "offline-first"

Most apps labeled offline-first actually have a server behind them. A Trello, a Google Docs, a mail client: they cache, let you write offline, then sync and reconcile conflicts once the network comes back. That's a real engineering problem, conflict resolution, vector clocks, three-way merges, and a good chunk of the writing on the topic is about exactly that.

Mes mots doesn't have that problem, because it doesn't have a server at all. There's nothing to sync, because there's no one on the other end. The only thing that ever "arrives" from the network, occasionally, is a new version of the app's own code, not data. That's a much rarer category: not offline-first with a network fallback, but offline, period, with an occasional update.

That removes a whole class of problems (no conflicts, no auth token to refresh, no partial sync state to display). It brings in another one, narrower and stranger: getting a code update to arrive without ever disturbing whatever the user is doing right now, while having literally no way to talk to them other than through the screen they're already using.

What needs caching, and the file everyone forgets

The project uses the vite-plugin-pwa plugin, which generates a Workbox-powered service worker from a simple declarative config. The starting point is the list of files to precache:

workbox: {
  // phrase MP3s must be cached: without them the app is mute offline
  globPatterns: ['**/*.{js,css,html,png,svg,woff2,mp3}'],
  // a previous version's cache has already served a bundle gone from disk, which
  // silently hid a shipped feature: the family would stay on an old version
  // without knowing it, with no way to notice
  cleanupOutdatedCaches: true,
},

The trap lives in a forgotten extension. The {js,css,html,png,svg,woff2} pattern catches everything a typical frontend build produces. Except this app talks: every tile speaks its word by playing an .mp3 file. Without mp3 in the list, the app installs perfectly fine, opens offline with no error, tiles render, everything looks normal. And nothing comes out of the speaker. No exception in the console, no error screen: just a silence a parent would discover in airplane mode, at the worst possible moment.

That's the nature of a caching bug in a PWA: it doesn't break anything, it quietly removes a feature. The only safety net is listing, explicitly, everything the app needs to function without a network, not just the code.

A root domain, a subfolder, and a blank screen with no error

The app runs in two places: on the family's tablet, served at the root of a domain (anime-sanctuary.net), and as a public demo on GitHub Pages, under /mes-mots/. A service worker and its precached assets are scoped to a specific base URL, and a PWA built for the root looks for its files one level too high when served from a subfolder.

/**
 * The tablet serves the app at its domain root. The public demo lives under
 * `/mes-mots/` on GitHub Pages, and without this prefix it would look for its files
 * one level too high: blank page, no readable error. `BASE_PUBLIQUE` exists only for it.
 */
const BASE = process.env.BASE_PUBLIQUE ?? '/'

One environment variable, read once at build time, decides between the two cases. The lesson here isn't the variable itself, it's the failure mode: a wrong base path doesn't throw a clean exception you can read in devtools, it produces a blank page. The kind of bug you can't debug remotely, on the tablet of someone with no terminal.

An update that arrives without being asked for

registerType: 'autoUpdate' in the plugin config means the service worker downloads the new version as soon as it sees network, installs it in the background, and takes over on its own, no "an update is available" banner to click. What that mode doesn't tell you is that the page already open keeps running the old JavaScript until a full reload.

The browser exposes an event to find out: controllerchange, fired when a new service worker has just taken control of the page.

/**
 * The new service worker takes over on its own, but the displayed page keeps running
 * the old one until a reload: it took two loads to see a fix, and the listener only
 * lived in the parents screen, absent from the child's screen. Here it starts at
 * boot. `controller` doesn't exist on the very first install, which would otherwise
 * look like an update.
 */
const updateReady = ref(false)
if (navigator.serviceWorker?.controller) {
  navigator.serviceWorker.addEventListener('controllerchange', () => {
    updateReady.value = true
  })
}

The comment tells a real fixed regression: this listener originally lived in the parents screen, never visible from the child's screen, the one that actually runs all day. As a result, an update taken by the service worker was only detected if a parent opened their space, which could take days. The guard on controller avoids, in turn, confusing the very first install (where no controller exists yet) with an actual update.

Never reload a tile mid-speech

Detecting that an update is ready doesn't say when to apply it. Reloading the page immediately would cut a word mid-speech, erase a phrase being composed on the sentence bar, or lose a parent's half-filled form in the settings screen. For a kid who doesn't understand why the screen changed, each of those is a bad surprise.

The fix is defining an idle state and only applying the update then:

/**
 * The reload waits until the tablet isn't serving anyone: otherwise it would cut off
 * a word mid-speech, erase a phrase being composed, or lose a parent's half-filled
 * form. At rest, it goes unnoticed.
 */
const tabletIdle = computed(
  () =>
    mode.value === 'child' &&
    speakingTileId.value === null &&
    !isReadingPhrase.value &&
    phrase.value.length === 0,
)
watch([updateReady, tabletIdle], () => {
  if (updateReady.value && tabletIdle.value) location.reload()
})

Four conditions have to be true at once: the child screen is shown (not the parents space), no tile is currently speaking, the phrase bar isn't being read back, and the composed phrase is empty. The reload waits patiently for all four to turn green, and only fires then. On an app that runs all day and never gets closed, that moment always ends up arriving, usually within minutes.

It's a rule that generalizes well beyond this project: a silent background update is harmless for a blog or a dashboard you can refresh without thinking. It turns dangerous the moment the page's state has value, a form in progress, media mid-playback, unsaved input. At that point you need an explicit safe point, not just a timer.

Knowing which version is running, without opening devtools

Last detail, tiny in the code, decisive in practice: every build prints its own build date.

/** Build date, shown in the parents screen: "which version is running?" should be
 *  answered by looking at the screen, not by digging through a service worker cache. */
const BUILD_VERSION = new Date().toISOString().slice(0, 16).replace('T', ' ')

A developer diagnoses a caching problem by opening the Application tab in devtools. A parent will never open that tab, and shouldn't have to. Showing the version in plain sight in the parents screen turns a debugging question ("did the service worker actually pick up the latest version?") into one anyone can answer just by looking at the screen.

Conclusion

The network, in web development, we've learned to be wary of it: timeouts, retries, loading states, a whole vocabulary built around its absence. The silence of an update arriving unseen, much less so. Building an app that never asks for the network doesn't remove complexity, it moves it to a place you rarely think about: the exact moment it becomes safe to swap the ground under the feet of someone who's currently standing on it.

Web security — free interactive course

Odilon HUGONNOT — Thu, 17 Sep 2026 08:45:31 +0000

Web security

Current with the OWASP Top 10 2025 (Injection moved to A05, most courses still teach the old ranking), with real sandboxed attack labs, not just theory.

Free interactive web security course in 14 lessons. OWASP Top 10 2025, SQL injection, XSS, CSRF, CORS, cookies, CSP. PHP, Go and JS examples with safe challenges. No signup.

Free, interactive, no signup: https://www.web-developpeur.com/en/learning/securite/

Coding with AI — free interactive course

Odilon HUGONNOT — Thu, 17 Sep 2026 08:44:18 +0000

Coding with AI

Not a language course: it's the missing manual for coding alongside AI without losing control of the review, the debugging or the architecture.

Learn to use AI as a development tool, not a crutch. Effective prompts, code review, debugging, architecture, agents and MCP. Free interactive lessons.

Free, interactive, no signup: https://www.web-developpeur.com/en/learning/coder-avec-ia/

An AAC app for a kid who doesn't talk yet

Odilon HUGONNOT — Wed, 16 Sep 2026 09:00:02 +0000

Someone I know has a kid who doesn't talk yet, and can't read yet either. The diagnosis is autism. What works for him, right now, is a tile you touch with a finger, and it says the word for him. I got asked whether I could build that.

Serious AAC (augmentative and alternative communication) apps already exist. Proloquo2Go, TouchChat, LAMP Words for Life: vocabularies of several thousand words, built by speech therapists over years, priced between 150 and 300 dollars on iOS, up to 550 pounds for Grid 3 on desktop. Free and open alternatives exist too, CBoard (backed by UNICEF) and AsTeRICS Grid (University of Applied Sciences Vienna), both serious and actively maintained. None of that was missing from the world. What was missing was a simple grid, right now, for one specific kid, with nothing to get through before a finger can hit a tile.

I spent a weekend, then several weeks, building Mes mots: Vue 3, TypeScript, no server, no account. The code is on GitHub under the AGPL license. Here's what building it taught me, and why it doesn't look like a typical AAC app.

A tile that never moves

The spec fit in a conversation, not a document. A kid who can't read yet learns by the position of his finger, not by the word printed under the picture. If the "eat" tile moves to a different corner from one day to the next because a word got added elsewhere, the kid starts over from zero. That constraint drove the whole board architecture: tiles get added, they never reorganize themselves.

That's not a claim dropped into a README. It's checked by tests, and by a mutation testing harness that makes sure those tests actually bite, more on that below.

Nothing leaves the tablet

Second constraint, just as non-negotiable as the first: no data leaves the device. The photos added to a tile are the family's own, not generic pictograms. The voices recorded are the family's own, a kid recognizes his mother's voice before he recognizes a word. Sending that to a server, even encrypted, even for a perfectly harmless service, was out of the question.

Concretely, everything lives in IndexedDB, four separate object stores accessed through a single local repository. The code comment spells out exactly what that means:

/**
 * Local configuration repository (requirement T1). Nothing leaves the tablet.
 *
 * Image and sound stores are keyed by tile id: a family photo or voice, never
 * the configuration itself, which only carries a text reference
 * (`perso/<tileId>`, see `customIdentifier` in planche.ts).
 */
interface SchemaMesMots extends DBSchema {
  config: { key: string; value: Configuration | ConfigurationV2 | Planche | number }
  images: { key: string; value: Blob }
  sons: { key: string; value: Blob }
  journal: { key: number; value: EntreeJournal }
}

No account, no sync, no automatic cloud backup. The trade-off, acknowledged and reminded by the app itself, is that the family has to back up by hand. The backup format isn't proprietary: it's an .obz file, the Open Board Format standard, readable by other AAC apps the day the family needs to switch tools.

The app runs as a PWA, installable and usable in airplane mode. That deserves its own article, the pitfalls of an app that never asks for the network, and how it still gets updated. That's next.

Vue 3 with zero UI libraries

No component library, no CSS framework. Every button, every tile, every transition is hand-written, in <style scoped> per component. Not out of purism: for a grid of about fifteen tiles and two screens (child, parents), a component library adds more code to maintain than time it saves, and it imposes its own default accessibility where this one needed to be designed for a specific use case. CSS you write yourself, you know exactly what it does and why.

A mutation harness, because a green suite proves nothing

611 Vitest unit tests, 234 Playwright end-to-end tests, across two profiles: an 800×1280 tablet and an iPhone 13 on Safari. On paper, comfortable coverage. Except a fully green suite only proves nothing broke, not that something is actually being checked.

A typical unit test calls a function, compares the result to what it expects, and turns green as soon as the call succeeds, even if the assertion barely checks anything. The only way to know whether a test actually bites is to deliberately break the code it's supposed to protect, and check that the test goes red. That's what mutation testing does, and that's what mutation.sh does in the project: one mutation per backlog task, not per file, pattern by pattern.

The comment at the top of the script explains why it exists: "at step E1 the audio interruption could have vanished entirely without a single check flinching." In other words, a full regression on interrupting audio, not a detail, an entire behavior, had slipped through more than 600 green tests. The matching mutation pattern caught it.

The script has its own set of traps to stay honest. The first one: a forgotten vite preview server on port 4173 would keep serving the old code on every mutation, and every mutation would "pass" as caught while the tests actually run against code that never changed.

# A forgotten server on the port would serve the old code and the mutation would pass as caught.
if ss -lptn 'sport = :4173' 2>/dev/null | grep -q LISTEN; then
  echo "  port 4173 already in use, end-to-end mutations would be skewed"
  exit 1
fi

The second trap is subtler: a mutation that breaks compilation leaves dist/ untouched, so vite preview keeps serving the old bundle. Without a vite build before every end-to-end attempt, the harness would blame the tests for missing the mutation, when it was really the server serving code that never got rebuilt.

What it doesn't do, and why that matters

No eye tracking, no switch access, it's used with a finger. No vocabulary of several thousand words built over years by speech therapists, the way Proloquo2Go or LAMP Words for Life carry. No grammar, no conjugation: you place words, you don't build complex sentences. No multi-device sync, by choice, not by oversight.

It's also not a medical device or a treatment. An AAC app is chosen together with a speech therapist, who knows the child and can say whether a grid of pictograms fits, and which ones. This app was written for one specific child, based on needs his family described. I'm publishing it because it might serve someone else, not because it would suit everyone.

Conclusion

The hardest part wasn't Vue, or IndexedDB, or even the mutation harness. It was resisting the urge to add more: more words, more options, more settings. A kid who can't read yet gains nothing from a grid of two hundred tiles. He needs ten, that never move.

The code is out there if someone else needs it. So is the demo.

My captcha never protected against spam, only against laziness

Odilon HUGONNOT — Tue, 01 Sep 2026 09:00:03 +0000

Back in March, a bot named RobertqueRy spammed a post about gRPC with an ad for online gambling in Bangladesh. I added a honeypot field, a CSRF token, an IP-based rate limit and a math captcha. In the conclusion of that article, I wrote a line I should have taken more seriously: "a dedicated bot that parses the HTML could bypass it." Five months later, two comments showed up on the article that explains how my comment system works. The target is almost funny. The payload wasn't: a car rental ad for Marrakech, hardcoded link included.

All four layers let the bot through. Not one crack, a clean bypass.

What actually failed

The honeypot is a hidden field only a bot fills in. Empty: the bot didn't fill it, it loaded the page and read the form correctly. The CSRF token changes every session and must match the one sent with the POST: present and valid, so the bot made a real GET request before its POST, like a normal browser. The rate limit allows 3 comments per IP every 10 minutes: only one comment posted, nothing to trigger. That left the captcha, my last line of defense, a single-digit addition displayed in plain text on the page.

That's where my March mistake came back to bite me. The captcha wasn't rendered as an image, wasn't obfuscated in any way. It was just two numbers and a + sign, in plain text, in the HTML:

<label for="bk-captcha">Anti-spam check: 6 + 1 = ?</label>

A bot that downloads the page and greps the digits doesn't even need to pretend it understands addition. It extracts the numbers, computes the sum, and posts. That's exactly the test I ended up writing myself to verify the fix, without thinking twice about it: a script that loads the page, pulls the CSRF token and the two captcha operands out with two lines of grep, computes the sum, and posts. Three minutes of bash on a Friday morning. If I can automate solving my own captcha in three minutes, a real spam bot automated it a long time ago.

The real problem: I was checking the wrong thing

Honeypot, CSRF, rate limit, captcha: all four layers answer the same question, "did a human fill out this form normally?" The bot that spammed me answered yes to all four. It loaded the page, left the trap empty, grabbed the right token, respected the rate limit, and solved a single-digit sum. From its perspective, it was a human visitor filling out a form once. What none of these layers ever asks is what the comment actually contains.

That's the gap. A whole system for verifying who the visitor is, and zero lines looking at what they're trying to publish. Two comments pushing car rentals in Marrakech and Cesme sailed through four security layers without a single one reading their content.

The real filter: judge the content, not the visitor

The site runs three near-identical comment forms (blog, book reviews, skill pages), each with its own handler file, deliberately duplicated rather than centralized in a shared lib: three small independent PHP files are cheaper to reason about than one shared abstraction. I added the same block to all three. Two rules:

// Anti-spam: external links and common spam-comment keywords
$linkText = $author . ' ' . $content;
$hasExternalLink = (bool)preg_match('/<a\b/i', $linkText);
if (!$hasExternalLink && preg_match_all('/https?:\/\/\S+|www\.\S+/i', $linkText, $urlMatches)) {
    foreach ($urlMatches[0] as $url) {
        if (!preg_match('/^(https?:\/\/)?(www\.)?web-developpeur\.com/i', $url)) {
            $hasExternalLink = true;
            break;
        }
    }
}
if ($hasExternalLink) {
    // reject: external link
}

$spamWords = ['rent a car', 'car rental', 'car hire', 'escort', 'casino', 'viagra', 'cialis', 'forex trading', 'crypto signals', 'seo services', 'backlink', 'loan offer'];
foreach ($spamWords as $spamWord) {
    if (stripos($content, $spamWord) !== false || stripos($author, $spamWord) !== false) {
        // reject: spam keyword
    }
}

No external links, none of the keywords that show up in 90% of comment spam. It's not elegant, it's not machine learning, it's a list of patterns describing the spam I actually received. And unlike the captcha, it asks nothing of the visitor: someone writing a real comment almost never needs to paste a link or say "rent a car."

The trap of a filter that's too strict

Almost never, not never. One reader, Domenico, once flagged a broken page by pasting the failing URL into a comment. A "zero links" filter would reject that bug report exactly the way it rejects a car rental ad, and I'd only find out if Domenico pushed back. That's the danger of a filter that's too broad: it doesn't warn you when it's wrong, it just silently rejects.

The fix lives in the same regex: links to web-developpeur.com itself stay allowed. The filter blocks whatever pulls the reader off the site, not whatever helps them report a problem on it. The distinction is one line of code, but I only thought of it because I had the real case in front of me.

Testing the fix with the bot's own trick

A passing php -l proves nothing about the form's actual behavior. To verify, I replayed exactly what a half-competent bot does: load the live page, pull the CSRF token and captcha operands out of it, compute the sum, and post a real comment with curl. Two runs: one comment with an external link, one clean.

PAGE=$(curl -s -c cookies.txt "https://www.web-developpeur.com/blog/deuxieme-spam-commentaires-liens-php")
CSRF=$(echo "$PAGE" | grep -oE 'name="csrf_token" value="[^"]*"' | sed -E 's/.*value="([^"]*)"/\1/')
# ... same captcha-reading move a bot would make

curl -s -b cookies.txt -D - "https://www.web-developpeur.com/blog/comment-handler" \
  --data-urlencode "content=Check my site https://spam-example.test for deals" \
  --data-urlencode "csrf_token=$CSRF" --data-urlencode "captcha=$SUM" | grep Location
# -> comment_error=Only+links+to+this+site+are+allowed...

The external link got rejected, the clean comment got published. That second test comment landed in the real production comment file, exactly like the bot's comment five months earlier. I deleted it by hand over FTP within the minute, before it sat there in front of real readers.

Takeaway

A plain-text captcha doesn't test whether you're human, it tests whether you can parse a DOM. I spent five months believing my four layers protected against spam, when they protected against one specific kind of bot: the one that posts blind without reading the page. Against a bot that reads it, they're just friction.

Real protection was never about proving a visitor is human. It's about looking at what they're trying to publish. That sounds obvious written down. It wasn't, until two car rental ads for Morocco reminded me.

📚 The full Web Security course, free and interactive

Telling real security from the appearance of security is the core of the Web Security course: OWASP, authentication, CSRF, access control, with runnable attack labs right in the page.

A news-watch bot, an 8,900-character prompt, and three rules that never reached the model

Odilon HUGONNOT — Wed, 19 Aug 2026 09:00:03 +0000

The item sits at the top of the feed. "Last trading day for the first US spot Bitcoin ETF to shut down." Tagged as verified fact. Three sources, all dated August 3rd.

It is August 17th.

Fourteen days between the most recent source and the event it announces. A shutdown can be postponed. A vote can be adjourned. And I had written a rule for exactly that case a week earlier, a rule that says in plain words never to copy a two-week-old announcement as if it were still true.

The setup, for anyone arriving cold

This site runs an automated watch. Every 24 hours a scheduled task wakes a Node script that calls a language model with web search access, asks it to sweep the last few hours of crypto and financial news, and demands strict JSON in return. That JSON feeds a public page: a list of items, each with a title, a summary, its sources and their publication dates.

The model does not decide alone what it brings back. It receives a prompt, meaning the instruction text sent on every call, here 8,900 characters acting as a specification: which categories to cover, which sources to favour, the expected output format, and freshness rules defining what deserves to enter the feed. The prompt also requires a certainty label on every item, from FAIT_VERIFIE when two independent primary sources agree, down to SPECULATIF for a hypothesis.

The part that matters here: this prompt is not a fixed text, it is a template. It holds markers in double braces that the script fills in right before the call. {{DATE}} becomes the current timestamp, {{FREQUENCY_HOURS}} becomes the number of hours in the cycle, {{PRICES}} becomes a block of prices pulled from an API. So the text the model receives is never exactly the one I read in my editor.

That gap is where the whole story happens.

The watch pipeline: scheduled task, template markers filled in, model call with web search, strict JSON, public page. The defect sits at the fill-in step. Scheduled task, every 24 h Template: markers get filled in the defect is here Model + web search Strict JSON, one object per item Public watch page

The model never sees the template, only the text produced at the fill-in step. Whatever goes wrong there shows up nowhere else.

The rule existed, and it did nothing

Here is the relevant passage, exactly as stored in the config:

If you keep an item whose event falls in this cycle but whose sources are
ALL older than {{FREQUENCY_HOURS}}h, you may NOT publish it as is.
Check that it still holds:
- search for a RECENT source confirming the event is happening ;
- if you find one, add it to the sources and keep FAIT_VERIFIE ;
- otherwise downgrade to PROBABLE, and say in the summary that the
  deadline has not been reconfirmed.

The rule offers two outcomes and the output honours neither. No recent source added, no downgrade, no caveat in the summary. The item ships as verified fact, with the confidence of something confirmed that very morning.

At this point the first reflex is always the same: the model disobeyed. Push harder, add more capitals, repeat the instruction in two places.

That reflex is what cost me the most time.

The expensive reflex: blaming the wording

I did what everyone does. I reread the prompt and found it poorly built, which it was: four separate blocks each declare themselves top priority, the freshness rule and the reconfirmation rule contradict each other on edge cases, and across 8,900 characters there is exactly one concrete example.

So I rewrote it. Section tags instead of shouty headers, a priority block that explicitly ranks the families of rules, the freshness test turned into a numbered four-branch procedure, two worked examples including one deliberately downgraded item. From 8,900 to 11,800 characters, and a far clearer read.

Then comes the only question that matters: better, but measured how?

A prompt that returns JSON has the pleasant property of being gradeable without being read. The rule above translates into an executable test almost word for word:

const times  = item.sources.map(s => Date.parse(s.published_date)).filter(Number.isFinite);
const newest = Math.max(...times);
const stale  = newest < now - WINDOW_H * 3_600_000;
const hedged = /reconfirm|no recent source/i.test(item.summary);

// The rule: all sources outside the window => PROBABLE + explicit caveat.
const violation = stale && (item.certainty === 'FAIT_VERIFIE' || !hedged);

Six lines, and nobody's opinion on the beauty of the prompt carries any weight anymore. A run is now a number: how many items break the rule. All that is left is to run the old prompt and the new one, and compare.

Forty minutes for zero items

The old prompt goes first. Ten minutes later the subprocess is killed on timeout. Automatic retry, ten more minutes, killed again. The new prompt takes over and does exactly the same thing. Four calls, forty minutes, zero items compared.

At that point there are two ways forward. Shrug and say the API is slow today, or go and look. "The environment" is not a diagnosis, it is the name we give to whatever we have not measured.

The good news is that this kind of call can be watched closely. The CLI's stream mode emits one JSON event per step, and timestamping each line as it comes out is enough to see where the time goes:

claude --print --output-format stream-json --verbose \
       --tools 'WebSearch,WebFetch' -p "$PROMPT" \
| while IFS= read -r line; do printf '%s\t%s\n' "$(date +%s.%N)" "$line"; done > trace.jsonl

A small script then aggregates the gaps between events. Verdict:

total: 618.9s across 492 events
   458.8s  74%  model generation between tools
    81.0s  13%  tool results
    70.7s  11%  writing the final answer
tool calls: 20 WebSearch, 6 WebFetch
longest wait outside the final answer: 7.6s

No holes. No sixty-second pause that would betray a backoff, and every quota event says the call is allowed. The closing event gives the rest: 343 seconds of API time out of 619 seconds of wall clock, the remainder being web search latency, roughly ten seconds per call across twenty-six calls.

The job is simply long. The ceiling was ten minutes, the previous ten cycles landed between 296 and 445 seconds, and the margin looked comfortable until a busy news day ate it. The worst part is the retry design: it starts from scratch. A run killed at 599 seconds does not resume at 599, it restarts at 0 and gets killed a second time. Paying twice to fail twice.

Ceiling raised to fifteen minutes. The next cycle came in at 836 seconds, precisely the zone where the old setting guaranteed an empty feed.

The bug was one word

That leaves the original question, the ignored rule. And here is a reflex worth its weight in gold: do not reread the prompt in your editor, look at what actually left the machine. The process is running, its arguments are readable.

ps -eo pid,etimes,args | grep 'claude --print'

And in the text scrolling past, this:

A news item = a recent EVENT within the last {{FREQUENCY_HOURS}} hours.

The placeholder is still there. Not substituted. The model received, word for word, a rule with a hole where its threshold should be.

The cause is the following line, which looks perfectly innocent:

prompt = rawPrompt
    .replace('{{DATE}}', new Date().toISOString())
    .replace('{{FREQUENCY_HOURS}}', String(frequencyHours))
    .replace('{{DEDUP_HINT}}', dedupHint);

In JavaScript, String.prototype.replace called with a string pattern replaces only the first occurrence. You need a regular expression with the global flag, or replaceAll, to reach the others. It has been in the spec forever, it throws no error, no linter warning, and it goes unnoticed as long as a template holds a single occurrence of each marker.

This prompt held four of that particular marker. One substituted, three untouched.

The template holds the same marker four times. Substitution fills only the first: the next three rules reach the model with a hole where their threshold should be. The prompt template What reaches the model Cycle mission {{FREQUENCY_HOURS}} Item freshness {{FREQUENCY_HOURS}} No recent article {{FREQUENCY_HOURS}} Deadline reconfirmation {{FREQUENCY_HOURS}} replaced untouched untouched untouched Cycle mission 24 hours Item freshness {{FREQUENCY_HOURS}} No recent article {{FREQUENCY_HOURS}} Deadline reconfirmation {{FREQUENCY_HOURS}} .replace() with a string pattern: first occurrence only

Only one of the four rules gets its threshold. The other three arrive with a hole, and a rule without a threshold filters nothing.

That detail changes the whole diagnosis. The model never disobeyed. It applied to the letter a text I had not read, because I was rereading the one in my editor rather than the one going over the wire. Pushing harder, adding capitals, repeating the instruction: none of it would have worked, because the problem was never the wording.

The same threshold lived in three places

While fixing this I ran into the cousin of the same problem. The prompt hardcoded a 48-hour freshness window. The code computed it:

const windowH = Math.max(frequencyHours * 1.5, 48);

For a 24-hour cycle both land on 48. By luck, not by construction. Change the watch frequency and the two silently diverge without anything breaking: the prompt keeps announcing a threshold the pipeline no longer applies.

And there was a third copy, in the log line that recomputed the formula on its own to display it. As a result the logs announced a 36-hour threshold while the code applied a different one. Debugging from logs that lie costs you an evening.

The fix is not clever, and that is the point. One exported function owns the formula, the code uses it, and the prompt receives it through a dedicated marker instead of copying it by hand. A constant that code and prompt must agree on needs a single owner, exactly like between two modules.

What measurement did to my rewrite

Once substitution was repaired, the comparison became possible. Three runs, the same six-line judge, the same day:

                          items   violations   duration
old prompt, as is             8         2         703 s
old prompt, fixed             9         0         836 s
my rewrite                    4         0         680 s

The second row says it all. The one-word fix drops violations to zero, on the prompt I had judged badly written. My rewrite improves nothing on that criterion and returns half as many items, on the same news cycle.

The reason sits in my own text. I had added a priority block ranking volume last, plus a sentence saying five solid items beat fifteen padded ones. The model obeyed. On a daily feed where nine sourced items were perfectly legitimate, it threw five away to please me.

My prompt was better to read and worse to execute. I spent an hour writing the solution to a problem that did not exist, and the only piece worth keeping is the two worked examples: they are why the model produced a downgraded item flagged as probable with an explicit note that the deadline had not been reconfirmed. The rest goes in the bin, including the block I was proudest of.

What I take away

A prompt is code. Nobody judges a function by reading it aloud, you run it and look at what comes back. Yet put a prompt in front of people and everyone starts debating style and phrasing as if it were a cover letter.

The three moves that actually produced knowledge here are utterly mundane. Read what goes over the wire rather than what sits in the editor. Timestamp the events rather than assume things are slow. Write six lines of automated judge rather than wonder whether it is better.

And one more, the least pleasant: accept that measurement can invalidate your work. My rewrite was objectively clearer, better structured, nicer to read. It was also worse. Without the judge I would have shipped it, sincerely convinced I had improved the system.

When a model ignores a rule, the first question is not "how do I word this better". It is "did the rule arrive".

A hacked WordPress, two attackers, and the hole no antivirus saw

Odilon HUGONNOT — Wed, 12 Aug 2026 09:00:03 +0000

The site is down. Blank page, HTTP 200, no PHP error in the logs. Nothing to grab onto.

Googlebot, meanwhile, gets full pages. Links to counterfeit shops, indexed for three days under the client's name. The site is dead for humans and very much alive for crawlers. That is the worst case: nobody sees the content dirtying the domain, and the domain gets dirty anyway.

The antivirus has just finished its sweep. 373 files scanned, zero detections.

It took three days to untangle: where they came in, what they left behind, and what had to change so none of them could do it again. The client, the domain and the addresses are anonymised. The rest is exact, including the times I chased the wrong lead. Those are usually the useful parts.

The blank page was not the problem

The first file I open is index.php. It has indeed been swapped for a trojaned version. Perfect suspect, case closed.

Except that reading the code properly, the trojan never stops for a normal visitor. It serves the expected page and only deviates for search crawlers. Cloaking: one content for Google, another for you. That file had no reason to produce a blank page.

Wrong lead. The real cause was dumber, and more brutal: the wp-content directory had been renamed. Plugins, theme, media, everything WordPress looks for in there had become unreachable. The core booted, found no theme, rendered nothing. A whole site killed by an mv.

Which leaves the question of why an attacker would sabotage the site he is milking. It makes no sense. A dead site earns nothing, and it brings the administrator running. That is exactly what you avoid when you sell links.

The logs answered. There were two of them, and they hated each other.

The second group in had dropped an .htaccess allowing only its own backdoors. The earlier group's were locked out. That group came back a few hours later and took two 403s on its own tools, on a server it considered its own. Its answer was to re-drop its backdoor, then rename wp-content. Three seconds separate the two actions in the log.

So the sabotage was not aimed at the client. It was a message to a competitor. The site was only the ground they fought on.

Eighteen seconds

Which left the question that brawl had hidden: how did they get in?

To find out, you do not look for odd files. You look for what happened right before the first odd file appeared. An attacker deletes his files easily enough. The request that created them, far less so.

So I timestamped every suspicious file, kept the oldest, and walked the access log back to that minute. Two lines are enough.

04:04:35  POST /?wpmudev-hub=<token>   200
04:04:53  GET  /<backdoor>.php          200

Eighteen seconds between the call to the plugin and the first request to the webshell. Nobody types a random URL and lands on a file that did not exist twenty seconds earlier.

That wpmudev-hub parameter belongs to the WPMU DEV Dashboard, a multi-site management plugin. Versions up to 5.0.0 carry an authentication bypass, tracked as CVE-2026-15459. It only opens in one case: when the service API key was never filled in. The plugin then accepts an anonymous call, and lets you install another plugin. Installing a plugin means running code. A hole does not need to be subtler than that.

Two details turned the hypothesis into a diagnosis. In the database, the wpmudev_apikey option was empty: the vulnerable state, exactly. And the nonce the plugin had stored matched the suffix of the attack URLs. This was no longer a plausible story. It was the trace of the exploitation, written by the plugin itself.

Nine successful exploits, nine different addresses, the last one three days after the first. Nobody had bothered to close the door behind them.

Intrusion timeline: anonymous call to the plugin, webshell eighteen seconds later, self-repair kit, cloaking, nine exploits over three days, then sabotage between rival groups. 04:04:35 The plugin accepts an anonymous call CVE-2026-15459, API key never filled in + 18 s The dropped webshell already answers the server now runs their code then Six files disguised as images and scripts the kit that puts the backdoor back then index.php swapped: spam for the crawlers the human visitor sees nothing 3 days Nine exploits, nine addresses the hole stays open the whole time finally Retaliation between groups: the site falls wp-content renamed, blank page

Three days between the door left open and the outage that finally revealed it.

What the antivirus missed, and what I missed

Finding the door is one thing. Finding everything they left behind is another. And there, my tools failed me one after the other.

ClamAV first: 0 detections across 373 files. That is not a flaw in the product. It is the ceiling of signature matching, against code written to look like nothing.

Next reflex, grep base64_decode. Nothing either. Sensitive function names are assembled character by character at runtime, and command URLs are encoded. There is no string to find: the file does not contain the words that would give it away, it builds them.

Then the modification times. They had been backdated to blend into the original install. A file dropped the previous week proudly showed the same date as the rest of WordPress core. Only the inode change time holds, the ctime, because PHP cannot rewrite it. That is what dated the intrusion to the second. Without it I would still be looking.

The neatest piece was the self-repair kit. Six files disguised as images and scripts, with names built to survive a quick review: server-sied-renderr.min.js, icon_twistedd.png. You read fast, you see a minified script and an icon, you move on. Each actually held a byte-for-byte copy of the trojaned index.php and its .htaccess. Fake wp-login.php files put them back. Delete the backdoor without deleting the kit and you watch it return.

And I missed a file on the first pass.

To sweep recently modified files, I had excluded wp-content/cache and its 78,000 legitimate files. An exclusion of convenience, made so the command would return something readable. A 468-byte remote loader had survived in there, with its own .htaccess allowing it. I found it on the second pass, the one I ran with no exclusions at all, assuming the first had failed.

That is the lesson I keep from the whole episode. An exclusion in a sweep is a hiding place you hand over. A directory too big to search does not need an exemption, it needs its own check.

Clean up, then make cleaning up pointless

Once the inventory is genuinely complete, cleanup becomes the boring part. Good.

WordPress core was compared file by file against the official archive of the same version, and every difference replaced. Exactly one was legitimate: the file carrying the install's language variant. Plugins were not cleaned but reinstalled from their official source. That is faster and safer, because nobody can review thirty plugins by hand. The custom theme could not be replaced. It was checked line by line. It was untouched.

At that point the site is clean. It is also exactly as vulnerable as before.

That is what most remediations forget: cleaning repairs the past, it writes nothing about the future. The part that counts starts here.

The web server can no longer write a single line of code. The whole tree belongs to root, the web server group is read-only, directories are 750 and files 640. A few directories have to stay writable, of course, since WordPress drops media and cache there. For those, PHP execution is denied at the Apache level:

<DirectoryMatch "…/wp-content/(uploads|cache|upgrade)(/|$)">
    <FilesMatch "\.(?i:php|phtml|php[0-9]|pht|cgi|pl|py|sh)$">
        Require all denied
    </FilesMatch>
</DirectoryMatch>

On one side the places you can write to, on the other the places where PHP runs. The intersection is empty, and that is the whole point.

Two disjoint sets: directories the web server can write to on one side, directories where PHP runs on the other. No directory belongs to both. Web server can write PHP runs uploads/ cache/ upgrade/ WordPress core plugins theme ∅ intersection

Nowhere to drop code and run it.

Replay the 04:04:35 attack against this setup. The vulnerable plugin still accepts the anonymous call. It still tries to write its file. It fails. The hole is intact, the exploitation is not.

The config that closes what was left

Banning writes breaks one thing along the way, and not a small one: WordPress can no longer update itself. The DISALLOW_FILE_MODS constant makes that block explicit, and removes plugin installation from the admin UI in the same move.

That is deliberate, but it creates a new risk. A site that stops receiving patches becomes a different problem, and frankly a worse one than the one just fixed.

The compensation is a weekly job run by root. It flips the constant, dumps the database, updates, restores permissions, verifies core checksums, tests that the site still answers, rebuilds the monitoring baseline, and emails a report. The block holds against an intruder, and patches still land. It is the only arrangement I found that sacrifices neither.

The rest of the config comes down to three questions.

Who is still logged in? Salts were regenerated. Every open session drops, the attacker's included.

Who can still trigger code? WordPress's internal cron was disabled in favour of a system job. The internal one depends on traffic: it fires from outside, on a plain visit. That is a door that opens on request.

What is still visible from the street? The login URL moved. The file editor is off, admin forced over HTTPS, TLS 1.0 and 1.1 refused. Files that talk too much are denied: debug logs, readme, composer.json, anything shaped like a .env. And a fail2ban jail bans on the first attempt against an exploitation pattern.

No Content Security Policy, though, and I stand by that. The site loads a tag manager, two ad networks, a map and an external booking widget. A CSP written blind breaks the site silently. A CSP permissive enough to allow everything protects nothing while ticking the box in the audit. Better no CSP than a fake one. It is a project, not a config line.

On secrets, the rule was simple: anything readable during the compromise is lost. Admin passwords, database credentials, SSH keys regenerated. One key unused for a year disappeared along the way. Third-party API keys were inventoried with their scope, to be rotated next.

Then I looked at the backups. They existed: one a week at the host, kept on a 28-day rolling window, plus one full server snapshot a year.

On paper that is reassuring. In this incident, much less so. The intrusion dates from 7 August and was only found on the 10th. With a weekly backup, the most recent one stood every chance of already containing the backdoor: restoring meant reinstalling the problem. And 28 days of retention leaves four restore points, when a compromise can sit quiet for months before showing itself.

A backup is only worth what you know about its contents. That is why a reference state was frozen separately, once the site was clean: site archive, database dump, plugin list with versions, and a file of 11,784 SHA-256 hashes that stands as provable evidence of a clean state at a given date. It is not one more backup, it is the only restore point known to be clean.

Along the way, four years of custom code that had never been versioned went into a Git repository, with an archive kept off the server. Until then, the only copy of the theme was the one running in production. On the machine that had just been breached.

Knowing before the attacker does

A hardened site with no monitoring is a site you do not know has fallen again. Hardening lowers the probability, it does not zero it. Monitoring is what closes the gap.

So a job runs every thirty minutes, and only sends mail when there is something to say. Silence is success.

First check, core integrity. It asks wordpress.org about the version actually installed. I first compared against an archive frozen on disk, which produced a fine false alarm on the very first update: a frozen reference goes wrong on its own, without anyone doing anything wrong.

Then comes the fingerprint. Roughly 11,500 files compared against a reference: every PHP file, every .htaccess, and the theme's scripts and stylesheets. The cache, too volatile to sit in a fingerprint, gets its own check. Exactly the one that would have saved me from missing the 468-byte loader.

Two more checks, dumb and effective: no executables in the media directory, and not one file in the must-use plugin directory. That last one is the handiest hiding place in WordPress, because nothing dropped there can be disabled from the admin UI.

In the database, a sentinel watches fourteen sensitive values: site address, admin email, registration flag, default role, active theme, active plugins, and the full list of accounts, their roles and their application passwords. Creating a quiet admin account is any intruder's first move for persistence. It now raises an alert within thirty minutes.

The last check requests the same page twice. Once as a browser, once as Googlebot, then compares response sizes. That is the cloaking signature. The very one serving spam while the site sat empty.

There was still the trap that kills every monitoring setup, and it is not technical: fatigue. The first version alerted on any exploitation pattern in the logs, which came to 856 lines a day. Almost all of them generic scanners raking the whole internet and taking 404s. A daily 856-line report, you read it twice, then you file it unopened.

I first sorted those patterns into two families, keeping only the ones aimed at this site. The noise dropped to almost nothing. Almost: twice a day a bot replayed the original hole, and the homepage answered it 200, since the plugin no longer exists. Two daily alerts for an attack that had become impossible.

So we cut harder. Mail now only goes out when something succeeded: a file appeared, a file changed, a database value moved, the site went down. Attempts are still logged on the server, available for an investigation, but they no longer wake anyone up.

Two columns: on the left what is only logged, the attempts; on the right what sends mail, the real consequences. What stays in the log an exploit URL tried a scan of /.env, /xmlrpc.php a 404 on an old backdoor no mail What sends mail a PHP file added or changed a sensitive database value moved the vulnerable plugin reappearing the site going down or cloaking mail straight away Alert on the outcome, never on the intent.

An attempt teaches nothing, they arrive by the thousand. A file that moves does.

And the intrusion vector is no longer watched in the logs. What is watched is the condition that makes it possible: if the vulnerable plugin ever shows up again in the plugin directory, the alert fires. We no longer watch for someone trying the door. We watch that the door is still bricked up.

An alert that fires every day for nothing stops being read. And that is the day the real one slips through.

The whole stack, ready to copy

Everything above fits into three scheduled jobs and two scripts. Here they are in full, anonymised: paths, domain, database and addresses are replaced with example values, the rest is the code that actually runs.

The cron first. Nothing exotic, and that is the point: a monitoring system you cannot read at a glance will never get debugged on the day it gets something wrong.

# /etc/cron.d/surveillance
# Monitoring: every 30 minutes
*/30 * * * *  root  /root/surveille.sh       >/dev/null 2>&1
# Updates: Tuesday 04:17, off-peak
17   4 * * 2  root  /root/maj.sh             >/dev/null 2>&1
# Antivirus: Sunday 03:17
17   3 * * 0  root  /root/scan-antivirus.sh  >/dev/null 2>&1

Then the updates. This script exists only because DISALLOW_FILE_MODS stops WordPress from updating itself. It lifts the lock, works, puts it back, verifies, and reports. Two details matter. The trap on line 53, without which a crash halfway through would leave the site writable until the following week. And the guard on line 30: if the database dump fails, nothing gets updated. A safety net you believe is there and is not is worse than no net at all.

#!/bin/bash
# WordPress updates driven by the system, not by WordPress.
# DISALLOW_FILE_MODS stops WordPress from updating itself. That is
# deliberate: www-data must not write to core. This script does it
# instead, as root, with checks before and after.
SITE=/var/www/example.com/public
STATE=/var/lib/site-watch
DEST=alerte@example.com
FROM=surveillance@example.com
LOG=/var/log/maj.log
cd "$SITE" || exit 1

dispo=$(wp plugin list --update=available --field=name --allow-root 2>/dev/null)
coeur=$(wp core check-update --minor --field=version --allow-root 2>/dev/null | grep -E "^[0-9]+\." | head -1)

if [ -z "$dispo" ] && [ -z "$coeur" ]; then
    echo "$(date -u '+%F %T') rien a mettre a jour" >> "$LOG"
    exit 0
fi

RAPPORT="Mises a jour appliquees sur example.com le $(date -u '+%F a %H:%M UTC').
"

# Dump the database before touching anything. Sans elle on ne met rien a jour :
# un filet de securite qu'on croit tendu et qui ne l'est pas vaut moins que pas
# de filet du tout, parce qu'on prend le risque en pensant etre couvert.
mkdir -p /root/backups
DUMP="/root/backups/db-avant-maj-$(date -u +%Y%m%dT%H%M%S).sql.gz"
set -o pipefail
mysqldump --single-transaction wordpress_prod 2>/dev/null | gzip > "$DUMP"
etat_dump=$?
set +o pipefail
if [ "$etat_dump" -ne 0 ] || [ "$(stat -c %s "$DUMP" 2>/dev/null || echo 0)" -lt 100000 ]; then
    rm -f "$DUMP"
    printf '%s' "Mise a jour ANNULEE sur example.com le $(date -u '+%F a %H:%M UTC').

La sauvegarde de la base a echoue, ou le fichier produit est trop petit pour
etre credible. Rien n'a ete mis a jour : on ne touche pas au site sans point
de retour. Verifier MySQL et l'espace disque, puis relancer /root/maj.sh." \
        | mail -s "[MAJ] ECHEC sauvegarde, mise a jour annulee" -a "From: $FROM" "$DEST"
    echo "$(date -u '+%F %T') sauvegarde impossible, mise a jour annulee" >> "$LOG"
    exit 1
fi
find /root/backups -name 'db-avant-maj-*.sql.gz' -mtime +30 -delete 2>/dev/null

# DISALLOW_FILE_MODS also blocks wp-cli, so we lift it for the operation.
# The trap puts the lock back whatever happens. Without it, a crashed wp or
# a reboot mid-run would leave the site writable until the next Tuesday,
# with nothing to signal it.
remettre_le_verrou() {
    sed -i "s/define( 'DISALLOW_FILE_MODS', false );/define( 'DISALLOW_FILE_MODS', true );/" "$SITE/wp-config.php"
}
trap remettre_le_verrou EXIT INT TERM
sed -i "s/define( 'DISALLOW_FILE_MODS', true );/define( 'DISALLOW_FILE_MODS', false );/" wp-config.php

if [ -n "$dispo" ]; then
    RAPPORT="$RAPPORT
## Plugins
$(wp plugin update --all --allow-root 2>&1 | tail -20)
"
fi
if [ -n "$coeur" ]; then
    RAPPORT="$RAPPORT
## WordPress core (current branch)
$(wp core update --minor --allow-root 2>&1 | tail -6)
$(wp core update-db --allow-root 2>&1 | tail -2)
"
fi

remettre_le_verrou

# Files written by root must stay readable by www-data
chown -R root:www-data "$SITE/wp-admin" "$SITE/wp-includes" "$SITE/wp-content/plugins" 2>/dev/null
find "$SITE/wp-admin" "$SITE/wp-includes" "$SITE/wp-content/plugins" -type d -exec chmod 750 {} + 2>/dev/null
find "$SITE/wp-admin" "$SITE/wp-includes" "$SITE/wp-content/plugins" -type f -exec chmod 640 {} + 2>/dev/null

# Verification
RAPPORT="$RAPPORT
## Verification after the update
$(wp core verify-checksums --allow-root 2>&1 | tail -3)
$(wp plugin verify-checksums --all --allow-root 2>&1 | tail -2)
"
code=$(curl -s -o /dev/null -w '%{http_code}' --max-time 40 https://www.example.com/)
RAPPORT="$RAPPORT
Page d'accueil apres mise a jour : HTTP $code
"
[ "$code" != "200" ] && RAPPORT="$RAPPORT
ATTENTION : le site ne repond pas normalement. Restauration possible depuis /root/golden/ et les dumps /root/backups/.
"

# The monitoring baseline must follow, or it will alert for nothing
/root/surveille.sh --rebaseline >/dev/null 2>&1
RAPPORT="$RAPPORT
Empreinte de surveillance reconstruite.
"

printf '%s' "$RAPPORT" | mail -s "[MAJ] example.com" -a "From: $FROM" "$DEST"
echo "$(date -u '+%F %T') mises a jour appliquees, HTTP $code" >> "$LOG"

And the monitoring. Ten checks, only one of which reads the logs, and that one never sends mail. Email is reserved for what succeeded: a file that appeared, a file that changed, a database value that moved.

#!/bin/bash
# Watches example.com. Sends mail ONLY when something is wrong.
# Run from cron. Prints nothing when all is well: silence is success.

SITE=/var/www/example.com/public
REF=/root/reference/wordpress
STATE=/var/lib/site-watch
DEST=alerte@example.com
FROM=surveillance@example.com
HOST=$(hostname)
mkdir -p "$STATE"
chmod 700 "$STATE"

ALERTS=""
add() { ALERTS="${ALERTS}
## $1
$2
"; }

# --- 1. WordPress core integrity ---------------------------------------------
# Ask wordpress.org about the version ACTUALLY installed: a frozen reference
# archive goes wrong on the very first core update.
if command -v wp >/dev/null; then
    chk=$(cd "$SITE" && wp core verify-checksums --allow-root 2>&1)
    if echo "$chk" | grep -q "^Success:"; then
        :   # coeur conforme
    elif echo "$chk" | grep -qiE "couldn't fetch|failed to|could not resolve|error establishing"; then
        :   # wordpress.org injoignable : on ne crie pas au loup
    else
        add "WordPress core integrity" "$(echo "$chk" | grep -vE '^(Success|Warning: Could not)' | head -30)"
    fi
else
    # Un controle qui disparait en silence est pire qu'un controle absent :
    # on croit etre surveille alors qu'on ne l'est plus.
    add "Controle du coeur impossible : wp-cli introuvable" "Reinstaller wp-cli, sinon l'integrite du coeur n'est plus verifiee."
fi

# --- 2. Any PHP file missing from the baseline fingerprint -------------------
# Baseline taken after cleanup. After a legitimate WordPress update,
# rebuild it with: /root/surveille.sh --rebaseline
BASE="$STATE/baseline-php.sha256"
if [ "${1:-}" = "--rebaseline" ]; then
    find /var/www/example.com \( -name "*.php" -o -name ".htaccess" -o -path "*/themes/montheme/*.js" -o -path "*/themes/montheme/*.css" \) -not -path "*/cache/*" -not -path "*/node_modules/*" -not -path "*/.git/*" -print0 2>/dev/null \
        | xargs -0 sha256sum 2>/dev/null | sort > "$BASE"
    echo "Reference reconstruite : $(wc -l < "$BASE") fichiers PHP"
    exit 0
fi
if [ -f "$BASE" ]; then
    current=$(mktemp)
    find /var/www/example.com \( -name "*.php" -o -name ".htaccess" -o -path "*/themes/montheme/*.js" -o -path "*/themes/montheme/*.css" \) -not -path "*/cache/*" -not -path "*/node_modules/*" -not -path "*/.git/*" -print0 2>/dev/null \
        | xargs -0 sha256sum 2>/dev/null | sort > "$current"
    drift=$(comm -13 "$BASE" "$current" | awk '{print $2}' | head -30)
    gone=$(comm -23 "$BASE" "$current" | awk '{print $2}' | head -10)
    rm -f "$current"
    [ -n "$drift" ] && add "PHP files added or modified since the baseline" "$drift"
    [ -n "$gone" ] && add "PHP files gone since the baseline" "$gone"
else
    add "Baseline fingerprint missing" "Lancer : /root/surveille.sh --rebaseline"
fi

# --- 3. Executables in uploads (never normal) --------------------------------
badup=$(find "$SITE/wp-content/uploads" -type f \( -iname "*.php*" -o -iname "*.phtml" -o -iname "*.cgi" -o -iname "*.pl" -o -iname "*.py" \) 2>/dev/null | grep -v "index.php$" | head -20)
[ -n "$badup" ] && add "Executable files in uploads/" "$badup"

# --- 3b. Unexpected PHP inside the cache -------------------------------------
# The baseline skips the cache (thousands of volatile files), which makes it
# an ideal hiding place: a backdoor survived the cleanup in there. W3TC only
# writes to the subdirectories listed below.
cachephp=$(find "$SITE/wp-content/cache" -name "*.php" 2>/dev/null \
    | grep -avE "/cache/(db|page_enhanced|minify|object|fragment|stats|tmp)/" | head -20)
[ -n "$cachephp" ] && add "Unexpected PHP file in the cache" "$cachephp"

# --- 4. mu-plugins must stay empty -------------------------------------------
mu=$(find "$SITE/wp-content/mu-plugins" -type f 2>/dev/null | head -10)
[ -n "$mu" ] && add "Files found in mu-plugins (auto-loaded, cannot be disabled)" "$mu"

# --- 4b. The plugin used as the entry vector must stay gone ------------------
# We no longer watch for the `wpmudev-hub` call in the logs: with the plugin
# uninstalled the parameter does nothing and the homepage answers 200 to every
# passing bot, i.e. two pointless alerts a day. We now watch the condition
# that would make the attack possible, not the attempt.
wpmu=$(find "$SITE/wp-content/plugins" -maxdepth 1 \( -iname "*wpmudev*" -o -iname "*wpmu-dev*" \) 2>/dev/null | head -3)
[ -n "$wpmu" ] && add "The WPMU DEV plugin is back (the intrusion vector)" "$wpmu
Verifier la version : les 5.0.0 et anterieures portent CVE-2026-15459."

# --- 5. Exploitation attempts: logged, NEVER emailed -------------------------
# An attempt is not an event, it is internet background noise: bots try every
# known URL on every site, all the time. Mail is reserved for what SUCCEEDED,
# meaning a new file, a modified file or a changed database value (checks 1,
# 2, 3, 3b, 4, 4b and 10). The trail stays in /var/log/tentatives.log for
# any later investigation.
# Only log what was never logged: otherwise the same events come back on
# every run for as long as they sit inside the window.
# TRUSTED: admin addresses, excluded so our own tests do not raise alerts.
TRUSTED="203.0.113.10 203.0.113.10 203.0.113.10"
SEEN="$STATE/last-exploit-epoch"
last_epoch=$(cat "$SEEN" 2>/dev/null || echo 0)
max_epoch=$last_epoch

# Two families of patterns, two different rules.
# CIBLE (targeted): what aims at this site, the entry vector and the webshells
# dropped back then. Logged even on failure (403, 404, 503): touching those
# means knowing the site's history. Two exceptions, where the request never
# reached WordPress: the http->https 301, which comes straight back over https
# and would be counted twice, and the 421 SNI mismatch.
# BALAYAGE (sweeps): scanners raking the whole internet, dozens a day, all of
# them 404s. Logged only if the server answered something other than a refusal.
# A daily alert about nothing stops being read, and that is the day the real
# one slips through.
# Deliberately absent: PHP-CGI argument injection (auto_prepend_file,
# allow_url_include). PHP runs as mod_php, so the homepage answers 200 to every
# attempt while none of them execute, verified with a marker that never came
# back. If such an injection ever landed, it would leave a file, and check 2
# is the one that would see it.
CIBLE="\?action=[A-Za-z0-9]{15,}|shell1\.php|shell2\.php|shell3"
BALAYAGE="eval-stdin|/bin/sh|/\.env|/\.git/|phpinfo|wp-config\.php[.~]|/shell|alfa-?rex|/xmlrpc\.php"
# Fichiers de travail dans $STATE (root, 700) et non dans /tmp : root y ecrit,
# et /tmp est inscriptible par www-data. Le noyau bloque deja le detournement
# par lien symbolique, mais un script de securite ne se repose pas sur un sysctl.
ACCES="$STATE/acces-du-jour.txt"

YDAY=$(date -u -d "yesterday" "+%d/%b/%Y")
TODAY=$(date -u "+%d/%b/%Y")
expl=""
while IFS= read -r line; do
    [ -z "$line" ] && continue
    ip=${line%% *}
    case " $TRUSTED " in *" $ip "*) continue ;; esac
    ts=$(printf '%s' "$line" | grep -oE '[0-9]{2}/[A-Za-z]{3}/[0-9]{4}:[0-9:]{8}')
    [ -z "$ts" ] && continue
    ep=$(date -u -d "$(printf '%s' "$ts" | tr ':' ' ' | awk '{print $1" "$2":"$3":"$4}' | sed 's|/| |g')" +%s 2>/dev/null || echo 0)
    [ "$ep" -le "$last_epoch" ] && continue
    [ "$ep" -gt "$max_epoch" ] && max_epoch=$ep
    expl="${expl}${line}
"
done <<EOF
$(
    grep -ahE "$YDAY|$TODAY" /var/log/apache2/*access*.log 2>/dev/null \
        | sed -E 's/^[a-z0-9.-]*lesmontheme\.com:[0-9]+ //' > "$ACCES"
    {
        grep -aiE "$CIBLE" "$ACCES" | awk '$9 !~ /^(301|302|421)$/'
        grep -aiE "$BALAYAGE" "$ACCES" | awk '$9 !~ /^(301|302|400|401|403|404|408|421|429|503)$/'
    } | awk '{printf "%-16s %s %s %s %s\n", $1, $4, $6, $7, $9}' | sed 's/\[//' | sort -u | head -40
    rm -f "$ACCES"
)
EOF
echo "$max_epoch" > "$SEEN"
if [ -n "$expl" ]; then
    {
        echo "===== $(date -u '+%F %T UTC') ====="
        printf '%s' "$expl"
    } >> /var/log/tentatives.log
    # Nobody prunes this file, so bound it here or it grows forever.
    if [ "$(stat -c %s /var/log/tentatives.log 2>/dev/null || echo 0)" -gt 5000000 ]; then
        tail -n 5000 /var/log/tentatives.log > /var/log/tentatives.log.tmp \
            && mv /var/log/tentatives.log.tmp /var/log/tentatives.log
    fi
fi

# --- 6. Is the site answering? -----------------------------------------------
# Second try before crying wolf: an Apache reload or a passing network
# hiccup must not raise an alert.
verifier_site() {
    : > "$STATE/derniere-page.html"
    curl -s -o "$STATE/derniere-page.html" -w "%{http_code}" --max-time 25 \
         -A "Mozilla/5.0 (surveillance)" https://www.example.com/ 2>/dev/null
}
code=$(verifier_site)
if [ "$code" != "200" ] && [ "$code" != "503" ]; then
    sleep 20
    code=$(verifier_site)
fi
size=$(stat -c %s "$STATE/derniere-page.html" 2>/dev/null || echo 0)
if [ "$code" != "200" ] && [ "$code" != "503" ]; then
    add "The site is not answering normally" "code HTTP = $code, taille = $size octets"
elif [ "$code" = "200" ] && [ "$size" -lt 5000 ]; then
    add "The site answers 200 but the page is nearly empty" "taille = $size octets (une page vide en 200 est exactement le symptome de l'incident d'aout 2026)"
fi

# --- 7. Cloaking: a bot and a human must get the same page -------------------
if [ "$code" = "200" ]; then
    sbot=$(curl -s -o /dev/null -w "%{size_download}" --max-time 25 \
        -A "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" \
        https://www.example.com/ 2>/dev/null)
    if [ -n "$sbot" ] && [ "$sbot" -gt 0 ] 2>/dev/null; then
        diff=$(( sbot > size ? sbot - size : size - sbot ))
        limit=$(( size / 5 ))
        [ "$diff" -gt "$limit" ] && add "Content differs by visitor (possible cloaking)" \
            "navigateur = $size octets, Googlebot = $sbot octets"
    fi
fi

# --- 8. Services and disk ----------------------------------------------------
for svc in apache2 mysql; do
    systemctl is-active --quiet "$svc" || add "Service stopped" "$svc"
done
use=$(df --output=pcent / | tail -1 | tr -dc '0-9')
[ "${use:-0}" -gt 85 ] && add "Disk space" "partition / occupee a ${use} %"

# --- 9. Result of the weekly ClamAV scan (read here, not run here) -----------
# The scan itself lives in /root/scan-antivirus.sh (weekly cron): too slow to
# run on every pass.
if [ -f "$STATE/clamscan-last.txt" ]; then
    hits=$(grep -a "FOUND" "$STATE/clamscan-last.txt" 2>/dev/null | head -20)
    [ -n "$hits" ] && add "ClamAV detections (last weekly scan)" "$hits"
fi

# --- 10. Database sentinel ---------------------------------------------------
# An attacker with database access can act without touching a single file:
# that is how the fake plugin got activated. So we watch the values that
# matter, not the whole database.
DBREF="$STATE/db-sentinelle.txt"
dbnow=$(mysql -N -r wordpress_prod 2>/dev/null -e "
SELECT CONCAT('option:', option_name, '=', LEFT(option_value, 400))
  FROM wp_options
 WHERE option_name IN ('siteurl','home','admin_email','users_can_register',
                       'default_role','template','stylesheet','active_plugins')
 ORDER BY option_name;
SELECT CONCAT('user:', u.ID, ':', u.user_login, ':', u.user_email, ':', IFNULL(m.meta_value,''))
  FROM wp_users u
  LEFT JOIN wp_usermeta m ON m.user_id = u.ID AND m.meta_key = 'wp_capabilities'
 ORDER BY u.ID;
SELECT CONCAT('apppass:', COUNT(*)) FROM wp_usermeta WHERE meta_key = '_application_passwords';
SELECT CONCAT('mu-plugin-option:', COUNT(*)) FROM wp_options WHERE option_value REGEXP 'eval\\\\(|base64_decode|gzinflate';
")

if [ -n "$dbnow" ]; then
    if [ -f "$DBREF" ]; then
        dbdiff=$(diff "$DBREF" <(printf '%s\n' "$dbnow") 2>/dev/null | grep -E '^[<>]' | head -20)
        [ -n "$dbdiff" ] && add "Database: a sensitive value changed" "$dbdiff
(< = valeur de reference, > = valeur actuelle. Si le changement est legitime :
 /root/surveille.sh --rebaseline)"
    else
        printf '%s\n' "$dbnow" > "$DBREF"
    fi
elif [ -f "$DBREF" ]; then
    # La sentinelle a deja fonctionne : si elle ne rend plus rien, c'est la
    # sentinelle qui est tombee, pas la base qui s'est videe.
    add "Sentinelle base de donnees muette" "La requete de controle ne rend plus rien.
Verifier que MySQL repond et que les acces de root sont valides, sinon les
comptes et les options ne sont plus surveilles du tout."
fi

# --- Sending -----------------------------------------------------------------
if [ -n "$ALERTS" ]; then
    # Dedup: do not report an identical anomaly again within 12 h, or a
    # persistent one floods the mailbox.
    sig=$(printf '%s' "$ALERTS" | sha256sum | cut -d' ' -f1)
    last="$STATE/last-alert-$sig"
    if [ -f "$last" ] && [ $(( $(date +%s) - $(stat -c %Y "$last") )) -lt 43200 ]; then
        echo "$(date -u '+%F %T') anomalie identique deja signalee, mail supprime" >> /var/log/surveillance.log
        exit 0
    fi
    find "$STATE" -name 'last-alert-*' -mtime +2 -delete 2>/dev/null
    touch "$last"
    # Always keep a local trace: the alert survives a failed send.
    {
        echo "===== $(date -u '+%F %T UTC') ====="
        echo "$ALERTS"
    } >> /var/log/alertes.log
    {
        echo "Anomalies detectees sur $HOST ($(date -u '+%Y-%m-%d %H:%M UTC'))."
        echo "Site : https://www.example.com/"
        echo "Serveur : 203.0.113.10"
        echo "$ALERTS"
        echo
        echo "--"
        echo "Surveillance automatique installee apres l'incident du 10 aout 2026."
        echo "Script : /root/surveille.sh — journal : /var/log/surveillance.log"
        echo "Ce mail ne part que si un fichier ou une valeur en base a change."
        echo "Les tentatives d'URL sont journalisees sans alerte : /var/log/tentatives.log"
    } | mail -s "[ALERTE] example.com" -a "From: $FROM" "$DEST"
    echo "$(date -u '+%F %T') ALERTE envoyee" >> /var/log/surveillance.log
else
    echo "$(date -u '+%F %T') OK" >> /var/log/surveillance.log
fi

One piece is not in this picture and cannot be: the backup cadence on the host side. One a week over 28 days is four restore points, none of them guaranteed clean. No script fixes that.

What actually made a difference

I ran this remediation with Claude as a pair, and asked it to score the site's security before and after: 70, then 92. Let me say it straight away, that number means nothing. It adds up measures that do not carry the same weight, and it comes from the very party that did the work. It is good for one thing, and that is why I keep it: checking we are moving the right way.

What deserves keeping is the breakdown. The antivirus saw nothing. The grep on dangerous functions saw nothing. Modification times actively lied. The only checks that produced knowledge were the ones comparing the install against an outside reference: the official core hashes, and the ctime the attacker cannot rewrite. Everything else cost me time.

And the one change that would have stopped the intrusion is neither an antivirus, nor a fingerprint, nor a detection rule. It is a line of permissions. Recognising a specific attack gets bypassed by changing the attack. Making a whole class of attacks impossible does not.

Detection is how you know. Architecture is what protects.

Wrapping Go errors: where, and mostly why

Odilon HUGONNOT — Wed, 29 Jul 2026 09:00:02 +0000

Mid-review on an order confirmation mailer, I confidently explained that the right call is to wrap an error once, at the package boundary, not at every internal step. The functions that build the message stay silent, only the public function that sends the email adds useful context. I even quoted Dave Cheney to back it up. Except while researching this article, I found out Dave Cheney changed his own mind since then, and the real rule isn't about where you wrap, it's about what you choose to expose.

The advice I just gave

The code in question looked like this: two internal functions that build an email's content, and two public functions that call them before sending:

func buildOrderMessage(order Order) (Message, error) {
    tpl, err := loadTemplate("order-confirm")
    if err != nil {
        return Message{}, err // no wrap, propagate as-is
    }
    return tpl.Render(order)
}

func mailOrder(order Order) error {
    msg, err := buildOrderMessage(order)
    if err != nil {
        return fmt.Errorf("failed to build message for order %d: %w", order.ID, err)
    }
    return sender.Send(msg)
}

The idea: wrapping at every layer produces a noisy message like failed to send: failed to build message: failed to load template: open failed. Nobody reads those intermediate layers in production, only the final message and the root cause matter. Wrapping once, at the right spot, keeps the message readable and adds the context that actually carries value, here the order ID. That's a correct argument. It's just not the whole picture.

Except Dave Cheney changed his mind

In 2016, Dave Cheney published Don't just check errors, handle them gracefully and popularized pkg/errors. His recommendation at the time: wrap at every layer with errors.Wrap, specifically to build a readable stack of context without a traditional stack trace:

func ReadFile(path string) ([]byte, error) {
    f, err := os.Open(path)
    if err != nil {
        return nil, errors.Wrap(err, "open failed")
    }
    // ...
}

func ReadConfig() ([]byte, error) {
    config, err := ReadFile(path)
    return config, errors.Wrap(err, "could not read config")
}

The final message becomes could not read config: open failed: open /path: no such file or directory. Back then, the argument held: without wrapping at every level, a failure inside authenticate() bubbles up to the top of the stack showing just "no such file or directory," with no clue where or why.

Then in 2021, while looking for a maintainer to take over pkg/errors, Cheney wrote that he no longer wraps his errors at all. His reason: structured logging (slog fields rather than a concatenated string) carries that debug context better than nested layers of Wrap. The creator of the pattern ended up finding it redundant with modern tooling.

What matters here isn't picking a side between "always wrap" and "never wrap." It's understanding why both positions are defensible depending on what you're actually trying to solve: a readable production error message, or an API contract between packages.

The real question: what does %w commit you to

Since Go 1.13, %w in fmt.Errorf does more than decorate a message. It makes the inner error inspectable via errors.Is and errors.As from any caller. That means %w isn't a style choice, it's an API commitment:

// %w: exposing the inner error, callers can match on it
func LookupUser(id string) (*User, error) {
    row, err := db.QueryRow(...)
    if err != nil {
        return nil, fmt.Errorf("lookup user %s: %w", id, err)
    }
    // ...
}

// The caller can now do this, and rely on it staying true over time
if errors.Is(err, sql.ErrNoRows) {
    return http.StatusNotFound
}

The classic trap: LookupUser runs on database/sql today. The day it switches to an ORM, or a cache gets added in front of the query, sql.ErrNoRows stops bubbling up. The caller's errors.Is check silently breaks, no compile error, just a 404 that quietly becomes a 500 in production. The %w had turned an implementation detail (we use database/sql) into a public promise.

Which gives a simple rule straight from the Go 1.13 docs: default to %v, because it preserves the abstraction and exposes nothing. Reach for %w only when you consciously decide the caller needs to inspect the inner error, and in that case define your own sentinel rather than leaking a dependency's:

var ErrUserNotFound = errors.New("user not found")

func LookupUser(id string) (*User, error) {
    row, err := db.QueryRow(...)
    if errors.Is(err, sql.ErrNoRows) {
        return nil, fmt.Errorf("lookup user %s: %w", id, ErrUserNotFound)
    }
    if err != nil {
        return nil, fmt.Errorf("lookup user %s: %v", id, err) // internal detail, not exposed
    }
    // ...
}

The caller now matches on ErrUserNotFound, a sentinel the package owns. If database/sql disappears tomorrow, the contract still holds.

The public/private rule that actually holds up

Which leaves the "where" question. The most useful rule I found while digging comes from Efe Karakus's blog: wrap only at public boundaries, let private functions propagate as-is. His example makes the point well: a file path repeated three times in the error message, because every private layer kept adding its own redundant context.

// ❌ Every level wraps, even private functions
// "failed to read config from /etc/x: failed to read file from /etc/x: failed to open /etc/x: ..."

// ✅ Only public functions wrap
func readFile(path string) ([]byte, error) {
    f, err := os.Open(path)
    if err != nil {
        return nil, err // private, propagate as-is
    }
    return io.ReadAll(f)
}

func Read(path string) ([]byte, error) { // public, wraps once
    data, err := readFile(path)
    if err != nil {
        return nil, fmt.Errorf("failed to read config from %s: %w", path, err)
    }
    return data, nil
}

That's exactly what the order confirmation mailer was already doing, and that's why the initial advice wasn't wrong. It was just incomplete: it answered "where," not "%w or %v."

What I'm changing in the mailer

Revised version, applying both criteria together: where to wrap (the public boundary), and what to expose (only when the caller needs to act on it):

func buildOrderMessage(order Order) (Message, error) {
    tpl, err := loadTemplate("order-confirm")
    if err != nil {
        return Message{}, err // private, propagate as-is
    }
    return tpl.Render(order)
}

func mailOrder(order Order) error {
    msg, err := buildOrderMessage(order)
    if err != nil {
        // %v: the caller has nothing to match on for a template error,
        // it just needs to know which order failed
        return fmt.Errorf("failed to build message for order %d: %v", order.ID, err)
    }
    if err := sender.Send(msg); err != nil {
        if errors.Is(err, smtp.ErrRateLimited) {
            // %w here: the caller MUST be able to tell "retry later"
            // apart from a permanent send failure
            return fmt.Errorf("send order %d: %w", order.ID, smtp.ErrRateLimited)
        }
        return fmt.Errorf("send order %d: %v", order.ID, err)
    }
    return nil
}

The difference comes down to one line: the SMTP rate limit is wrapped with %w because the caller has a genuinely different action to take (retry with backoff). Everything else goes through %v, useful context for a human reading logs, without promising anyone that *template.Error will still be exposed ten versions from now.

Conclusion

The real trap isn't picking the wrong side between "wrap everywhere" and "wrap once." It's treating %w as a cosmetic message flourish when it's actually a contract clause. Once you ask "am I committing my package to always return this specific error," the position in the code becomes secondary. It also taught me to verify a quote before dropping it in a code review, even when it comes from a name as solid as Dave Cheney's.

Meta Tags Generator: Title, Open Graph and Twitter Card at Once

Odilon HUGONNOT — Tue, 28 Jul 2026 09:00:02 +0000

The familiar scene: you publish a page, share it on LinkedIn, and the preview shows the wrong title, a blank description, and a random image pulled from somewhere on the page. Or worse, no title tag at all because you forgot the CMS doesn't generate one automatically.

Meta tags live in the <head> and are invisible on the page. That's exactly why they get forgotten. Yet they're what separates a link someone clicks from a link that looks like a phishing attempt.

Three layers, three sets of rules

Search engines read <title> (shown as the blue clickable line in results, 50-70 characters) and <meta name="description"> (the grey snippet below it, 150-160 chars). Google can override your description with something it generates itself, but if you leave it blank, it picks the first paragraph it finds — rarely your best writing.

Social platforms (Facebook, LinkedIn, WhatsApp, Slack) read Open Graph tags: og:title, og:description, og:image and og:url. Without them, the preview is assembled from whatever the crawler finds first. The ideal OG image is 1200 × 630 px — large enough to be readable in a feed, small enough to load quickly.

Twitter/X has its own format, Twitter Cards, with twitter: prefixed tags. In practice, Twitter falls back to OG tags, but the twitter:card tag lets you choose between a small thumbnail (summary) and a full-width image (summary_large_image).

The most common mistakes

A title that's too long: Google truncates past roughly 600 pixels — not characters, because a capital "W" takes more space than a lowercase "i". A title that's too short wastes keyword real estate.

A generic description copy-pasted across every page: "Welcome to our website. Browse our products and services." The description doesn't directly affect rankings, but it's your only chance to speak to someone who's hovering over your link and hasn't decided to click yet. A generic description loses that conversion.

A missing OG image: the LinkedIn preview shows a blank square. The link looks broken.

OG tags in French but not in English, or different content per language: social crawlers don't follow language redirects. They see whatever the server returns for the URL they request.

Using the generator

I built a free meta tags generator that outputs all three layers at once: SEO, Open Graph and Twitter Card.

It shows a live preview of how your page will appear in Google (title + description + URL), on Facebook/LinkedIn (image + title + domain), and on Twitter (summary or large image). You see exactly what your visitors will see before you publish anything.

Character counters turn red when you exceed the limits. A global SEO score (out of 100) checks the basics: title length, description present, canonical URL set, OG image provided, viewport defined.

The generated code is ready to paste into your <head>, in the recommended order (charset and viewport first, then OG tags).

Open the meta tags generator

A few rules worth keeping

Put your main keyword early in the title, ideally in the first three words. The description doesn't directly affect Google rankings, but it drives click-through rate, which feeds back into position over time.

The canonical tag (<link rel="canonical">) prevents duplicate content when a page is accessible from multiple URLs: with and without www, with and without a trailing slash, with UTM parameters.

To validate after publishing: the Facebook Sharing Debugger (developers.facebook.com/tools/debug), the Twitter Card Validator, and Google Rich Results Test. The generator lets you skip the publish-test-fix-republish loop by showing the result upfront.

GOMAXPROCS and Kubernetes: Go App Throttled, How to Fix It

Odilon HUGONNOT — Mon, 27 Jul 2026 09:00:03 +0000

The Go pod is running in production. CPU limit set to 2, metrics look reasonable. But under load, P99 latencies spike intermittently with no obvious cause. No errors, no goroutine leaks, just latency blowing up on traffic bursts.

The root cause is usually invisible: GOMAXPROCS equals the number of CPUs on the physical node, not the container limit. Your Go app thinks it has 32 CPUs when it only has 2. The Linux kernel handles the gap in its own way — CFS throttling.

What GOMAXPROCS reads (and what it ignores)

By default, the Go runtime computes GOMAXPROCS via runtime.NumCPU(), which reads the number of CPUs available at the OS level. On a 32-core Kubernetes node, that returns 32 — regardless of what resources.limits.cpu says in your pod spec.

Kubernetes CPU limits are enforced through Linux cgroups (v1 or v2). Cgroups are transparent to processes: a pod with limits.cpu: "2" doesn't see two virtual CPUs, it sees all the node's CPUs and gets suspended when it consumes too much. The Go runtime, historically, never read cgroups. It trusted the physical core count.

package main

import (
    "fmt"
    "runtime"
)

func main() {
    // Inside a pod with limits.cpu: "2" on a 32-core node
    fmt.Println(runtime.NumCPU())      // → 32
    fmt.Println(runtime.GOMAXPROCS(0)) // → 32
}

CFS throttling: how the kernel slows you down

The Linux CFS (Completely Fair Scheduler) enforces CPU limits via two cgroup parameters: cpu.cfs_quota_us (allowed CPU time) and cpu.cfs_period_us (measurement window, 100 ms by default). A pod limited to 2 CPUs gets at most 200 ms of CPU time per 100 ms window.

When Go spawns 32 OS threads for 32 parallel goroutines, those threads compete for physical CPUs. Once their combined usage exceeds the cgroup quota within the current window, the kernel suspends all threads in the cgroup until the next window starts. That's throttling: a complete application freeze lasting anywhere from a few milliseconds to several tens of milliseconds. A handful of these per second is enough to tank your P99.

The signal shows up in Kubernetes Prometheus metrics:

container_cpu_cfs_throttled_periods_total
container_cpu_cfs_periods_total

# Throttling ratio (should be near 0)
rate(container_cpu_cfs_throttled_periods_total[5m]) /
rate(container_cpu_cfs_periods_total[5m])

The fix: automaxprocs (and Go 1.25)

Uber published go.uber.org/automaxprocs exactly for this. The package reads the cgroup at startup (v1 or v2), computes the effective quota by dividing quota_us / period_us, and calls runtime.GOMAXPROCS(n) with the right value.

Integration is a single blank import:

package main

import (
    "fmt"
    "runtime"

    _ "go.uber.org/automaxprocs" // reads cgroup in init()
)

func main() {
    // Inside a pod with limits.cpu: "2"
    fmt.Println(runtime.GOMAXPROCS(0)) // → 2
}

go get go.uber.org/automaxprocs

If you'd rather avoid the dependency and know the value at deploy time, set it manually:

import "runtime"

func init() {
    runtime.GOMAXPROCS(2) // match your limits.cpu
}

The downside of the manual approach: change the pod spec and forget to update the code, and you're back to square one. With automaxprocs, the value tracks the cgroup automatically.

Go 1.25: fixed in the runtime itself

Since Go 1.25 (August 2025), the runtime reads cgroups by default on Linux. If the process runs inside a container with a CPU quota lower than the node's core count, GOMAXPROCS is adjusted automatically — no external dependency needed. Go 1.25 also watches for quota changes at runtime, which matters if Kubernetes adjusts resource limits on a live pod.

Two GODEBUG options let you opt out if needed:

# Ignore cgroup quota (pre-1.25 behavior)
GODEBUG=containermaxprocs=0

# Keep the initial value, skip dynamic updates
GODEBUG=updatemaxprocs=0

On Go 1.25 and above, automaxprocs is no longer needed. On Go 1.24 and below, Uber's package remains the standard solution.

Verify it's working

To check the effective value inside a running pod:

# Open a shell in the pod
kubectl exec -it <pod-name> -- /bin/sh

# Read the cgroup v2 quota
cat /sys/fs/cgroup/cpu.max
# → 200000 100000  (= 2 CPUs: 200ms quota / 100ms period)

# Read cgroup v1 values
cat /sys/fs/cgroup/cpu/cpu.cfs_quota_us
cat /sys/fs/cgroup/cpu/cpu.cfs_period_us

If you expose Go runtime metrics via Prometheus, the gauge go_sched_gomaxprocs_threads (available with the runtime/metrics collector since Go 1.17) shows the current value. It should match your CPU quota, not the node's core count.

Conclusion

CFS throttling in Go Kubernetes workloads is a silent problem: no error logs, no obvious cause, just P99 latency spikes under load. The root cause is almost always the same — GOMAXPROCS sized to the node, not the container.

The fix is proportionate to the problem: one import or one runtime flag. On Go 1.25, it's handled by upgrading. On older versions, automaxprocs has been battle-tested at Uber scale for years. Either way, it's a five-minute change with a measurable effect the next time your traffic spikes.

Go 1.25 testing/synctest: No More Flaky Concurrent Tests

Odilon HUGONNOT — Sun, 26 Jul 2026 09:00:01 +0000

Three green tests locally, two red on CI. The outcome depends on how busy the build server is that day. This isn't a logic bug — it's a time.Sleep that assumes 100 ms is always enough for a goroutine to finish.

That test doesn't verify anything. It gambles.

testing/synctest, stable since Go 1.25 (August 2025), solves this at the root. No more guesswork, no more arbitrary waits: the clock advances on demand, inside an isolated bubble.

The test that lies

Debounce is a classic example of time-dependent concurrent code: trigger an action only if no call has happened in the last N milliseconds. Here's a simple implementation.

func Debounce(fn func(), delay time.Duration) func() {
    var timer *time.Timer
    var mu sync.Mutex
    return func() {
        mu.Lock()
        defer mu.Unlock()
        if timer != nil {
            timer.Stop()
        }
        timer = time.AfterFunc(delay, fn)
    }
}

And the naive test that goes with it:

// ✗ Fragile
func TestDebounce(t *testing.T) {
    called := 0
    debounced := Debounce(func() { called++ }, 50*time.Millisecond)

    debounced()
    debounced()
    debounced()

    time.Sleep(200 * time.Millisecond) // fingers crossed

    if called != 1 {
        t.Fatalf("want 1 call, got %d", called)
    }
}

This passes locally because your machine is fast and 200 ms feels like plenty. On a loaded CI runner, the time.AfterFunc goroutine may not have had a chance to run by the time called is read. Flaky. No error message explains why.

go test -race won't catch this either. There's no concurrent memory access at the same instant — the problem is purely temporal. Two different tools, two different failure modes.

What testing/synctest does

testing/synctest runs your test inside a "bubble". Inside the bubble, the time package uses a fake clock controlled by the test. That clock doesn't tick on its own: it only advances when all goroutines in the bubble are durably blocked — waiting on a channel, a mutex, a timer — but not on a syscall or network I/O.

The Go 1.25 API has exactly two functions:

synctest.Test(t, f) — runs f in a new bubble and waits for every goroutine in that bubble to exit before returning to the test framework.
synctest.Wait() — blocks until all other goroutines in the bubble are durably blocked. The explicit synchronization point: "background work is done for now."

Go 1.24 had an experimental preview behind GOEXPERIMENT=synctest with a slightly different API (synctest.Run() instead of synctest.Test()). Go 1.25 graduated the package to the standard library and finalized the API. No flags, no build tags — direct import.

Before and after

The same debounce test, rewritten with testing/synctest:

// ✓ Deterministic
import "testing/synctest"

func TestDebounce(t *testing.T) {
    synctest.Test(t, func(t *testing.T) {
        called := 0
        debounced := Debounce(func() { called++ }, 50*time.Millisecond)

        debounced()
        debounced()
        debounced()

        // Advances the virtual clock by 100 ms instantly.
        // The AfterFunc goroutine (fires at 50 ms) wakes up along the way.
        time.Sleep(100 * time.Millisecond)
        synctest.Wait()

        if called != 1 {
            t.Fatalf("want 1 call, got %d", called)
        }
    })
}

What happens inside the bubble:

The three debounced() calls each schedule a time.AfterFunc at 50 ms, canceling the previous one.
time.Sleep(100ms) blocks the test goroutine. All goroutines in the bubble are now durably blocked, so the runtime advances the virtual clock to 50 ms. The AfterFunc goroutine wakes up and runs called++.
The clock continues to 100 ms. The test goroutine wakes up. synctest.Wait() confirms the AfterFunc goroutine has finished.
The assertion on called is reliable. Always. No gambling.

Actual wall-clock time for the test: microseconds, not 200 ms.

What synctest doesn't cover

testing/synctest isn't a silver bullet. Three categories slip through.

Network I/O. A goroutine blocked on a network read isn't "durably blocked" in synctest's terms — an external process can unblock it at any time. If your code makes real HTTP calls, synctest.Wait() can't reason about its state. Fix: replace the network layer with an interface backed by a deterministic test implementation (net.Pipe(), httptest.Server, or a hand-rolled mock).

Syscalls and cgo. Same reason: the Go runtime doesn't control what the OS or C code does in the background. These calls are not considered durably blocking.

Package-level sync.WaitGroup (var wg sync.WaitGroup). These can't be associated with a bubble. Declare them inside the test function instead.

Ideal use cases: timers, tickers, debounce, rate limiters, TTL caches, retry with backoff — anything that chains time.After, time.Sleep, or time.AfterFunc with goroutines communicating over channels.

Conclusion

go test -race remains the right tool for real data races. testing/synctest targets a different class of failure: tests whose reliability depends on wall-clock timing. The two tools are orthogonal — use both.

If you have time.Sleep in your Go tests to "give the goroutine time to finish," that's the signal. testing/synctest will do the job faster and correctly. Go 1.25 has been out since August 2025 — no flags, no configuration, just import it.

📚 Related reading. If goroutines that never exit are your problem, the natural follow-up is Goroutine leaks in Go: detect, understand, fix.

39,000 Torrents: The Bug My Green Benchmark Never Caught

Odilon HUGONNOT — Sat, 25 Jul 2026 09:00:02 +0000

[Torrent911] Inception.2010.TRUEFRENCH.1080p.BluRay.x264-YIFY

That's a real filename. The goal: extract "Inception" and "2010" from it, query the TMDB API (The Movie Database, the open reference for film and TV metadata), and display the right card. Poster, rating, synopsis. Automatically, for a library of a few hundred files.

Context: a self-hosted PHP media server. Think Plex, but lighter, no cloud, no account. The interface looks vaguely like Netflix. The hard part is automatic recognition. Torrent release names follow no formal standard: [Torrent911] can come before the title, TRUEFRENCH can interrupt it, a release group trails at the end. Films have a year. TV shows almost never do. Popular anime have episode numbers with four digits: One.Piece.S01E1164.VOSTFR.

I had accumulated 39,188 real torrent names over the years: 22,058 films, 17,130 series. Enough to build an honest test bench. I used an AI assistant to replay variants quickly and analyze where the metrics leaked.

Two things happened. First, a clean data-driven optimization: measure, iterate, stop at the right time. Then, a classic trap: the benchmark was green while a silent bug turned every single TV show into a film match.

Two different lessons about what data measures — and what it doesn't.

A minefield inside a string

The pipeline has two distinct stages.

Stage one: extract a clean title and a year from the filename. This is the foundation. If it returns "Inception 1080p BluRay x264" instead of "Inception", TMDB finds nothing useful. Release naming puts the title first and technical metadata after, but the variations are endless: site prefixes ([Torrent911]), language tags (VOSTFR, TRUEFRENCH), codec (x264, H.265), resolution (1080p, 4K), release groups.

Stage two: query TMDB with the extracted title and pick the right result. A search for "Breaking Bad" returns dozens of results, including films and documentaries with the same name. You have to score and decide.

Three metrics to measure extraction quality: cleanliness (% of titles with no residual technical tag — a stray 1080p, WEB-DL, or S01E02 that should have been stripped), coverage (% of non-empty titles), and year recall (how often the year is correctly extracted). Cleanliness is the main metric. A "dirty" title produces false positives or zero results on TMDB. Coverage was already near 100% from V0.

V0 → V4: measure before you optimize

The test bench is entirely offline. It replays the extraction algorithm on all 39,188 corpus filenames, computes the metrics, and returns a report in a few seconds. Deterministic, 100% replayable, no network calls. This is where the AI assistant genuinely helps: building the bench, writing the variants, analyzing the 65 remaining leaks to find patterns. Not "the AI optimized the algorithm" — more like "the AI let me iterate five times in one hour".

Version

Approach

Cleanliness

Leaks

V0 — naive

Separator normalization only (. and _ → space)

10.84%

34,939

V1 — basic

Strip 8 known tags (1080p, BluRay…)

50.23%

19,492

V2 — cut at 1st tag

Truncate at the first technical marker found

99.83%

V3 — episode fix

Extended episode pattern to 3-4 digits (\d{0,4})

99.93%

V4 — experimental

Article normalization, bracket-rescue…

99.93%

V0 applies basic separator normalization. 10.84% cleanliness, 34,939 leaks. Catastrophic baseline, but that's the honest starting point.

V1 strips 8 known technical tags. 50%. The corpus has far more than eight conventions. This approach doesn't scale: every site and every release group has its own patterns.

V2 introduces the central idea: don't try to enumerate all known tags, just truncate at the first technical marker found. Release naming places the title at the start and technical metadata after. Cutting at the first tag brings cleanliness from 10.84% to 99.83%. 65 leaks remain across 39,000 names. This single shift in approach delivers 99% of the total gain.

// Step 1 — bracket prefixes are stripped first, no enumeration needed:
// [Torrent911], [HorribleSubs], [FW]... one pattern covers them all.
$clean = preg_replace('/\[.*?\]/', '', $clean);
$clean = preg_replace('/[._()[\]{}:]+/', ' ', $clean);

// Step 2 — everything after (and including) the first technical marker is discarded.
// This is what takes cleanliness from 10.8% to 99.8%.
$title = preg_replace(
    '/\b(multi|vff|truefrench|french|vostfr|bluray|web-?dl|hdtv|x264|x265|hevc'
  . '|10bit|remux|2160p|1080p|720p|480p|uhd|hdr|dts|aac|ac3|proper|repack'
  . '|extended|directors?-?cut|complete|s\d{1,2}e?\d{0,4}|e\d{2,4})\b.*/i',
    '', $clean
);
// s\d{1,2}e?\d{0,4}: the {0,4} (instead of {0,2}) catches 3-4 digit episodes
// (One Piece S01E1164, Plus Belle La Vie S12E248) — found by analyzing 39,000 torrents.

V3 fixes a case revealed by analyzing the 65 remaining leaks. Three and four digit episode numbers weren't captured: S01E1164 for One Piece, S12E248 for Plus Belle La Vie. One change in the regex (\d{0,2} → \d{0,4}) cuts the leak count in half, from 65 to 28. You can't guess this by looking at ten examples manually. The corpus forces it to appear in the statistics.

V4 tests more aggressive normalization. Result: 29 leaks — one more than V3. The data says stop. Adding complexity to regress is a no.

Extracted title cleanliness V0 to V4 — from 10% to 99.9% Extracted title cleanliness (% with no residual technical tag) 10.84% V0 naive 34,939 leaks 50.23% V1 basic cut 19,492 leaks 99.83% V2 cut at 1st tag 65 leaks 99.93% V3 + episode fix 28 leaks 99.93% V4 experimental 29 leaks

V2 (truncate at the first technical tag) delivers 99% of the gain. V3 halves the remaining leaks with a single regex character change.

Three lessons from this part.

The big lever was binary. Going from "keep everything" to "cut at the first technical tag": 10.8% → 99.8% cleanliness in one idea. Refinement (V3, V4) comes after, not before.

Large-scale data finds what the eye can't see. Without 39,000 examples, four-digit episode numbers stay invisible. The corpus forces them into the statistics.

Know when to stop. V4 is more complex and slightly worse than V3. YAGNI proven by data, not assumed.

TMDB matching and the premature optimization

Extracting a clean title is half the work. The other half: query TMDB and pick the right result from the candidates.

The scoring combines several signals. Title similarity counts for 65% (normalized comparison across all title variants). Year gap: 15%. Type coherence (film or TV): 10 to 18% depending on confidence. TMDB popularity (vote count): 12%, so obscure namesakes lose to well-known titles. Above 55: strong match. Between 35 and 55: quarantine for manual review. Below: rejected.

When the exact query doesn't produce a good result, the algorithm shortens it word by word until the confidence threshold is reached. A title with unusual punctuation may not match directly: we drop words from the end one at a time and keep the longest query that clears the threshold.

// Releases often append a subtitle or language tag after the actual title.
// Drop words FROM THE END, one at a time; keep the LONGEST query that clears
// the confidence threshold (55). The title is always at the start.
$words    = explode(' ', trim($title));
$minWords = max(1, count($words) - 4);   // at most 5 tries
$best = null; $bestScore = 0;
for ($n = count($words); $n >= $minWords; $n--) {
    $query = implode(' ', array_slice($words, 0, $n));
    foreach (tmdb_search($query, $year, $preferTv) as $cand) {
        $score = score_candidate($query, $year, $cand, $preferTv);
        if ($score > $bestScore) { $bestScore = $score; $best = $cand; }
    }
    if ($bestScore >= 55) break; // reliable match → stop truncating
}

This handles cases like "Avengers: Age of Ultron" (the colon disappears during cleanup) or foreign-language titles that don't match the English filename. Each iteration costs an API call, but avoids zero-match on titles with punctuation or special characters.

This is where an "optimization" had quietly slipped into the code. The variable $preferTv was supposed to signal to TMDB to prioritize TV results. To save API calls, I'd set the algorithm to query the "probable" type first (movie by default, unless the filename explicitly looked like a series), and only query tv if there were no candidates at all. The network test bench showed flattering numbers: 90% match rate for films, 96.7% for series.

I navigated it in real conditions. Every single TV show matched a film.

Breaking Bad → Breaking Bad Wolf (an indie film with the same name)
Game of Thrones → Game of Thrones: The Last Watch (an HBO documentary)
The Mandalorian → The Mandalorian and Grogu (an upcoming film)
One Piece → One Piece: Gold (a feature film)

0 correct out of 11 series.

Why the benchmark was showing green

The offline bench only measured extraction. Fair enough, that's its scope. It knew nothing about matching.

The network bench, though, provided the content type (movie or tv) directly from the known category of each corpus entry. It passed $preferTv = true for series directly. It never exercised the automatic type detection, because it didn't need to: the right answer was already baked into the request parameters. The auto-detection that ran in production had never been tested. The aggregate metric was green on a path that real navigation never took.

A benchmark tests what you tell it to test. Its blind spots are silent.

Before and after the fix: film/TV type detection for TMDB matching BEFORE « Breaking Bad » type = movie (auto-guessed) TMDB: movies only Breaking Bad Wolf ✗ false positive AFTER « Breaking Bad » movie + tv in parallel scoring picks by popularity Breaking Bad (TV series) ✓ correct match

The "save an API call" optimization removed tv from the circuit. Scoring can't choose between two types if it only sees one.

The fix: four levers

1. Always query both types.

Movie and tv are now queried every time. Scoring decides. Breaking Bad (TV series on TMDB, high popularity) beats any obscure namesake film on popularity score alone. This was the central regression: saving one API call had broken the selection logic. The fix is literally removing the condition that blocked the second type.

2. Folder structure as a signal.

Parsing the filename to guess film or series is fragile. A show organized in seasons — "Show/Season 1/", "Show/01/", "Show/02/" — may have no S01E01 in its folder name at all. The folder structure doesn't lie.

// A folder is a SERIES if it has multiple episodes (≥ 2 videos at top level)
// OR numbered season subdirectories (Season 1, Saison 3, 01, 02…).
// A film has a single video file. Folder SHAPE beats parsing the filename.
$videoCount = 0; $seasonish = 0;
foreach (scandir($folder) as $sub) {
    if ($sub[0] === '.') continue;
    if (is_dir("$folder/$sub")) {
        if (preg_match('/^(s\d{1,2}\b|saison|season|saga|arc|part|\d{1,3}$)/i', $sub))
            $seasonish++;
    } elseif (in_array(strtolower(pathinfo($sub, PATHINFO_EXTENSION)), $VIDEO_EXTS, true)) {
        $videoCount++;
    }
}
$isSeries = ($videoCount >= 2 || $seasonish >= 1);

This signal is used to weight the type coherence bonus in the score. It doesn't replace querying both types; it refines the decision.

3. Multilingual title merging.

TMDB returns multiple title variants per result: original, local, alternative. When you search for "Attack on Titan", TMDB may return "L'Attaque des Titans" (French) and "Shingeki no Kyojin" (Japanese). Deduplication by ID was keeping only the first title encountered: low similarity with the English query, potentially missed match.

Fix: aggregate all title variants by ID and score against the best one. Unexpected bonus: Spirited Away went from a weak match to a strong match, because "Sen to Chihiro no Kamikakushi" (the Japanese original title) was already in TMDB data and contributed to the similarity score.

4. Reinforced type coherence bonus.

When the folder structure clearly confirms a series (numbered subdirectories, dozens of video files), the tv type bonus in the score goes from 10% to 18%. The structural signal amplifies the scoring decision without overriding it.

Result: 11 series and anime matched correctly at high confidence, 18 films stay films. 0 → 11 on series.

Full source code

ShareBox is open source. The algorithm described in this article lives in functions.php: extract_title_year() (extraction), tmdb_match() (word-removal loop), tmdb_score_candidate() (scoring).

Conclusion

The large-scale data brilliantly optimized extraction. From 10.8% to 99.9% cleanliness in three measured iterations, across 39,000 names. The corpus surfaced a blind spot (S01E1164) no human would test manually, and it said stop when a more complex variant regressed by one unit.

The matching failed differently. An API cost optimization caused a regression invisible to the aggregate metrics, because the benchmark was feeding itself the right answer. Thirty seconds of real navigation found what a 96.7% success rate had hidden.

The real lesson: data optimizes brilliantly what it measures. A benchmark measures one path. It doesn't know others exist. The aggregate metric was an honest indicator; it wasn't a complete one. Testing in real conditions stays irreplaceable, even when everything is green.