DEV Community

takahiro hashito
takahiro hashito

Posted on

A guard that unlocks by calendar is a timer, not a guard

#webdev #programming

Background

I run a site that translates what overseas anime fans post about each episode. Most of the update pipeline is automated.

At one point I audited my own data and found that 68 of the 71 published episodes contained generated text rather than quotes from real posts. Commenter names were invented placeholders, and there was no original text and no source URL anywhere. That was 6,084 items.

Meanwhile the site's own about page said: comments are translated individually rather than summarized, sources are cited, and copyright for each comment belongs to its original poster.

What the site promised and what the site did had drifted completely apart. I had two options — rewrite the promise to match reality, or rewrite reality to match the promise. I picked the second. This post is about what actually had to change, because writing "do not fabricate" as a rule turned out to be the least effective part of it.

How it works

The fix ended up in three layers, and the order matters:

[1] remove the incentive  ... drop the per-title quota
[2] block the output      ... unqualified data never ships (fail-closed)
[3] detect the intrusion  ... flag it the moment it enters the data
Enter fullscreen mode Exit fullscreen mode

Skipping [1] and adding only [2] and [3] just redirects the effort into shapes the checks do not catch.

[1] Remove the incentive

The cause was a quota I had set myself: at least 150 reactions per title.

On a day when only three real posts could be found, a floor of 150 makes filling the gap the obvious move. "Write something based on the general sentiment" is the most natural compromise available in that situation. There was no bad intent anywhere. The quota was demanding the lie.

So the quota is gone. The current rules:

  • No minimum. However many real comments were found is however many get published. Three is fine.
  • Writing entries to hit a number is forbidden. No invented handles, no invented scores.
  • If nothing could be found, update with a different subject entirely — streaming availability, cultural context, how to read the metrics.

The third rule carries an explicit warning that it is not permission to write reactions. Any escape hatch you leave open without saying it is closed will eventually be used as one.

[2] Block the output

Nothing is deleted from the data. Filtering happens at publish time, so the history survives.

function isPublishableReaction(r) {
  if (!r || !String(r.text || "").trim()) return false; // untranslated -> not published
  if (REQUIRE_SOURCED && !(r.original && r.sourceUrl)) return false;
  return true;
}
function isAired(ep) {
  return !ep.aired || String(ep.aired) <= BUILD_DATE;
}
Enter fullscreen mode Exit fullscreen mode

BUILD_DATE is the build date in JST, and aired is a "2026-08-12" style string, so a lexicographic comparison gives the correct ordering with no Date objects and no timezone accidents. An episode that has not aired is not published, because "what a viewer thought" cannot exist before the episode does.

Episodes with no publishable reactions get dropped, and a title with no publishable episodes gets no page at all — it disappears from the sitemap too. There is deliberately no path that publishes an empty page as an empty page.

Implementation

The date guard was doing less than it looked

isAired had a hole that nearly cost me.

An audit found 342 unsourced reactions sitting on episodes that had not aired yet — 188 on one title's episode 1 (scheduled 7/25) and 154 on another's episode 1 (scheduled 8/12).

At that moment isAired was working exactly as written: none of it was public. The problem is that the guard releases itself when the air date arrives. Left alone, those 188 items would have gone live on 7/25 on their own.

A guard whose only condition is something that advances by itself does not block publication. It postpones it. So I added a flag that only a human can clear:

/* The data contained 342 reactions written for episodes that had not aired,
 * set to publish themselves once the air date arrived.
 * An episode with `blockPublish: true` is not emitted even after its air date.
 * Clear the flag once it is filled with real reactions
 * (until then it stays unpublished = fail-closed). */
function isPublishableEpisode(ep) {
  if (ep.blockPublish === true) return false;
  return isAired(ep);
}
Enter fullscreen mode Exit fullscreen mode

blockPublish does not expire. Here is what a real record looks like:

{
  "num": 1,
  "title": "第1話",
  "aired": "2026-08-12",
  "threadNote": "この話はまだ放送されていません。放送後に、実在する海外の投稿を出典つきで掲載します。",
  "sourced": false,
  "blockPublish": true
}
Enter fullscreen mode Exit fullscreen mode

Creating the shell of an episode ahead of time — number, air date, title — is fine. The line is that reactions stays empty. Forgetting to clear the flag fails toward "not published", which is the direction a mistake should fall.

[3] Detect it in the data, not just at the exit

Blocking output does not stop bad data from being written in the first place, so a separate scan runs over the data itself.

/**
 * Detects "what a viewer thought" attached to episodes that have not aired.
 * Nobody can have an opinion about an episode before it exists, so
 * a single hit is fabrication. The build blocks these via blockPublish,
 * but this catches them entering the data at all.
 */
function findUnairedWithReactions(animeList, { today } = {}) {
  const t = today || todayJST();
  const out = [];
  for (const a of animeList) {
    for (const ep of a.episodes || []) {
      if (!ep.aired || String(ep.aired) <= t) continue;
      const rs = (ep.reactions || []).filter((r) => r && !r.sourceUrl);
      if (!rs.length) continue;
      out.push({ slug: a.slug, title: a.title, num: ep.num, aired: ep.aired,
        unsourced: rs.length, blockPublish: ep.blockPublish === true });
    }
  }
  return out;
}
Enter fullscreen mode Exit fullscreen mode

Zero hits is the normal result. When there is a hit, the alert says what to do about it, not just that something is wrong — clear the reactions and set blockPublish: true. A check that only reports an anomaly gets postponed on a busy day; one that states the fix tends to get fixed on the spot.

Gotchas

Let the machine estimate, but never conclude

A helper script estimates how many episodes have aired, counting weeks from the première:

const weeks = Math.floor(daysBetween(today, firstAired) / 7);
estimatedAired = Math.max(0, weeks + 1); // the première day counts as episode 1
Enter fullscreen mode Exit fullscreen mode

This is deliberately left as an approximation. Broadcasts get pre-empted, and when they do this arithmetic is simply wrong. So every run emits the caveat alongside the number:

estimatedAired は毎週放送仮定の概算。実放送・休止は要外部確認。
(estimatedAired assumes weekly broadcast. Actual airing and pre-emptions need external confirmation.)
Enter fullscreen mode Exit fullscreen mode

The machine proposes candidates; confirmation comes from outside. If the première date cannot be read at all, it returns null rather than guessing. Publishing the limits of an estimate proved safer than trying to improve its accuracy.

"Whatever loaded first" as a source of truth

One more near miss. The estimator used to decide the current season from the season field of the first file it read — which, given alphabetical ordering, meant the first title's metadata decided it for everything.

A title that had actually finished airing in summer 2024 was registered as summer 2026. The estimator duly proposed episodes 2 through 4 as missing, and 2024 reactions came very close to being published as this season's.

The season now comes from a single currentSeason setting, falling back to the most common value only when that file cannot be read. It is worth grepping your own code for anything that quietly treats whatever it happened to see first as the authority.

The result

The site that runs on all of this: https://anime.autoarticles.net

The about page states plainly that some earlier episodes carried composed text rather than verbatim quotes, and those are being replaced with real, sourced posts. Silently deleting the history would have looked cleaner, but a reader would have had no way to know what changed.

Wrap-up

Three things actually moved the needle, and none of them was the rule itself:

  1. Delete the metric that rewards filling gaps. A number that conflicts with your stated principles will win against the principle every time.
  2. Make "not published" the default state. If a forgotten flag fails toward invisible, a mistake is not an incident.
  3. Never let a guard's only condition be something that advances on its own.

That last one generalizes well beyond content pipelines. A condition based on a date, a version number, or a countdown is not a guard — it is a timer, and it will eventually fire whether or not anyone did the work. Always layer in one condition that requires a human to move.

This article is about my own side project. It was written with AI assistance.

Top comments (0)