DEV Community

Four bugs my publisher's validator could not see

Four bugs my publisher's validator could not see

I publish everything I build to a small private ops portal: scheduled job reports, generated images, deploys, drafts. One Python CLI called publicar writes a JSON file per publication into data/publications/, rebuilds the feed manifest in data/index.json, and has a validar subcommand that is supposed to stop a broken record from going live.

Between October 1 and October 6, seven publications went out with a file path where the article body should have been. The validator ran on every one of them and printed OK. The feed showed cards that opened a page whose entire content was /tmp/corpo.md.

This is what those failures had in common, and the check that now catches them.

The flag never looked at its input

--markdown takes either a literal string or - for stdin. A file path is also a literal string. That is the whole implementation:

markdown = args.markdown
if markdown == "-":
    markdown = sys.stdin.read()
Enter fullscreen mode Exit fullscreen mode

I had been typing --markdown /tmp/corpo.md, expecting the CLI to read the file because that is what most tools do. It stored the path. Eighteen characters, exit code 0, no warning. The fix belongs in the validator, not in the flag, because changing the flag's meaning would silently break the calls that already pipe through stdin.

Agreement between two files is not correctness

The validator compared the manifest against the per-publication JSON. Both were produced from the same corpo field, so both contained the path, so they agreed, so the check passed. A check that compares a value to a copy of itself is a check on the copying code. It says nothing about whether the value is right.

What made this detectable was an independent statement about the shape of a body. A body has line breaks. A body is usually longer than a path. A body rarely ends in .md. The current check encodes exactly those three properties and nothing else:

corpo = (p.get("corpo") or "").strip()
if corpo and "\n" not in corpo and len(corpo) < 300 \
        and (corpo.startswith(("/", "~/")) or corpo.startswith(".")) \
        and re.search(r"\.(md|markdown|txt|html)$", corpo):
    problemas.append(
        f"{f.name}: corpo e um caminho de arquivo, nao o texto "
        f"({corpo}) — reidrate com: publicar publicar --atualizar "
        f"{pid} --markdown - < {corpo}")
Enter fullscreen mode Exit fullscreen mode

Two things matter more than the regex. First, the error message carries the repair command, so a failing run tells me what to type. Second, the check is deliberately narrow. A conservative rule that fires on real defects beats a clever rule that fires on files with no newline for legitimate reasons.

I verified it in both directions: fed it a record with a path body and watched it fail, then hydrated the same record with the real text and watched it pass. A validator I have only seen pass is a validator I have not tested.

--tags a,b,c became one tag with commas in it

--tags is action="append", so the argument arrives as a list. The old code assumed a string and split on commas only when the type matched. It also appended to whatever tags the record already had, which is the behaviour I wanted for a second call. The result was a chip in the feed that read projeto,cliente,enrique as a single label.

"tags": sorted(set(
    [t for bruto in (args.tags or []) for t in bruto.split(",")]
    + anterior.get("tags", []))),
Enter fullscreen mode Exit fullscreen mode

The quiet part is the + anterior.get("tags", []). Update operations in this CLI add to lists, they do not replace them. Tags and media accumulate. Links are rewritten from scratch, because I decided a link list is a statement of the record's current state rather than a log. Two different merge policies in the same function, chosen per field. Every one of them needs to be written down, because neither is discoverable from the flag name.

A type string that nothing validated, and three readers that broke silently

Media entries carry tipo. I typed ::imagem once, in Portuguese, out of habit. The CLI accepted it, wrote it into the JSON, and three separate consumers broke, each in a different and silent way:

  • resumo_card builds the feed card and picked the cover by comparing tipo == "image". It found nothing, returned capa: None, and the feed served a 404 for the card image.
  • The publication page's JavaScript fell through to its default branch and rendered the file name as text instead of an image.
  • The media viewer filtered by type, got an empty group, and did nothing when the thumbnail was clicked.

publicar validar passed all three times. It had checked that the media file existed and that the manifest matched the JSON. Both were true. The manifest had recorded "no cover" and the JSON had recorded "no cover", and consistency was the only property being measured.

The fix normalises in one place and accepts both spellings at the boundary:

# O tipo da midia pode vir em ingles ("image", valor que o `--midia`
# grava por padrao) ou em portugues ("imagem", que tambem aceita).
# Aceitar so "image" deixava a capa em None sem nenhum aviso: o
# manifesto ficava coerente com o JSON e o `publicar validar` nao
# reclamava, enquanto o card do feed servia um 404 em silencio.
for m in mids:
    if m.get("tipo") in ("image", "imagem"):
        capa = m["path"]
        break
Enter fullscreen mode Exit fullscreen mode

Accepting both is the smaller half of the lesson. The larger half is that an enumeration written by hand should be normalised at the moment of entry, not matched at the moment of use, because every consumer that pattern-matches on it is a place the match can be wrong.

Update appended, so correcting a piece doubled it

A carousel went from five slides to six. I ran the update flags again with the new set of images. The publication ended with eleven media: six new and five from the old run. The next correction made twelve.

The fix is not "filter the old files out", because the numeric prefix in each stored file name is rewritten on every run, so a filter based on position works once and fails the second time. The reliable key was already in the file name, appended by the CLI when it copies media into the publication folder: a content hash. 07-slide-1-2f78bcbb.jpg and 01-slide-1-2f78bcbb.jpg are the same image. Two paths sharing a hash are one media, regardless of their prefixes.

What the checks look like now

Every table above reduces to a question I ask before trusting a green run:

  1. Does the check measure the property I care about, or does it compare two artifacts produced by the same code path? If the manifest and the record are both generated from one field, agreement between them proves nothing about the field.
  2. Have I seen this check fail on a genuinely broken input? A check whose only observed output is OK has an unknown failure mode.
  3. Is the enum normalised where it enters, or matched where it is used? The second option multiplies the number of places that can disagree.
  4. Which fields does an update add to, and which does it replace? Write the answer next to the flag.

The one that cost the most time was the first. I had built a validator, I ran it before every deploy, and it was measuring consistency the entire time. Consistency is the cheapest property to verify and the least useful one to be sure about.

If you have built a publishing pipeline of your own and hit a similar case, I would like to hear which invariant you ended up checking. Meantime, the whole system is mine, small, and under a scheduler, so the failures above are all reproducible rather than hypothetical.


Se você mantém um pipeline de publicação parecido e prefere português, deixa um comentário que eu respondo em português. Esse texto saiu de um CLI que publica relatórios, imagens e deploys num portal próprio, e a parte de validação foi a que me ensinou mais.

Top comments (0)