<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Jennifer Evangelista</title>
    <description>The latest articles on DEV Community by Jennifer Evangelista (@jenniferevangelistade).</description>
    <link>https://dev.to/jenniferevangelistade</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4201499%2Fda7c57a6-9225-47b5-8bec-bd61e8b256c6.png</url>
      <title>DEV Community: Jennifer Evangelista</title>
      <link>https://dev.to/jenniferevangelistade</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/jenniferevangelistade"/>
    <language>en</language>
    <item>
      <title>Ghost Hunt · Who Else Was There? Every face has a heartbeat. Except one.</title>
      <dc:creator>Jennifer Evangelista</dc:creator>
      <pubDate>Mon, 09 Nov 2026 21:20:20 +0000</pubDate>
      <link>https://dev.to/jenniferevangelistade/ghost-hunt-who-else-was-there-every-face-has-a-heartbeat-except-one-1jcg</link>
      <guid>https://dev.to/jenniferevangelistade/ghost-hunt-who-else-was-there-every-face-has-a-heartbeat-except-one-1jcg</guid>
      <description>&lt;p&gt;&lt;em&gt;This is a submission for the &lt;a href="https://dev.to/devteam/join-the-presage-smartspectra-sdk-challenge-3000-in-prizes-across-10-winners-1370"&gt;Presage SmartSpectra SDK Challenge&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;Marina had her bag on her shoulder, her keys in her hand and a car waiting outside. At the door it felt like a hand on her shoulder, so heavy she couldn't breathe. &lt;em&gt;Stay home.&lt;/em&gt; When she was nine, her mother went out by that same door and never came back. So Marina cancelled the car and stayed. Twenty minutes later, that same car crashed on the avenue.&lt;/p&gt;

&lt;p&gt;She lives alone, and she saw nothing. But the hallway camera was on. And at second 8 of that tape, on the dark stairs behind her, there is a second face.&lt;/p&gt;

&lt;p&gt;Its pulse reads &lt;strong&gt;0 beats per minute&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;(Marina is a character, and her tape is drawn. In the game, "0 BPM" means the detector got no reliable pulse from that face. It is a story clue, never a health result: a living person in bad light can get it too.)&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;▶ Play it now, no install:&lt;/strong&gt; &lt;a href="https://jenniferevangelista-de.github.io/who-else-was-there/" rel="noopener noreferrer"&gt;jenniferevangelista-de.github.io/who-else-was-there&lt;/a&gt;. It opens in the browser, on a computer or a phone.&lt;/p&gt;

&lt;p&gt;This game only exists because of Presage SmartSpectra: without a pulse read from a plain camera, there is no way to tell which face is missing one.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Signals&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;7 live Presage SmartSpectra signals: pulse, chest movement, breathing rate, talking, blinks, expression, validation status&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Platform&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Electron desktop app, English and Portuguese&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Privacy&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Camera only after consent, no frames saved, no video recorded, SDK telemetry off&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Tested&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;19,000 synthetic-signal tests with known answers (my rules, not Presage's accuracy). A living face was called a ghost &lt;strong&gt;0 times in 400&lt;/strong&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Play it now&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;
&lt;a href="https://jenniferevangelista-de.github.io/who-else-was-there/" rel="noopener noreferrer"&gt;Web demo in your browser&lt;/a&gt;, on sample numbers. The live camera reading runs in the desktop app, because the Presage SDK is native (Node.js/Electron)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Code&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;
&lt;a href="https://github.com/jenniferevangelista-de/who-else-was-there" rel="noopener noreferrer"&gt;github.com/jenniferevangelista-de/who-else-was-there&lt;/a&gt; (MIT)&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h2&gt;
  
  
  What I Built
&lt;/h2&gt;

&lt;p&gt;Most camera-vitals apps measure the pulse that is there. &lt;strong&gt;Ghost Hunt&lt;/strong&gt; is a cute, spooky game that hunts for the pulse that is &lt;strong&gt;missing&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;Plenty of people would play it for real. In a 2009 Pew Research Center survey, &lt;strong&gt;18% of U.S. adults&lt;/strong&gt; said they had seen a ghost or been in the presence of one, and &lt;strong&gt;29%&lt;/strong&gt; said they had felt in touch with someone who had already died (&lt;a href="https://www.pewresearch.org/short-reads/2015/10/30/18-of-americans-say-theyve-seen-a-ghost/" rel="noopener noreferrer"&gt;Pew Research Center&lt;/a&gt;). Ghost Hunt gives that feeling an instrument, and a rule: nothing counts unless the camera measured it.&lt;/p&gt;

&lt;p&gt;Three tiny ghost hunters point a new instrument at Marina's tape, the Ghost Detector. Living faces have a heartbeat. The face that does not is the case.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The story, in four beats&lt;/strong&gt;&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th&gt;What happens&lt;/th&gt;
&lt;th&gt;What the detector measured&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;Marina stops at her door and stays home&lt;/td&gt;
&lt;td&gt;A held breath of almost 4 seconds, and her pulse climbing&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;td&gt;A second face stands on the stairs behind her&lt;/td&gt;
&lt;td&gt;Two faces in the frame, one of them at 0 beats per minute&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;3&lt;/td&gt;
&lt;td&gt;You ask the spirit, and it answers: Odette, her mother, who left by that door and never came back&lt;/td&gt;
&lt;td&gt;The glass moves only when your own chest breathes out&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;4&lt;/td&gt;
&lt;td&gt;The crew says goodbye. Then something else writes on the board&lt;/td&gt;
&lt;td&gt;Nobody is breathing out. The glass moves anyway&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fz3f3l2ig3niw408xfg77.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fz3f3l2ig3niw408xfg77.jpg" alt="The ending: the whole screen says I AM HERE" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Odette was a mother standing between her daughter and that door. The ending asks the question in the title one more time: who else was there?&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Point&lt;/th&gt;
&lt;th&gt;Ask&lt;/th&gt;
&lt;th&gt;Report&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F75cmjllm0pz63cjvlpe1.jpg" alt="The detector locked on a second face that reads 0 beats per minute" width="800" height="450"&gt;&lt;/td&gt;
&lt;td&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fwxck2mxiwoc7fyog5jos.jpg" alt="The spirit board, where the glass moves only when you breathe out" width="800" height="450"&gt;&lt;/td&gt;
&lt;td&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fs2zaq0c02bhprhiyl1qa.jpg" alt="The report with a verdict, built only from measured cards" width="800" height="450"&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;strong&gt;What you can do in it&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Browse 30 nights of tape&lt;/strong&gt; on a haunted calendar, and let the detector analyze any night.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Analyze the tape:&lt;/strong&gt; the detector watches the whole hallway and reports the moment a second face appears.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Point the detector at one spot:&lt;/strong&gt; it finds the second face and locks on. 0 beats per minute.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Collect evidence cards&lt;/strong&gt; that jump to the exact moment on the tape.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Ask the spirit&lt;/strong&gt; on a board that moves only when &lt;strong&gt;you&lt;/strong&gt; breathe out, and waits if you talk.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Learn what it is.&lt;/strong&gt; Each spirit gets one of seven kinds, picked from the evidence: Guardian, Haunter, Watcher, Intruder, Shade, Veiled or Visitor. Luna, the medium, deciphers what it wants. Dr. Bigotes, the skeptic, refuses to sign it. Xochi writes it all down.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Get read yourself in The Room:&lt;/strong&gt; your real pulse, breathing and expression, live from your camera. A figure moves only when you blink. On your report, a small spirit board keeps listening: breathe out and the glass slides to YES. In the web demo, the report writes itself after ten seconds on sample numbers, so you can read one without installing anything.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Upload your own video,&lt;/strong&gt; once you confirm everyone in it agreed. Presage reads it as it plays, and you get a report with a verdict: is there a spirit in it?&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fj5ez4z0o8vjae6pzhrne.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fj5ez4z0o8vjae6pzhrne.jpg" alt="Nine screens of the app. From top left: the crew, the 30 nights, the tape, one spot, the spirit board, The Room, the main menu, the proof tab and the rules." width="800" height="453"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;It is a desktop app in Electron, in English and Portuguese, with original music and voices. Marina's story is drawn and tagged as sample data on every screen. The Room and uploaded videos are real SDK readings.&lt;/p&gt;

&lt;h2&gt;
  
  
  Demo
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Demo, with the live signal:&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;  &lt;iframe src="https://www.youtube.com/embed/Z8e7KSFJ59o" width="710" height="399"&gt;
  &lt;/iframe&gt;
&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Pitch:&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;  &lt;iframe src="https://www.youtube.com/embed/Uq_3nqf3R5w" width="710" height="399"&gt;
  &lt;/iframe&gt;
&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Play it in your browser, no install:&lt;/strong&gt; &lt;a href="https://jenniferevangelista-de.github.io/who-else-was-there/" rel="noopener noreferrer"&gt;https://jenniferevangelista-de.github.io/who-else-was-there/&lt;/a&gt; (sample numbers).&lt;/p&gt;

&lt;p&gt;No login is needed. Without a Presage key the app opens on sample numbers, and every sample screen is tagged. With a key, The Room reads your camera live.&lt;/p&gt;

&lt;h2&gt;
  
  
  Code
&lt;/h2&gt;


&lt;div class="ltag-github-readme-tag"&gt;
  &lt;div class="readme-overview"&gt;
    &lt;h2&gt;
      &lt;img src="https://assets.dev.to/assets/github-logo-5a155e1f9a670af7944dd5e12375bc76ed542ea80224905ecaf878b9157cdefc.svg" alt="GitHub logo"&gt;
      &lt;a href="https://github.com/jenniferevangelista-de" rel="noopener noreferrer"&gt;
        jenniferevangelista-de
      &lt;/a&gt; / &lt;a href="https://github.com/jenniferevangelista-de/who-else-was-there" rel="noopener noreferrer"&gt;
        who-else-was-there
      &lt;/a&gt;
    &lt;/h2&gt;
    &lt;h3&gt;
      
    &lt;/h3&gt;
  &lt;/div&gt;
  &lt;div class="ltag-github-body"&gt;
    
&lt;div id="readme" class="md"&gt;&lt;div class="markdown-heading"&gt;
&lt;h1 class="heading-element"&gt;Ghost Hunt · Who Else Was There?&lt;/h1&gt;
&lt;/div&gt;
&lt;p&gt;&lt;strong&gt;Every face in this video has a heartbeat. Except one.&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Most camera-vitals apps measure the pulse that is there. Ghost Hunt is a cute, spooky game that hunts for the pulse that is &lt;strong&gt;missing&lt;/strong&gt;. It runs on the &lt;a href="https://smartspectra.presagetech.com/docs" rel="nofollow noopener noreferrer"&gt;Presage SmartSpectra SDK&lt;/a&gt;, which reads a person's pulse and breathing from an ordinary camera.&lt;/p&gt;
&lt;p&gt;Entry for the DEV Presage SmartSpectra SDK Challenge, category &lt;strong&gt;Most Creative or Delightful Use&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;&lt;a href="https://github.com/jenniferevangelista-de/who-else-was-there/actions/workflows/ci.yml" rel="noopener noreferrer"&gt;&lt;img src="https://github.com/jenniferevangelista-de/who-else-was-there/actions/workflows/ci.yml/badge.svg" alt="ci"&gt;&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Play it in your browser:&lt;/strong&gt; &lt;a href="https://jenniferevangelista-de.github.io/who-else-was-there/" rel="nofollow noopener noreferrer"&gt;jenniferevangelista-de.github.io/who-else-was-there&lt;/a&gt; (sample numbers; the live camera reading needs the desktop app)&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Videos:&lt;/strong&gt; &lt;a href="https://youtu.be/Z8e7KSFJ59o" rel="nofollow noopener noreferrer"&gt;Demo with the live signal (2:38)&lt;/a&gt; · &lt;a href="https://youtu.be/Uq_3nqf3R5w" rel="nofollow noopener noreferrer"&gt;Pitch (2:23)&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Contents:&lt;/strong&gt; &lt;a href="https://github.com/jenniferevangelista-de/who-else-was-there#the-game-in-one-minute" rel="noopener noreferrer"&gt;The game in one minute&lt;/a&gt; · &lt;a href="https://github.com/jenniferevangelista-de/who-else-was-there#everything-you-can-do" rel="noopener noreferrer"&gt;Everything you can do&lt;/a&gt; · &lt;a href="https://github.com/jenniferevangelista-de/who-else-was-there#the-challenges-questions-answered" rel="noopener noreferrer"&gt;The challenge's questions, answered&lt;/a&gt; · &lt;a href="https://github.com/jenniferevangelista-de/who-else-was-there#how-the-sdk-is-used" rel="noopener noreferrer"&gt;How the SDK is used&lt;/a&gt; · &lt;a href="https://github.com/jenniferevangelista-de/who-else-was-there#how-we-tested-it" rel="noopener noreferrer"&gt;How we tested it&lt;/a&gt; · &lt;a href="https://github.com/jenniferevangelista-de/who-else-was-there#run-it-yourself" rel="noopener noreferrer"&gt;Run it yourself&lt;/a&gt; · &lt;a href="https://github.com/jenniferevangelista-de/who-else-was-there#consent-and-data-handling" rel="noopener noreferrer"&gt;Consent and data&lt;/a&gt;&lt;/p&gt;
&lt;div class="markdown-heading"&gt;
&lt;h2 class="heading-element"&gt;The game in one minute&lt;/h2&gt;
&lt;/div&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;A case arrives.&lt;/strong&gt; Marina was about to go…&lt;/li&gt;
&lt;/ol&gt;&lt;/div&gt;
  &lt;/div&gt;
  &lt;div class="gh-btn-container"&gt;&lt;a class="gh-btn" href="https://github.com/jenniferevangelista-de/who-else-was-there" rel="noopener noreferrer"&gt;View on GitHub&lt;/a&gt;&lt;/div&gt;
&lt;/div&gt;


&lt;p&gt;Setup is in the README, along with a walkthrough of every screen. The SDK integration is in &lt;code&gt;app/renderer/presage.js&lt;/code&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  How It Responds to Human Signals
&lt;/h2&gt;

&lt;p&gt;Seven SmartSpectra signals, and each one has a job in the game.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fv80ptdfpf29oioi0yuy0.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fv80ptdfpf29oioi0yuy0.png" alt="Signal map" width="800" height="449"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;How to read it: each row is a signal, each column is a part of the game, and a big green dot means that signal changes something on that screen.&lt;/em&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Pulse&lt;/strong&gt; (&lt;code&gt;cardio.pulseRate&lt;/code&gt;) drives the heart and the detector line, in real time. Readings under 50% confidence do not count.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Chest movement&lt;/strong&gt; (&lt;code&gt;breathing.upperTrace&lt;/code&gt;) moves the séance glass when you breathe out, and finds a held breath.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Talking&lt;/strong&gt; (&lt;code&gt;face.talking&lt;/code&gt;) makes the glass wait while you speak.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Blinks&lt;/strong&gt; (&lt;code&gt;face.blinking&lt;/code&gt;) are the only thing that moves the figure in the Room.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Breathing rate&lt;/strong&gt; (&lt;code&gt;breathing.rate&lt;/code&gt;) and &lt;strong&gt;expression&lt;/strong&gt; (&lt;code&gt;face.expression&lt;/code&gt;) become your numbers and your evidence cards.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Validation status&lt;/strong&gt; (&lt;code&gt;validationStatus&lt;/code&gt;) turns "multiple faces" into &lt;em&gt;a second face&lt;/em&gt;, and poor light into "unreadable". A badly lit face is never called a ghost.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;One rule keeps the spirit honest: &lt;strong&gt;cite or vanish&lt;/strong&gt;. There is no language model. Every line the spirit says must cite a measured card, and every number in it must match that card, or the line is not said.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fdp6ssenpcjxbeqyqqliz.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fdp6ssenpcjxbeqyqqliz.png" alt="Architecture" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;How to read it: follow the top row from left to right. The camera goes into Presage's SDK, our code turns its numbers into evidence cards, and only then can the spirit speak.&lt;/em&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  How I tested it, including where it breaks
&lt;/h3&gt;

&lt;p&gt;The computer vision belongs to Presage, and I treat it as an instrument. What is mine is the logic that decides what the numbers mean, so I tested that. It works like hiding a fake ghost to see if the detector finds it: &lt;strong&gt;19,000 synthetic-signal tests where I already know the answer&lt;/strong&gt;, reproducible with &lt;code&gt;npm run benchmark&lt;/code&gt;. They test my rules, not Presage's accuracy, and no real person is in them.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fxrgjesgfuse9afi8mfmr.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fxrgjesgfuse9afi8mfmr.png" alt="1,000 right answers out of 1,000" width="800" height="449"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;How to read it: the left side is what I hid, the top is what the detector answered. Numbers on the green diagonal are right answers. Anything off it would be a mistake.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;A living face was called a ghost &lt;strong&gt;0 times in 400&lt;/strong&gt;. Then I pushed the noise up until the rules failed, because that is the useful part.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fhbi3pljufbt8tcrqsv8r.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fhbi3pljufbt8tcrqsv8r.png" alt="Pulse stress test" width="800" height="449"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;How to read it: each number is how many of 200 tests got the right answer. Green is right, pink is wrong. Going up, the reading gets more jittery. Real jumps are always caught, but from 9 bpm of jitter the alarm starts going off by itself.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;The README explains every chart and lists every limit I found.&lt;/p&gt;

&lt;h2&gt;
  
  
  What the SDK Taught Me
&lt;/h2&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Confidence is part of the signal.&lt;/strong&gt; A pulse under 50% confidence does not count. Without that rule, a shaky reading looks exactly like a ghost.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Bad light is not a spirit.&lt;/strong&gt; The SDK's validation status says when a face cannot be read, and the game turns that into "unreadable" instead of a scary verdict.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The SDK wants a real device.&lt;/strong&gt; I tried a reading on a cloud server (GitHub Actions) and Presage refused to pair it (HTTP 403). The live readings in the demo come from a real computer with a camera.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Windows can block it.&lt;/strong&gt; With Smart App Control on, Windows blocks the SDK's unsigned libraries. The app still opens on tagged sample numbers, so nobody gets a blank screen.&lt;/li&gt;
&lt;/ol&gt;

&lt;h2&gt;
  
  
  Prize Categories
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Most Creative or Delightful Use.&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;What inspired it:&lt;/strong&gt; almost one in five U.S. adults says they have been in the presence of a ghost. I wanted to give that feeling an instrument, and one rule: nothing counts unless the camera measured it.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Unexpected:&lt;/strong&gt; most vitals apps look for the pulse that is there. This one hunts for the pulse that is missing, and a face with no heartbeat becomes the whole mystery.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Playful:&lt;/strong&gt; your own breath is the only thing that moves the glass, and it waits while you talk. A blink is a jump scare with rules. A skeptic cat refuses to sign anything the numbers can't prove.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Delight with care:&lt;/strong&gt; the scare never becomes a health claim. Bad light says &lt;em&gt;unreadable&lt;/em&gt;, never &lt;em&gt;ghost&lt;/em&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;One click to play:&lt;/strong&gt; the web demo opens in the browser, on a computer or a phone.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Consent and Data Handling
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Camera consent:&lt;/strong&gt; a rules screen comes first. The camera turns on only when you press a button, a pill at the top right turns it off at any time, and the microphone is never requested.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Other people's videos:&lt;/strong&gt; the file picker stays locked until you confirm that everyone in the recording agreed.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Storage and retention:&lt;/strong&gt; no frames saved and no video recorded. Readings live in memory and vanish when the app closes. A report is written only if you press Save, with numbers and words only.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Sharing:&lt;/strong&gt; the app has no server of its own and sends nothing to one.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The SDK:&lt;/strong&gt; frames are handed to Presage SmartSpectra inside the app, which uses your API key and Presage's authenticated channel. Presage's public documentation does not say where frames are processed, so I do not claim on-device processing. See Presage's Telemetry and Privacy page and Privacy Policy.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;SDK telemetry:&lt;/strong&gt; switched off (&lt;code&gt;enableTelemetry: false&lt;/code&gt;).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Deletion:&lt;/strong&gt; close the app and the readings are gone. Delete a saved report like any file.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;AI services:&lt;/strong&gt; none at run time.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Not medical:&lt;/strong&gt; this is a game. It does not diagnose, treat or monitor anyone.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Every Requirement, Checked
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;The challenge asks for&lt;/th&gt;
&lt;th&gt;Where it is&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;A functional app on the SmartSpectra SDK&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;app/renderer/presage.js&lt;/code&gt;, &lt;code&gt;@smartspectra/node-sdk&lt;/code&gt; 3.4.0&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;At least one live Presage signal&lt;/td&gt;
&lt;td&gt;Seven, listed above and shown live in the demo&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;A DEV post on the template, with the tags&lt;/td&gt;
&lt;td&gt;This post&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;A public repository with setup instructions&lt;/td&gt;
&lt;td&gt;The repository above, README&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;A demo video&lt;/td&gt;
&lt;td&gt;Linked in Demo&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Testing instructions if login is needed&lt;/td&gt;
&lt;td&gt;No login is needed&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Consent, storage, sharing, retention, deletion, SDK telemetry and AI services&lt;/td&gt;
&lt;td&gt;Consent and Data Handling, above&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;No diagnosis or treatment claims, no medical decisions&lt;/td&gt;
&lt;td&gt;It is a game, and says so on screen&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;No covert monitoring&lt;/td&gt;
&lt;td&gt;Consent comes first, and other people's videos stay locked until you confirm they agreed&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Built with AI coding assistance. The story, design decisions and testing direction are mine.&lt;/p&gt;

</description>
      <category>devchallenge</category>
      <category>presagechallenge</category>
      <category>webdev</category>
      <category>javascript</category>
    </item>
    <item>
      <title>The ghost refused to leave my ghost app: building Witching Hour with TabPFN and Gemma</title>
      <dc:creator>Jennifer Evangelista</dc:creator>
      <pubDate>Mon, 09 Nov 2026 20:05:29 +0000</pubDate>
      <link>https://dev.to/jenniferevangelistade/the-ghost-refused-to-leave-my-ghost-app-building-witching-hour-with-tabpfn-and-gemma-3co</link>
      <guid>https://dev.to/jenniferevangelistade/the-ghost-refused-to-leave-my-ghost-app-building-witching-hour-with-tabpfn-and-gemma-3co</guid>
      <description>&lt;p&gt;Witching Hour is a ghost-hunting app with one strict rule. You give it a recording of a place, the app measures what really happened in it (a light flickering, a knock, something moving), and the "spirit" on the Ouija board can only talk about those measured clues. No evidence, no ghost.&lt;/p&gt;

&lt;p&gt;On the Friday before Halloween, I was doing one last test before calling it done. I opened one recording, then switched to a second one. The board started to answer, and something was off: the spirit was talking about the first recording. The app was showing the analysis of the old file, with old data, on top of the new one.&lt;/p&gt;

&lt;p&gt;That sounds small, but it broke the one promise the app makes. A spirit that only speaks from evidence is worthless if the evidence comes from the wrong recording. And the deadline was a day and a half away.&lt;/p&gt;

&lt;p&gt;I laughed a little, because I had literally built a ghost that wouldn't leave, and then I opened the debugger.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;TL;DR:&lt;/strong&gt; &lt;strong&gt;Witching Hour&lt;/strong&gt; is a ghost-hunting web app that reimagines the Ouija board with real measurements and open models. &lt;strong&gt;TabPFN&lt;/strong&gt;, a foundation model for tables, is the "spirit meter": it learns what a house usually does and flags the nights that don't fit. &lt;strong&gt;Gemma 4&lt;/strong&gt; tells the story, but under one rule: &lt;strong&gt;cite or vanish&lt;/strong&gt;. Every sentence must point to a measured event, and every number in it must match, or it is deleted. In the main case, a staged hallway clip, the browser measures a light flicker at 7 seconds and three knocks, and a fictional spirit gets to use only those clues. &lt;strong&gt;The evidence is real. The spirits are fiction.&lt;/strong&gt; Built in Halloween week for the Hacktoberfest 2026 Open-Source AI Challenge, Week 4: Ghost in the Machine.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F5vl6fr9qdbw4y21kim38.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F5vl6fr9qdbw4y21kim38.png" alt="The Warning: a staged hallway scene, a woman in her coat near the front door" width="640" height="360"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  What I Built
&lt;/h2&gt;

&lt;p&gt;Imagine this: coat on, keys in hand, you are about to go out. But you get a strange feeling. Something feels wrong. You put the keys down and decide to stay home.&lt;/p&gt;

&lt;p&gt;Later, you watch the hallway recording. The light flickers. Three knocks. Something crosses the edge of the frame.&lt;/p&gt;

&lt;p&gt;Bad bulb? A draft? Or the reason you stayed?&lt;/p&gt;

&lt;p&gt;Plenty of people would answer the third option. In a 2009 Pew Research Center survey, &lt;strong&gt;18% of U.S. adults&lt;/strong&gt; said they had seen a ghost or been in the presence of one, and &lt;strong&gt;29%&lt;/strong&gt; said they had felt in touch with someone who had already died (&lt;a href="https://www.pewresearch.org/short-reads/2015/10/30/18-of-americans-say-theyve-seen-a-ghost/" rel="noopener noreferrer"&gt;Pew Research Center&lt;/a&gt;). And if you ask an AI to explain the hallway, it will happily invent an answer: NewsGuard found that leading chatbots repeated false claims in &lt;strong&gt;35% of news answers in August 2025, up from 18%&lt;/strong&gt; a year earlier (&lt;a href="https://www.vktr.com/ai-technology/ai-hallucinations-nearly-double-heres-why-theyre-getting-worse-not-better/" rel="noopener noreferrer"&gt;NewsGuard, via VKTR&lt;/a&gt;). A believer and a confident storyteller is a dangerous mix. That is why the spirit in Witching Hour is not allowed to invent anything: every sentence has to cite a real measurement from the recording, or it vanishes.&lt;/p&gt;

&lt;p&gt;That is &lt;strong&gt;The Warning&lt;/strong&gt;, the heart of the demo. Kenji, a fictional client in Japan, brings a twenty-second staged recording to a very small, very opinionated ghost-hunting crew:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Luna&lt;/strong&gt;, a calaverita medium, wants to ask the spirit.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Dr. Bigotes&lt;/strong&gt;, a black cat with a clipboard, wants to check the light bulb.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Xochi&lt;/strong&gt;, a xoloitzcuintli night watch, writes a calm note for the family in the morning.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Faxp2z7sawsxxnw9j01a3.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Faxp2z7sawsxxnw9j01a3.png" alt="Luna the Medium, Dr. Bigotes the Skeptic and Xochi the Night Watch" width="799" height="202"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The app, called &lt;strong&gt;PARANORMAL.exe&lt;/strong&gt;, opens three cases:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;The House:&lt;/strong&gt; four months of sensor readings from a real research home. Find the strange night, then hear a ghost story, a skeptic's explanation and a morning note, all about the same numbers.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The Unheard:&lt;/strong&gt; a recording. The browser measures light, motion and sound, marks the moments that stand out, and takes those clues to a fictional séance.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The Room:&lt;/strong&gt; your turn. Camera and microphone, a short baseline, and the board answers what changes around you.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Demo
&lt;/h2&gt;

&lt;p&gt;🎬 The film and the tour:&lt;/p&gt;

&lt;p&gt;  &lt;iframe src="https://www.youtube.com/embed/WquGHQsFt1o" width="710" height="399"&gt;
  &lt;/iframe&gt;
&lt;/p&gt;

&lt;p&gt;  &lt;iframe src="https://www.youtube.com/embed/biTBYK7cuwI" width="710" height="399"&gt;
  &lt;/iframe&gt;
&lt;/p&gt;

&lt;p&gt;🔮 &lt;strong&gt;Live demo:&lt;/strong&gt; &lt;a href="https://witching-hour.onrender.com" rel="noopener noreferrer"&gt;witching-hour.onrender.com&lt;/a&gt; (the free server sleeps, so give it a minute to wake up, like any good ghost).&lt;/p&gt;


&lt;div class="ltag-github-readme-tag"&gt;
  &lt;div class="readme-overview"&gt;
    &lt;h2&gt;
      &lt;img src="https://assets.dev.to/assets/github-logo-5a155e1f9a670af7944dd5e12375bc76ed542ea80224905ecaf878b9157cdefc.svg" alt="GitHub logo"&gt;
      &lt;a href="https://github.com/jenniferevangelista-de" rel="noopener noreferrer"&gt;
        jenniferevangelista-de
      &lt;/a&gt; / &lt;a href="https://github.com/jenniferevangelista-de/witching-hour" rel="noopener noreferrer"&gt;
        witching-hour
      &lt;/a&gt;
    &lt;/h2&gt;
    &lt;h3&gt;
      What happened in the house at 3 AM? TabPFN learns what was normal, Gemma tells the story, every sentence cites a sensor.
    &lt;/h3&gt;
  &lt;/div&gt;
  &lt;div class="ltag-github-body"&gt;
    
&lt;div id="readme" class="md"&gt;&lt;div class="markdown-heading"&gt;
&lt;h1 class="heading-element"&gt;Witching Hour&lt;/h1&gt;
&lt;/div&gt;
&lt;p&gt;&lt;strong&gt;What if someone tried to warn you, and you never heard them?&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Witching Hour is a fictional ghost-hunting experience built for Halloween and Día de los Muertos, using real sensor anomaly measurements. A crew of three tiny ghost hunters carries an open-source kit, and their terminal, &lt;strong&gt;PARANORMAL.exe&lt;/strong&gt;, opens three kinds of cases:&lt;/p&gt;
&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Case&lt;/th&gt;
&lt;th&gt;What you bring&lt;/th&gt;
&lt;th&gt;What the kit measures&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;🏚️ &lt;strong&gt;The House&lt;/strong&gt;
&lt;/td&gt;
&lt;td&gt;a house's sensor log (the demo uses a real house in Belgium, four months, every 10 minutes)&lt;/td&gt;
&lt;td&gt;
&lt;strong&gt;TabPFN&lt;/strong&gt;, the spirit meter, learns what every room should feel like at every minute of the night and flags the readings that break the pattern&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;🎞️ &lt;strong&gt;The Unheard&lt;/strong&gt;
&lt;/td&gt;
&lt;td&gt;a recording of the night you almost went out and something made you stay&lt;/td&gt;
&lt;td&gt;light, motion and sound, frame by frame, &lt;strong&gt;inside the browser&lt;/strong&gt; (OpenCV does the same in Python); the moments that stand out become numbered evidence&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;…&lt;/div&gt;
  &lt;/div&gt;
  &lt;div class="gh-btn-container"&gt;&lt;a class="gh-btn" href="https://github.com/jenniferevangelista-de/witching-hour" rel="noopener noreferrer"&gt;View on GitHub&lt;/a&gt;&lt;/div&gt;
&lt;/div&gt;


&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fmum2deat7z30xzd4drlr.gif" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fmum2deat7z30xzd4drlr.gif" alt="The glass moving across the recorded séance board" width="640" height="360"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  How It Works
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fn3xysbntfwfc5zy2zpvu.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fn3xysbntfwfc5zy2zpvu.png" alt="Witching Hour architecture: evidence measured by TabPFN and the browser, told by Gemma, checked by cite or vanish" width="800" height="467"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  The spirit meter knows what this house usually does
&lt;/h3&gt;

&lt;p&gt;For each sensor, TabPFN v2 reads the previous fourteen days as context and returns a range for every ten-minute slot from midnight to 6 a.m. Outdoor weather and sensors in other rooms are features. A reading far outside its range becomes a numbered event: &lt;code&gt;E1&lt;/code&gt;, &lt;code&gt;E2&lt;/code&gt;, &lt;code&gt;E3&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Then I planted ghosts on purpose: smooth artificial bumps in &lt;strong&gt;240 night-sensor pairs&lt;/strong&gt;, the same for every detector, with thresholds aimed at a 5% false-alarm rate.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Detector&lt;/th&gt;
&lt;th&gt;Planted cases found&lt;/th&gt;
&lt;th&gt;Strongest cases found&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;TabPFN v2&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;25.0%&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;37.8%&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;LightGBM quantile regression&lt;/td&gt;
&lt;td&gt;9.6%&lt;/td&gt;
&lt;td&gt;14.4%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Same-hour seasonal rule&lt;/td&gt;
&lt;td&gt;13.3%&lt;/td&gt;
&lt;td&gt;21.1%&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;TabPFN found the most, and three quarters of the faint ghosts still got away. I left that in the README on purpose. A ghost app that admits what it misses felt more honest than one that sees ghosts everywhere.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F86uw5l2j6xmddhx1p5fs.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F86uw5l2j6xmddhx1p5fs.png" alt="Chart: TabPFN catches 2x more planted ghosts. Ghosts caught: TabPFN 25%, seasonal rule 13%, LightGBM 10%. Found the right time: 87%, 66%, 61%. Honest 90% band: 95%, 84%, 59%. AUC 0.644, 0.634, 0.573" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F9b5r2hyqr7cp1x3me97b.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F9b5r2hyqr7cp1x3me97b.png" alt="The House: the haunting map" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Cite or vanish
&lt;/h3&gt;

&lt;p&gt;Gemma doesn't get the raw log. It gets small evidence cards, and every sentence comes back with the IDs it is based on:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nl"&gt;"text"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"At 05:40, the machines woke up."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nl"&gt;"cites"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"E1"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Then the validator decides if the sentence lives:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;check_sentence&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;cites&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;events_by_id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;night&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="kc"&gt;None&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;window&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;6&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="s2"&gt;"""None if the sentence is grounded, otherwise the reason it is not."""&lt;/span&gt;
    &lt;span class="n"&gt;valid&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;c&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;cites&lt;/span&gt; &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;events_by_id&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="ow"&gt;not&lt;/span&gt; &lt;span class="n"&gt;valid&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="s2"&gt;"cites no known event or check"&lt;/span&gt;
    &lt;span class="n"&gt;times&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;values&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;allowed_numbers&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;events_by_id&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;valid&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;night&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;extra&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;m&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;_NUM&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;finditer&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;m&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;group&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;is&lt;/span&gt; &lt;span class="ow"&gt;not&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;          &lt;span class="c1"&gt;# a time like 05:40&lt;/span&gt;
            &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"{int(m.group(1)):02d}:{m.group(2)}"&lt;/span&gt; &lt;span class="ow"&gt;not&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;times&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
                &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"time {m.group(0)} is not in the cited events"&lt;/span&gt;
        &lt;span class="k"&gt;else&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;                               &lt;span class="c1"&gt;# a number like 26&lt;/span&gt;
            &lt;span class="n"&gt;v&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kt"&gt;float&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;m&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;group&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;replace&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;","&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"."&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
            &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="ow"&gt;not&lt;/span&gt; &lt;span class="nb"&gt;any&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;abs&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;v&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="n"&gt;a&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;=&lt;/span&gt; &lt;span class="nb"&gt;max&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mf"&gt;0.11&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mf"&gt;0.02&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="nb"&gt;abs&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;a&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;a&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;values&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
                &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"number {m.group(3)} is not in the cited events"&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;No known citation: gone. A time that isn't in the cited events: gone. A number that doesn't match: gone. That is the whole trick, and it is surprisingly hard to argue with.&lt;/p&gt;

&lt;p&gt;It doesn't prove the &lt;em&gt;explanation&lt;/em&gt;. Dr. Bigotes may blame the kettle with total confidence, and the kettle is still a hypothesis. But it does stop the story from inventing the facts.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F7922dl6lrmzk0f2wxfvp.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F7922dl6lrmzk0f2wxfvp.png" alt="The House, one night and three voices: Luna tells the story, every sentence cites the measured spike, and the spirit meter reads a full apparition" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  The spirit has a contract too
&lt;/h3&gt;

&lt;p&gt;On the board, the spirit gets at most three words, known evidence IDs and an explicit "fictional" flag. When the Guardian spells &lt;strong&gt;I AM HERE&lt;/strong&gt;, it belongs to the fiction. The flicker at seven seconds still belongs to the clip.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F4vj0dtlj7omkyq5dv926.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F4vj0dtlj7omkyq5dv926.png" alt="The recorded séance: Luna reads the glass, Bigotes checks the evidence, Xochi keeps watch" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Partner Technologies
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;TabPFN v2:&lt;/strong&gt; the spirit meter, on CPU, no login, benchmarked against LightGBM and a seasonal rule.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Gemma 4 E4B via Ollama:&lt;/strong&gt; every story, every spirit, on my own machine. In a bake-off of four remote open-weight models through &lt;strong&gt;Backboard&lt;/strong&gt;, the local Gemma reference passed 45/45 sentences.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Tinker:&lt;/strong&gt; I fine-tuned Qwen3-8B with LoRA on stories the validator accepted. On unseen nights, sentences passing went from &lt;strong&gt;84/88 to 124/125&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;ElevenLabs:&lt;/strong&gt; the whole cast has a voice: 114 recorded files.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Temporal, Sentry, Mastra, Browserbase, SerpApi, Render, GitHub Actions:&lt;/strong&gt; a night investigation that survives a killed worker, traced spans for every model call, a TypeScript agent with memory, a skeptic who checks the official weather page himself, and the live demo.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Ft9fbfcrk4aum6osst3jp.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Ft9fbfcrk4aum6osst3jp.png" alt="Chart: which spirit makes things up the least? Share of story sentences whose numbers and times match the cited sensor: Gemma 4 E4B local 100% (45/45, 12/12 stories), Mistral Small 3.1 100% but only 4 of 12 valid stories, Qwen 2.5 72B 97%, Llama 3.3 70B 96%, Gemma 3 27B 96%" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Hackathon Experience
&lt;/h2&gt;

&lt;p&gt;The challenge theme was &lt;em&gt;Ghost in the Machine&lt;/em&gt;, and I had one rule for myself: no fake magic. If my app says a light flickered, it measured a light flickering.&lt;/p&gt;

&lt;p&gt;The week, honestly:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;October 28 was 60 commits.&lt;/strong&gt; The spirit meter, the narrator, the validator, the crew. I was having way too much fun naming a cat.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;That same afternoon, the story changed.&lt;/strong&gt; I had a second idea brewing, a terminal called PARANORMAL.exe where a &lt;em&gt;good&lt;/em&gt; spirit tries to warn someone: "don't go out today". The person stays. Only later does the board reveal the message. Two projects, one deadline. I decided to merge them, and that is how The Warning, The Unheard and The Room were born, all tied by one question: &lt;em&gt;what if someone tried to warn you, and you never heard them?&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;My home page was 10,035 pixels tall.&lt;/strong&gt; Every feature I added landed on the same scroll. A navigation bar and hash routes later, it was 1,360. Kill your darlings, or at least give them tabs.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The ghost that wouldn't leave.&lt;/strong&gt; The opening scene, on October 30. When you picked a new file, the old transcript could linger, and a slow replay finishing late could repaint the old story over the new case. Correct citations are useless if they belong to the wrong recording. The fix was a version number on every recording:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;beginRecording&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;kind&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;version&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="o"&gt;++&lt;/span&gt;&lt;span class="n"&gt;recordingVersion&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;   &lt;span class="c1"&gt;// a new case starts here&lt;/span&gt;
  &lt;span class="n"&gt;recordingRequest&lt;/span&gt;&lt;span class="o"&gt;?&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;abort&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;            &lt;span class="c1"&gt;// the old one is cancelled&lt;/span&gt;
  &lt;span class="n"&gt;recordingRequest&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nc"&gt;AbortController&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="nf"&gt;resetBoard&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"uh"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="c1"&gt;// ...every async answer checks recordingCurrent(version) before it paints&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;An old answer can still arrive. It just isn't allowed to paint anymore. Five Playwright tests now guard it, and the suite on &lt;code&gt;main&lt;/code&gt; passes 81 tests.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Halloween itself was 39 commits.&lt;/strong&gt; The last ones were the videos, with a creepy score made in code, at 11 p.m. The deadline was 03:59 my time, and I finished with a few hours to spare.&lt;/p&gt;

&lt;p&gt;What I learned: open models make it possible to &lt;em&gt;argue&lt;/em&gt; with your pipeline. I could swap the detector, plant the same ghosts, fine-tune a smaller narrator and check if it learned the same discipline. With a closed API, I would have had a story. With open models, I had a story &lt;em&gt;and&lt;/em&gt; the receipts.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fzfzzdrqtprhhbgpr6wxo.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fzfzzdrqtprhhbgpr6wxo.jpg" alt="The evidence receipts panel" width="800" height="537"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  What It Can't Do Yet
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;It misses most faint ghosts.&lt;/strong&gt; 25% of planted cases found at a 5% false-alarm target. Thresholds and evaluation share the same sample, so this is exploratory, not a guarantee.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Citations locate, they don't prove.&lt;/strong&gt; "Cite or vanish" catches invented numbers and missing references. It doesn't verify every explanation.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The public demo uses recorded or template answers&lt;/strong&gt; for the model, so it stays fast and free. The live model path needs a configured provider.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;One house, from 2016.&lt;/strong&gt; The "tonight" forecast applies that house's model to today's weather without matching validation.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Privacy depends on the install.&lt;/strong&gt; Video is analysed in the browser, but summoning sends event descriptions to the app. A fully local install can switch the external services off.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Thank You
&lt;/h2&gt;

&lt;p&gt;To &lt;strong&gt;DEV&lt;/strong&gt; and &lt;strong&gt;MLH&lt;/strong&gt; for Hacktoberfest 2026 and a theme that let me build something spooky &lt;em&gt;and&lt;/em&gt; serious. To the teams behind &lt;strong&gt;TabPFN&lt;/strong&gt;, &lt;strong&gt;Gemma&lt;/strong&gt; and every partner tool that had a free tier generous enough for a Halloween-week builder. To the researchers who published the house dataset. And to Luna, Dr. Bigotes and Xochi, who argued with each other so I didn't have to.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The evidence is real. The spirits are fiction. The feeling is yours.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;All characters, clients and spirits are fictional; the clips are staged.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>mlhacks</category>
      <category>ai</category>
      <category>opensource</category>
      <category>machinelearning</category>
    </item>
    <item>
      <title>I had never written a line of Solana. Two days later, my wallet was refusing scams on-chain</title>
      <dc:creator>Jennifer Evangelista</dc:creator>
      <pubDate>Mon, 09 Nov 2026 20:05:27 +0000</pubDate>
      <link>https://dev.to/jenniferevangelistade/i-had-never-written-a-line-of-solana-two-days-later-my-wallet-was-refusing-scams-on-chain-4m91</link>
      <guid>https://dev.to/jenniferevangelistade/i-had-never-written-a-line-of-solana-two-days-later-my-wallet-was-refusing-scams-on-chain-4m91</guid>
      <description>&lt;p&gt;My first Solana program has one job: stand between an older parent and a scammer.&lt;/p&gt;

&lt;p&gt;Picture an older mother getting a message from her "son": he lost his phone, he is in trouble, please send money right now. In a normal wallet she taps Send and the money is gone for good. Night Guard Wallet adds a pause first: it reads the message, simulates the transfer, checks who is on the other side, and asks her daughter to approve before anything leaves the vault.&lt;/p&gt;

&lt;p&gt;Two days earlier I had never written a line of Solana. By the end of the weekend, a program I wrote was refusing scam payments on its own, seven times in a row, with my demo app deliberately pushing them through.&lt;/p&gt;

&lt;p&gt;This is the story of &lt;strong&gt;Night Guard Wallet&lt;/strong&gt;, built for the &lt;strong&gt;MLH Solana Hackathon&lt;/strong&gt;.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;TL;DR:&lt;/strong&gt; Night Guard Wallet is a guard for the people scammers target most: our parents and grandparents (and honestly, the rest of us too). Before Lucy, a fictional 74-year-old, signs anything, it simulates the transaction on Solana devnet, checks where every part comes from, lets Gemma 4 read the WhatsApp message that came with it, and explains the verdict in one plain sentence, read aloud. Her savings live in an on-chain vault whose rules nobody can switch off: daily limit, trusted contacts, a family guardian who signs, and a cooling-off for big amounts. When her daughter refuses a scam, the wallet goes on a community scam list on Solana, with on-chain proof, one vote per family. &lt;strong&gt;5/5 scams stopped, 2/2 family payments let through, verdict on screen in 0.2 s, 32/32 scam messages recognised with 0 false alarms, 7 refusals by the program itself.&lt;/strong&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F0ib95r3i3ensf5ntg0pd.gif" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F0ib95r3i3ensf5ntg0pd.gif" alt="Night Guard Wallet demo: Lucy checks a payment and gets a big red STOP in plain words" width="719" height="405"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  What I Built
&lt;/h2&gt;

&lt;p&gt;At 9 pm, Lucy gets a WhatsApp from a number she doesn't know:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;"Hi mom, it's me, I changed my number. I'm in trouble and need to pay a bill urgently today. Send 1.8 SOL to this wallet and don't tell dad."&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;With a bank, there is still a way back: the bank can block a suspicious transfer, call her to confirm, or try to reverse it. A blockchain has none of that. Once she signs, the money is gone and nobody can undo it.&lt;/p&gt;

&lt;p&gt;In 2025, people over 60 reported &lt;strong&gt;US$ 7.7 billion&lt;/strong&gt; lost to fraud to the FBI, &lt;strong&gt;US$ 4.3 billion&lt;/strong&gt; of it involving crypto (FBI IC3 2025, Elder Fraud). The stories are always the same few: the "son" with a new number, the fake support, the free airdrop, the prize with a fee to unlock, the address copied from history. And wallets explain all of them in a language only developers read.&lt;/p&gt;

&lt;p&gt;Night Guard Wallet has two screens and one program:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Lucy's wallet:&lt;/strong&gt; big letters, one button, "Check before signing". The verdict comes in one sentence, in English or Portuguese, and she can listen to it.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Ana's guardian page:&lt;/strong&gt; her daughter sees risky requests live, read straight from the chain, and decides with her own signature.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The &lt;code&gt;guardian_vault&lt;/code&gt; program:&lt;/strong&gt; an Anchor program that holds the savings and enforces the rules, so even if the app is fooled, Solana says no.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F5o6p3vnzyblapm7pg9uv.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F5o6p3vnzyblapm7pg9uv.png" alt="Before Lucy signs: STOP, in plain words" width="800" height="365"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Demo
&lt;/h2&gt;

&lt;p&gt;🎬 &lt;strong&gt;Video (3:23):&lt;/strong&gt; the problem, how it works and a real run on devnet, narrated by Olive the owl.&lt;/p&gt;

&lt;p&gt;  &lt;iframe src="https://www.youtube.com/embed/rZ3oyl6PMTw" width="710" height="399"&gt;
  &lt;/iframe&gt;
&lt;/p&gt;

&lt;p&gt;🧪 &lt;strong&gt;Try it yourself:&lt;/strong&gt; &lt;a href="https://jenniferevangelista-de.github.io/night-guard-wallet/demo/" rel="noopener noreferrer"&gt;interactive demo&lt;/a&gt;. Click the seven situations, hear the verdict, open Ana's page and paste a message you got to see which scam tricks it finds. Every answer was recorded from a real devnet run, and every button that signs opens the real transaction on Solana Explorer.&lt;/p&gt;


&lt;div class="ltag-github-readme-tag"&gt;
  &lt;div class="readme-overview"&gt;
    &lt;h2&gt;
      &lt;img src="https://assets.dev.to/assets/github-logo-5a155e1f9a670af7944dd5e12375bc76ed542ea80224905ecaf878b9157cdefc.svg" alt="GitHub logo"&gt;
      &lt;a href="https://github.com/jenniferevangelista-de" rel="noopener noreferrer"&gt;
        jenniferevangelista-de
      &lt;/a&gt; / &lt;a href="https://github.com/jenniferevangelista-de/night-guard-wallet" rel="noopener noreferrer"&gt;
        night-guard-wallet
      &lt;/a&gt;
    &lt;/h2&gt;
    &lt;h3&gt;
      A Solana wallet guard for older people: simulate, explain in plain words, flag scams with local AI, and lock risky payments behind a family guardian on-chain
    &lt;/h3&gt;
  &lt;/div&gt;
  &lt;div class="ltag-github-body"&gt;
    
&lt;div id="readme" class="md"&gt;&lt;div class="markdown-heading"&gt;
&lt;h1 class="heading-element"&gt;🌙 Night Guard Wallet&lt;/h1&gt;
&lt;/div&gt;
&lt;p&gt;&lt;strong&gt;A Solana wallet guard for the people scammers target the most, our parents and grandparents, and for the rest of us too.&lt;/strong&gt; Before Lucy (74) signs anything, it &lt;strong&gt;simulates the transaction on Solana&lt;/strong&gt;, &lt;strong&gt;checks where every part comes from&lt;/strong&gt;, lets &lt;strong&gt;Gemma&lt;/strong&gt; read the message that came with it, explains the verdict &lt;strong&gt;in big plain words&lt;/strong&gt;, and keeps her savings in an &lt;strong&gt;on-chain vault&lt;/strong&gt; whose rules nobody can switch off. When her daughter refuses a scam, the wallet goes on a &lt;strong&gt;community scam list on Solana&lt;/strong&gt;, so &lt;strong&gt;one family's "no" protects every family&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;Built for the &lt;a href="https://events.mlh.com/events/15200-solana-hackathon" rel="nofollow noopener noreferrer"&gt;MLH Solana Hackathon&lt;/a&gt; · AI Track · Top 3 · DeFi · Beginner&lt;/p&gt;
&lt;p&gt;&lt;a href="https://github.com/jenniferevangelista-de/night-guard-wallet/actions/workflows/ci.yml" rel="noopener noreferrer"&gt;&lt;img src="https://github.com/jenniferevangelista-de/night-guard-wallet/actions/workflows/ci.yml/badge.svg" alt="ci"&gt;&lt;/a&gt;
&lt;a href="https://explorer.solana.com/address/8BtGEAMv12hh24JQYmWUJZ8Y547y72n4LDfppv3CTDJ5?cluster=devnet" rel="nofollow noopener noreferrer"&gt;&lt;img src="https://camo.githubusercontent.com/ea381755d808b65f9ce3331f483f5acd054456a94926529f7a92aa02d62e8d12/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f536f6c616e612532306465766e65742d677561726469616e5f5f7661756c742d3134463139353f6c6f676f3d736f6c616e61266c6f676f436f6c6f723d7768697465" alt="devnet program"&gt;&lt;/a&gt;
&lt;a href="https://github.com/jenniferevangelista-de/night-guard-wallet/docs/EVAL.md" rel="noopener noreferrer"&gt;&lt;img src="https://camo.githubusercontent.com/d3ab220d9afd8e3c524ebc17d4851abe37417444041c6eb0fd16ead1de1e91e2/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f41492d47656d6d61253230342532304532422532307669612532304f6c6c616d612d363934316336" alt="Gemma 4"&gt;&lt;/a&gt;
&lt;a href="https://github.com/jenniferevangelista-de/night-guard-wallet/LICENSE" rel="noopener noreferrer"&gt;&lt;img src="https://camo.githubusercontent.com/00f93b9dd76f23c21c0daee04734659b120bdfd64e642915b8dc212a86ffccf3/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d4d49542d306631623364" alt="license"&gt;&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;a href="https://youtu.be/rZ3oyl6PMTw" rel="nofollow noopener noreferrer"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fraw.githubusercontent.com%2Fjenniferevangelista-de%2Fnight-guard-wallet%2FHEAD%2Fdocs%2Fimg%2Fvideo_thumb.jpg" alt="Watch the 3-minute Night Guard Wallet video on YouTube"&gt;&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Video (3:23):&lt;/strong&gt; &lt;a href="https://youtu.be/rZ3oyl6PMTw" rel="nofollow noopener noreferrer"&gt;watch on YouTube&lt;/a&gt;: the problem, how it works, and a real run on Solana devnet, narrated by Olive the owl · &lt;strong&gt;Interactive demo:&lt;/strong&gt; &lt;a href="https://jenniferevangelista-de.github.io/night-guard-wallet/demo/" rel="nofollow noopener noreferrer"&gt;try it&lt;/a&gt; · &lt;strong&gt;Showcase page:&lt;/strong&gt; &lt;a href="https://jenniferevangelista-de.github.io/night-guard-wallet/" rel="nofollow noopener noreferrer"&gt;jenniferevangelista-de.github.io/night-guard-wallet&lt;/a&gt; ·…&lt;/p&gt;&lt;/div&gt;
  &lt;/div&gt;
  &lt;div class="gh-btn-container"&gt;&lt;a class="gh-btn" href="https://github.com/jenniferevangelista-de/night-guard-wallet" rel="noopener noreferrer"&gt;View on GitHub&lt;/a&gt;&lt;/div&gt;
&lt;/div&gt;


&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fjefkn9oeqrkhff6td3rm.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fjefkn9oeqrkhff6td3rm.png" alt="Ana, the guardian: live from Solana, say no and report" width="800" height="365"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  How It Works
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fpb1vw6ob7xxb2i2u6wqe.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fpb1vw6ob7xxb2i2u6wqe.png" alt="Night Guard Wallet architecture" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Five steps, every time Lucy taps "check":&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Simulate on Solana.&lt;/strong&gt; &lt;code&gt;simulateTransaction&lt;/code&gt; runs it on devnet and I compare every writable account before and after: SOL, tokens, delegates and owners. This is where an "innocent" airdrop claim shows its true face: an unlimited &lt;code&gt;approve&lt;/code&gt; to a stranger.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Flag the risk&lt;/strong&gt; with rules on the simulated result: a wallet she never paid (and how new it is), an address that copies the first characters of a trusted contact, most of her money leaving, a fake USDC that only calls itself USDC (Token-2022 metadata vs. the verified mint).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Check the sources.&lt;/strong&gt; Where does each part come from? The sender, the link, every wallet, every program (how many days old, can its owner still change it, is its source verified) and every token.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Read the message with AI.&lt;/strong&gt; Gemma 4 E2B, running free on my own machine through Ollama, lists the scam tricks: new number, rush, secrecy, fake support, fee to unlock.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Lock and ask the family.&lt;/strong&gt; On STOP, nothing is signed. One tap opens an on-chain request for Ana.&lt;/li&gt;
&lt;/ol&gt;

&lt;h3&gt;
  
  
  The model can only add warnings
&lt;/h3&gt;

&lt;p&gt;This was the first design decision, and the one I am proudest of: &lt;strong&gt;the final call always belongs to people, never to the AI.&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;The wallet checks.&lt;/strong&gt; Fixed, testable rules look at where the money really goes, who receives it (a brand-new wallet, or an address made to look like a trusted contact's), her daily limit, and scam reports from other families. A clear danger means STOP; a doubt means CAUTION. Either way, a payment from her vault only &lt;em&gt;pauses&lt;/em&gt;. Nothing is blocked forever.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The AI explains.&lt;/strong&gt; Gemma reads the message and highlights the scam tricks in plain words, like "urgent" or "don't tell dad", so Lucy understands why the wallet paused.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The family decides.&lt;/strong&gt; With one tap, Lucy asks Ana to look. Ana approves or refuses with her own signature. If she approves, the vault sends the money (a big amount still waits a short cooling-off). If it was a false alarm, it costs Ana one tap, not Lucy's savings. For the riskiest signatures, like handing control of her tokens to a stranger, the app does not sign at all and alerts Ana instead.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The AI also has two guardrails, so a scammer cannot use it against her. It can only point to words that really exist in the message, so it cannot invent reasons. And it has no power to remove a warning, so a message saying "this is safe, approve it" changes nothing.&lt;/p&gt;

&lt;p&gt;The explanation has its own checks, because a friendly sentence can still be wrong:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="c1"&gt;// Any amount the model writes must be the real one (no invented numbers).&lt;/span&gt;
&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;okAmount&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;said&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;every&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;x&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="n"&gt;real&lt;/span&gt; &lt;span class="o"&gt;!==&lt;/span&gt; &lt;span class="kc"&gt;null&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="nb"&gt;Math&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;abs&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;x&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="n"&gt;real&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="mf"&gt;1e-9&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;saysSafe&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="o"&gt;/&lt;/span&gt;\&lt;span class="nf"&gt;b&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;seguro&lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;&lt;span class="n"&gt;safe&lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;&lt;span class="n"&gt;tranquil&lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;&lt;span class="n"&gt;pode&lt;/span&gt; &lt;span class="n"&gt;seguir&lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;&lt;span class="n"&gt;go&lt;/span&gt; &lt;span class="n"&gt;ahead&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;\&lt;span class="n"&gt;b&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;i&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;test&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="c1"&gt;// money leaves: never "you received"&lt;/span&gt;
&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;wrongWay&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="o"&gt;!!&lt;/span&gt;&lt;span class="n"&gt;facts&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;moneyOut&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="o"&gt;/&lt;/span&gt;\&lt;span class="nf"&gt;b&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;recebeu&lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;&lt;span class="n"&gt;receber&lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;&lt;span class="n"&gt;recebe&lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;&lt;span class="n"&gt;recebendo&lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;&lt;span class="n"&gt;received&lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;&lt;span class="n"&gt;receive&lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;&lt;span class="n"&gt;receiving&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;\&lt;span class="n"&gt;b&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;i&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;test&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="n"&gt;text&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="n"&gt;okAmount&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="n"&gt;wrongWay&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="n"&gt;words&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="mi"&gt;60&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="n"&gt;isStop&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="n"&gt;saysSafe&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;facts&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;fallback&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;ms&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;by&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"template"&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That &lt;code&gt;wrongWay&lt;/code&gt; line has a story. More on it below.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fsm14jhztup2b14k7jky0.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fsm14jhztup2b14k7jky0.png" alt="Measured speed: verdict on screen in 0.2 s, simulation 15 ms" width="800" height="464"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Solana enforces it, not the app
&lt;/h3&gt;

&lt;p&gt;The rules live in Lucy's vault, a PDA only the program moves:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Rule&lt;/th&gt;
&lt;th&gt;On-chain&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Money only goes out at once to trusted contacts, under the daily limit&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;send&lt;/code&gt; refuses with &lt;code&gt;NotTrusted&lt;/code&gt; or &lt;code&gt;OverDailyLimit&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Adding a trusted contact needs &lt;strong&gt;two&lt;/strong&gt; signatures: Lucy &lt;strong&gt;and&lt;/strong&gt; Ana&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;add_trusted&lt;/code&gt; with &lt;code&gt;has_one = guardian&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Anything else becomes a request the guardian decides&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;request&lt;/code&gt; → &lt;code&gt;guardian_decide&lt;/code&gt; → &lt;code&gt;execute&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;A big payment waits a cooling-off, even after approval&lt;/td&gt;
&lt;td&gt;&lt;code&gt;StillCoolingOff&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;To prove it, my demo forces scams &lt;strong&gt;past the app, on purpose&lt;/strong&gt;, skipping preflight. The program refused all seven: &lt;code&gt;NotTrusted&lt;/code&gt; twice, &lt;code&gt;ConstraintHasOne&lt;/code&gt;, &lt;code&gt;Rejected&lt;/code&gt;, &lt;code&gt;StillCoolingOff&lt;/code&gt;, &lt;code&gt;NoProof&lt;/code&gt; and a duplicate report.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Filhbzuzbi6xploqmqe5z.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Filhbzuzbi6xploqmqe5z.png" alt="Seven refusals recorded on devnet" width="800" height="448"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  One family's "no" protects every family
&lt;/h3&gt;

&lt;p&gt;When Ana refuses a scam, one more tap puts the wallet on a &lt;strong&gt;community scam list on Solana&lt;/strong&gt;. Any wallet can read it.&lt;/p&gt;

&lt;p&gt;The first version was simple: a guardian refuses, the wallet goes on the list. Then I asked myself a question that changed it completely: &lt;em&gt;what if someone reports a wallet by mistake, or out of malice?&lt;/em&gt; A list anyone can write to is a weapon.&lt;/p&gt;

&lt;p&gt;So a report now needs &lt;strong&gt;proof&lt;/strong&gt;: a request from &lt;em&gt;your&lt;/em&gt; vault, to &lt;em&gt;that&lt;/em&gt; wallet, that &lt;em&gt;you&lt;/em&gt; refused. No refused request, no report.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight rust"&gt;&lt;code&gt;&lt;span class="k"&gt;pub&lt;/span&gt; &lt;span class="k"&gt;struct&lt;/span&gt; &lt;span class="nc"&gt;ReportScam&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="kt"&gt;'info&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="cp"&gt;#[account(mut)]&lt;/span&gt;
    &lt;span class="k"&gt;pub&lt;/span&gt; &lt;span class="n"&gt;guardian&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;Signer&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="kt"&gt;'info&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="cp"&gt;#[account(has_one = guardian)]&lt;/span&gt;
    &lt;span class="k"&gt;pub&lt;/span&gt; &lt;span class="n"&gt;vault&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;Account&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="kt"&gt;'info&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nc"&gt;Vault&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="c1"&gt;/// The proof: a request of THIS vault, to this wallet, refused by the guardian.&lt;/span&gt;
    &lt;span class="cp"&gt;#[account(has_one = vault @ VaultError::NoProof)]&lt;/span&gt;
    &lt;span class="k"&gt;pub&lt;/span&gt; &lt;span class="n"&gt;proof&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;Account&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="kt"&gt;'info&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nc"&gt;TransferRequest&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="cp"&gt;#[account(init_if_needed, payer = guardian, space = 8 + ScamReport::INIT_SPACE,
              seeds = [b"scam", address.as_ref()]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;bump&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
    &lt;span class="k"&gt;pub&lt;/span&gt; &lt;span class="n"&gt;report&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;Account&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="kt"&gt;'info&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nc"&gt;ScamReport&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="c1"&gt;// one mark per family and wallet: the same family can't vote twice&lt;/span&gt;
    &lt;span class="cp"&gt;#[account(init, payer = guardian, space = 8 + ReportMark::INIT_SPACE,
              seeds = [b"reported", vault.key().as_ref(), address.as_ref()]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;bump&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
    &lt;span class="k"&gt;pub&lt;/span&gt; &lt;span class="n"&gt;mark&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;Account&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="kt"&gt;'info&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nc"&gt;ReportMark&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="k"&gt;pub&lt;/span&gt; &lt;span class="n"&gt;system_program&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;Program&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="kt"&gt;'info&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nc"&gt;System&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;And inside the instruction, the proof has to match:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight rust"&gt;&lt;code&gt;&lt;span class="nf"&gt;require!&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;address&lt;/span&gt; &lt;span class="o"&gt;!=&lt;/span&gt; &lt;span class="n"&gt;ctx&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;accounts&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;vault&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;owner&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="n"&gt;ctx&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;accounts&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;vault&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;trusted&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;contains&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="n"&gt;address&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nc"&gt;VaultError&lt;/span&gt;&lt;span class="o"&gt;::&lt;/span&gt;&lt;span class="nc"&gt;BadSettings&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="nf"&gt;require_keys_eq!&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;proof&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;address&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nc"&gt;VaultError&lt;/span&gt;&lt;span class="o"&gt;::&lt;/span&gt;&lt;span class="nc"&gt;NoProof&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="nf"&gt;require!&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;proof&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;status&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="nc"&gt;Status&lt;/span&gt;&lt;span class="o"&gt;::&lt;/span&gt;&lt;span class="nc"&gt;Rejected&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="kt"&gt;u8&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nc"&gt;VaultError&lt;/span&gt;&lt;span class="o"&gt;::&lt;/span&gt;&lt;span class="nc"&gt;NoProof&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The rest of the fairness rules: one vote per family, trusted contacts can never be reported, a report can be withdrawn, and the weight grows slowly. &lt;strong&gt;1 family is a warning, 2 is strong, 3 families block.&lt;/strong&gt; One report is a reason to call your family, not a verdict.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fbn0c940dc3ns95onrzve.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fbn0c940dc3ns95onrzve.png" alt="The community list: one report is a warning, not proof" width="800" height="365"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Measured, not promised
&lt;/h3&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th&gt;A wallet without a guard&lt;/th&gt;
&lt;th&gt;With Night Guard Wallet&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;What Lucy sees before signing&lt;/td&gt;
&lt;td&gt;program, accounts, an amount&lt;/td&gt;
&lt;td&gt;⛔ STOP and one sentence in her words, in 0.2 s&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;The 5 scam scenarios&lt;/td&gt;
&lt;td&gt;each one signed with one tap&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;5/5 stopped&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;The message that came with it&lt;/td&gt;
&lt;td&gt;never read&lt;/td&gt;
&lt;td&gt;
&lt;strong&gt;32/32&lt;/strong&gt; scam messages recognised, &lt;strong&gt;0&lt;/strong&gt; false alarms on 28 family messages&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;After she taps "approve"&lt;/td&gt;
&lt;td&gt;final&lt;/td&gt;
&lt;td&gt;the vault refuses: &lt;strong&gt;7&lt;/strong&gt; refusals on devnet&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Cost to run&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;td&gt;free: local Gemma, no API&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Fixed rules alone caught 22/32 scam messages with 3 false alarms. Gemma reading the story caught the 10 the rules missed. That gap is the reason the AI is there.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fes4b2j0ua7hl8fjrpqe0.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fes4b2j0ua7hl8fjrpqe0.png" alt="60-message evaluation: rules vs. Gemma" width="800" height="400"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Partner Technologies
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Solana devnet + Anchor 0.31.1:&lt;/strong&gt; the vault, the guardian flow, events and the community list. 12 instructions.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;@solana/web3.js&lt;/code&gt;, &lt;code&gt;@solana/spl-token&lt;/code&gt;, Token-2022 metadata, Solana Pay and Blinks:&lt;/strong&gt; decoding, simulation diffs, fake-token detection, QR and link checks.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Gemma 4 E2B via Ollama:&lt;/strong&gt; structured JSON, every quote checked against the message, on a CPU, no API, no cost per check. The message Lucy received never leaves the machine.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;GitHub Actions + GitHub Pages:&lt;/strong&gt; build, deploy, the measured demo run, video and the interactive replay.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Hackathon Experience
&lt;/h2&gt;

&lt;p&gt;I came in with a project I love, Night Guard, a local-AI guard for kids online, and one question: what would that idea look like on a blockchain, where nobody can call the bank and undo a payment?&lt;/p&gt;

&lt;p&gt;Some real moments from the weekend:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;My cloud machine was too old for Solana.&lt;/strong&gt; The first build failed because the environment's system libraries were older than the Solana tools wanted. I switched to the official Anchor Docker image, and then my free cloud hours ran out mid-build. So the whole thing moved to GitHub Actions: build, deploy, the demo run, the video. My computer only opened the browser. Honestly, that ended up being the best decision of the weekend, because every number in the README comes from a run anyone can repeat.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Rust versions fought each other.&lt;/strong&gt; A dependency had moved to a newer Rust edition than the Solana toolchain spoke. Pinning two crates and one resolver flag fixed it. Finding which two took longer than writing them.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The public RPC said "slow down".&lt;/strong&gt; My demo run hit rate limits and crashed. I added patient retries and a small state cache, and the 42 checks finally finished in one go.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Gemma told Lucy she had &lt;em&gt;received&lt;/em&gt; money.&lt;/strong&gt; In Portuguese, in a scam where she was about to &lt;em&gt;send&lt;/em&gt; 1.8 SOL. The amount was right, the tone was right, and the direction was backwards. That is the &lt;code&gt;wrongWay&lt;/code&gt; line in the code above. It taught me that checking numbers is not enough: you also have to check which way the money goes.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;My first video sounded Spanish.&lt;/strong&gt; I had picked a voice that turned "Hi, I'm Jennifer" into something else entirely. Rerecorded with Ava, in English first, Portuguese as a plus.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;MLH strips images from the README on the project page.&lt;/strong&gt; My README had charts and screenshots, and the page showed a wall of text. So I built a one-page showcase with the poster and the big numbers, and made the interactive demo a real thing judges can click instead of a picture.&lt;/p&gt;

&lt;p&gt;What I learned: on Solana, the most important line of code is the one that says no. I spent more time deciding what the program should &lt;em&gt;refuse&lt;/em&gt; than what it should do, and that is exactly where the safety lives.&lt;/p&gt;

&lt;h2&gt;
  
  
  What It Can't Do Yet
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Devnet only.&lt;/strong&gt; The demo signs with demo keypairs on a server. A real product would run the guard inside the wallet app, with keys on the user's device.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The message evaluation is small and fictional:&lt;/strong&gt; 60 messages I wrote. It shows the approach, not field accuracy.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Gemma on a CPU takes about 5.5 s per message.&lt;/strong&gt; The verdict doesn't wait for it, only the explanation does.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The look-alike and phishing checks use built-in lists.&lt;/strong&gt; Production needs a maintained registry of verified tokens and domains.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The community list needs more families to matter.&lt;/strong&gt; Three is a block in the demo. In the real world, the threshold and the appeal process deserve a proper design with people who have been scammed.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Thank You
&lt;/h2&gt;

&lt;p&gt;To &lt;strong&gt;MLH&lt;/strong&gt; and the &lt;strong&gt;Solana Foundation&lt;/strong&gt; for a hackathon that welcomed someone writing their first Anchor program. To the people who maintain Anchor, the Solana docs and the devnet tools. To &lt;strong&gt;Google&lt;/strong&gt; for releasing Gemma with open weights, and to &lt;strong&gt;Ollama&lt;/strong&gt; for making it run on a normal laptop.&lt;/p&gt;

&lt;p&gt;And to every family that ever got the "hi mom, new number" message. This one is for you.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Every person and message in this project is fictional. Music in the video: "Funkorama" by Kevin MacLeod (incompetech.com), CC BY 4.0.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>mlhacks</category>
      <category>solana</category>
      <category>ai</category>
      <category>web3</category>
    </item>
    <item>
      <title>Switch from zero: ship a human-in-the-loop data quality agent to your team chat</title>
      <dc:creator>Jennifer Evangelista</dc:creator>
      <pubDate>Tue, 03 Nov 2026 22:40:24 +0000</pubDate>
      <link>https://dev.to/jenniferevangelistade/switch-from-zero-ship-a-human-in-the-loop-data-quality-agent-to-your-team-chat-4cm9</link>
      <guid>https://dev.to/jenniferevangelistade/switch-from-zero-ship-a-human-in-the-loop-data-quality-agent-to-your-team-chat-4cm9</guid>
      <description>&lt;p&gt;I spent the Flip the Switch hackathon building &lt;strong&gt;Shield&lt;/strong&gt;, an agent that checks a daily table, explains what it finds in a team channel and only fixes data after a person approves. This post is the guide I wish I had on day one: how to self-host Switch with no desktop app, how a custom agent joins a room, and the three things that will block you the first time.&lt;/p&gt;

&lt;p&gt;Everything here was run on Switch 0.31.0 in a GitHub Codespace. All the data is fictional.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Shield at a glance&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;6&lt;/strong&gt; incident types caught: stale business date on a green pipeline, row count drop, schema drift, null spike, duplicate keys and a late upstream file&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;5&lt;/strong&gt; Great Expectations expectations plus the business-date gate behind those checks&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;5&lt;/strong&gt; fix actions, and &lt;strong&gt;0&lt;/strong&gt; of them run without a person approving in the chat&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;42&lt;/strong&gt; unit tests, plus an end-to-end test that plays every scenario through a real Switch server on a fresh GitHub runner&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;US$ 0&lt;/strong&gt; to run: self-hosted Switch, a local open model, free CI&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;  &lt;iframe src="https://www.youtube.com/embed/kAeq5Iub_14" width="710" height="399"&gt;
  &lt;/iframe&gt;
&lt;/p&gt;

&lt;p&gt;Raw screen recording of the chat, no edits: &lt;a href="https://github.com/jenniferevangelista-de/shield-for-data/releases/download/v0.1.0/shield_demo.mp4" rel="noopener noreferrer"&gt;shield_demo.mp4&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fj743qz13fhy4hswnibxl.gif" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fj743qz13fhy4hswnibxl.gif" alt="Approving a fix in the channel: Shield holds the report, re-runs the partition, checks again and releases the hold" width="600" height="338"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  What Switch is, in one paragraph
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://github.com/sandbox-quantum/switch" rel="noopener noreferrer"&gt;Switch&lt;/a&gt; turns a chat channel (Slack, Teams, Discord, Telegram or Mattermost) into a &lt;strong&gt;room&lt;/strong&gt; where people and AI agents work together. Switch ships no agents and no models. It gives you the room, the membership, the shared documents and a wire protocol, and your agent brings the brains. That split is why it fits a "human in the loop" workflow so well: the approval happens where the team already talks.&lt;/p&gt;

&lt;p&gt;There are two ways in:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Switch Console&lt;/strong&gt;, the desktop app, runs a server for you and manages Claude Code, Codex or OpenCode agents. This is the 10-minute quickstart.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Self-hosting&lt;/strong&gt; with the published Docker Compose file, plus any agent that speaks the protocol. This is what I did, because I wanted a custom Python agent and a headless setup I could run in CI.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  The architecture in one picture
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fb449vv0viep80gmf97nm.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fb449vv0viep80gmf97nm.png" alt="Shield architecture: team chat, Switch server (bridge, room, agent protocol), Shield agent (approval gate, six checks, fix actions, owners, local Gemma) and the DuckDB warehouse" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The chat room is the only interface. Switch carries the messages, the members and the audit document. Shield reads the warehouse freely, but writes to it only after a person approves.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 1: start a Switch server with its own Mattermost
&lt;/h2&gt;

&lt;p&gt;You need Docker with Compose v2. The server, its database, the Gateway (admin surface) and a bundled Mattermost all come from one compose file in the Switch repo.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;git clone https://github.com/sandbox-quantum/switch.git &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="nb"&gt;cd&lt;/span&gt; switch
cp .env.example .env
bash deploy/local/generate-env-secrets.sh .env &lt;span class="kd"&gt;local&lt;/span&gt;   &lt;span class="c"&gt;# fills every password and token&lt;/span&gt;
sed -i &lt;span class="s1"&gt;'s/^SWITCH_VERSION=.*/SWITCH_VERSION=0.31.0/'&lt;/span&gt; .env
docker compose -f deploy/local/standalone-docker-compose.yml --project-directory . \
  --profile collab --profile gateway up -d
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Three things worth knowing:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Pin &lt;code&gt;SWITCH_VERSION&lt;/code&gt;.&lt;/strong&gt; The compose file refuses to start without it, on purpose. Take the version from the container registry, not from the GitHub releases page, which tracks the desktop app.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The &lt;code&gt;collab&lt;/code&gt; profile is Mattermost; &lt;code&gt;gateway&lt;/code&gt; is the admin surface.&lt;/strong&gt; With both on, a &lt;code&gt;setup&lt;/code&gt; container creates a Mattermost admin, a normal user called &lt;code&gt;user&lt;/code&gt;, a team called &lt;code&gt;switch&lt;/code&gt;, and links Mattermost as the default bridge. When &lt;code&gt;docker wait&lt;/code&gt; on that container returns 0, you are ready.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Ports&lt;/strong&gt;: 8000 is the API and the agent bridge, 3000 the Gateway, 8065 Mattermost. Everything binds to 127.0.0.1 by default. In my repo the script moves them to 18000/13000/18065 so they never collide with another Switch.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fakwwjr7nitcql3123o3l.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fakwwjr7nitcql3123o3l.png" alt="Mattermost after setup: the data-quality channel with Shield as a member" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 2: register a custom agent
&lt;/h2&gt;

&lt;p&gt;An agent is registered once, out of band, with the server's &lt;code&gt;AGENT_REGISTRATION_TOKEN&lt;/code&gt; from &lt;code&gt;.env&lt;/code&gt;. For a process that is always running (rather than a session Console starts on demand), use the lower-level &lt;code&gt;POST /agents&lt;/code&gt; with an &lt;code&gt;always_on&lt;/code&gt; connection model:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;

&lt;span class="n"&gt;r&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;post&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"{SWITCH}/agents"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;headers&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"Authorization"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"Bearer {REGISTRATION_TOKEN}"&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="s2"&gt;"name"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"shield"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;                      &lt;span class="c1"&gt;# what people type after @&lt;/span&gt;
    &lt;span class="s2"&gt;"description"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"Data quality teammate"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s2"&gt;"connector_type"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"custom-python"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s2"&gt;"integration_profile"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="s2"&gt;"connection_model"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"always_on"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="s2"&gt;"message_exchange"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;True&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="s2"&gt;"pre_invocation_mediation"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"post_invocation_mediation"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"event_reporting"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="s2"&gt;"command_capabilities"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"reset"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"unsupported"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"compact"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"unsupported"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"interrupt"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"unsupported"&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s2"&gt;"tools"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"models"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"metadata"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"overwrite"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;True&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;agent_id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;api_key&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"id"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"api_key"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;   &lt;span class="c1"&gt;# the key is shown once: store it&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The convenience route &lt;code&gt;POST /agents/register-known&lt;/code&gt; only knows Claude Code, Codex, OpenCode and Antigravity, so a custom agent goes through &lt;code&gt;POST /agents&lt;/code&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 3: open the event stream and keep it alive
&lt;/h2&gt;

&lt;p&gt;Down is one Server-Sent Events stream, up is plain HTTP. The part that trips people up is that &lt;strong&gt;the connection is identified by an id you generate&lt;/strong&gt;, not by the socket:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;conn&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kt"&gt;str&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;uuid&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;uuid4&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;   &lt;span class="c1"&gt;# generate once, reuse on every reconnect&lt;/span&gt;
&lt;span class="n"&gt;stream&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;session&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"{SWITCH}/agents/{agent_id}/events"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                     &lt;span class="n"&gt;params&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"connection_id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;conn&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"scope"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"single"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"filter"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"addressed"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                             &lt;span class="s2"&gt;"rooms"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;room_id&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                     &lt;span class="n"&gt;headers&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"Accept"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"text/event-stream"&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;stream&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="kc"&gt;True&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;ul&gt;
&lt;li&gt;The first frame is always &lt;code&gt;connection_state&lt;/code&gt;. Read the &lt;code&gt;rooms&lt;/code&gt; and &lt;code&gt;cursor&lt;/code&gt; from it rather than assuming.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Beat every 2 seconds&lt;/strong&gt; with &lt;code&gt;POST /agents/{id}/connection/beat&lt;/code&gt; and &lt;code&gt;{connection_id, cursor}&lt;/code&gt;. The TTL is 6 seconds. A 404 or 409 on the beat means "reopen the stream", not "back off".&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;filter=addressed&lt;/code&gt; means you only receive messages that mention your agent. That is what you want for a chat agent.&lt;/li&gt;
&lt;li&gt;Declare the room on the URL (&lt;code&gt;rooms=&lt;/code&gt;). A room subscribed after the stream opens can miss its buffered events.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Shield's whole client is about 150 lines: &lt;a href="https://github.com/jenniferevangelista-de/shield-for-data/blob/main/shield/switch_client.py" rel="noopener noreferrer"&gt;&lt;code&gt;shield/switch_client.py&lt;/code&gt;&lt;/a&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 4: create the room, and the channel appears
&lt;/h2&gt;

&lt;p&gt;Operations are &lt;code&gt;POST /agents/{id}/ops/{name}&lt;/code&gt; with the arguments as the body, plus an &lt;code&gt;X-Switch-Connection-Id&lt;/code&gt; header when the operation needs a connection. &lt;code&gt;create_room&lt;/code&gt; on the default bridge creates the Mattermost channel for you:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;room&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;op&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"create_room"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="s2"&gt;"name"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"data-quality"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s2"&gt;"description"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"Data quality incidents for sales.orders"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s2"&gt;"agent_names"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"shield"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;          &lt;span class="c1"&gt;# include yourself, or you are not a member&lt;/span&gt;
    &lt;span class="s2"&gt;"user_names"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"user"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;             &lt;span class="c1"&gt;# Mattermost usernames&lt;/span&gt;
    &lt;span class="s2"&gt;"channel_type"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"channel_public"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s2"&gt;"instructions"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"Shield never changes data without an approval in this room."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;connection_id&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;conn&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;To answer in the room, &lt;code&gt;POST /agents/{id}/message&lt;/code&gt; with &lt;code&gt;{room_id, content}&lt;/code&gt; needs no connection at all. Markdown renders in Mattermost.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 5: the three things that will block you
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;1. "You're not permitted to direct messages to me in this room."&lt;/strong&gt; A new agent answers only its owner, and an agent registered with the registration token is owned by a bootstrap account, not by you. Nobody can talk to it. The Gateway admin (credentials in &lt;code&gt;.env&lt;/code&gt;) can clear the policy:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;s&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Session&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;s&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;post&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"{SWITCH}/gateway/auth/login"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"email"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="no"&gt;ADMIN_EMAIL&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"password"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="no"&gt;ADMIN_PASSWORD&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;s&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;put&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"{SWITCH}/gateway/agents/{agent_id}/addressing-policy"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"policy"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;With no policy, room membership decides who can reach the agent. In production, set a policy that names the people allowed to approve.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;2. "I'm currently offline: my connector isn't reporting in."&lt;/strong&gt; Switch answers on your agent's behalf until it sees heartbeats. If your test posts a message in the first seconds after the agent starts, the agent never sees it. My end-to-end test simply asks &lt;code&gt;@shield help&lt;/code&gt; until the real answer comes back.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;3. Nothing happens and there is no error.&lt;/strong&gt; Messages that do not &lt;code&gt;@mention&lt;/code&gt; the agent are delivered to nobody when you use &lt;code&gt;filter=addressed&lt;/code&gt;. Check the Switch logs (&lt;code&gt;docker logs &amp;lt;project&amp;gt;-switch-1&lt;/code&gt;): lines from &lt;code&gt;switch_core.delivery.addressing&lt;/code&gt; tell you exactly why a message was or was not delivered.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 6: make the human the gate
&lt;/h2&gt;

&lt;p&gt;This is the part that makes it a workflow rather than a bot. Shield follows one rule: &lt;strong&gt;a scan only records findings; only an approval changes data.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fzwr18jhzye92reg54j8v.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fzwr18jhzye92reg54j8v.png" alt="Shield's finding in the channel: what broke, why it matters, who is affected, and the proposed fix" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Each finding gets an id and three possible answers, all plain messages in the room:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;@shield approve F-1&lt;/code&gt; runs the proposed actions, checks again, and posts the result.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;@shield edit F-1 hold_dashboard+notify_owners&lt;/code&gt; changes the plan before approving.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;@shield reject F-1 known test load&lt;/code&gt; closes it with a reason and touches nothing.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Switch passes the sender's name with every message, so the audit trail records who decided. Shield writes that trail to a &lt;strong&gt;room document&lt;/strong&gt; (&lt;code&gt;create_room_document&lt;/code&gt;, then &lt;code&gt;update_room_document&lt;/code&gt;), which every member of the room can read later.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fpv7gzd6r2lrw0m2cr05m.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fpv7gzd6r2lrw0m2cr05m.png" alt="The audit trail Shield keeps in the room: who did what, and when" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The main check is the business-date gate from my article &lt;a href="https://dev.to/jenniferevangelistade/your-pipeline-ran-green-and-shipped-last-weeks-data-anyway-1il3"&gt;Your pipeline ran green and shipped last week's data anyway&lt;/a&gt;: the greatest attempt per job and date wins, and a date is ready only when every required job's newest attempt succeeded for that date. The other five checks (schema, row count, nulls, duplicates, late files) are a Great Expectations suite, and owners come from a catalog adapter shaped like OpenMetadata's API.&lt;/p&gt;

&lt;h3&gt;
  
  
  Where the AI sits, and where it does not
&lt;/h3&gt;

&lt;p&gt;Shield's language model is a small open model, Gemma 3 4B, served on the same machine by &lt;a href="https://ollama.com" rel="noopener noreferrer"&gt;Ollama&lt;/a&gt; through its OpenAI-compatible API, so it costs nothing and no data leaves the host. It has two jobs:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;It writes the "why it matters" line from the finding. If its answer contains a number or runs long, Shield throws it away and uses a fixed sentence, because the numbers have to come from the checks.&lt;/li&gt;
&lt;li&gt;It understands plain language. &lt;code&gt;@shield are today's numbers ok to publish?&lt;/code&gt; runs a scan. &lt;code&gt;@shield sure, go ahead with F-1&lt;/code&gt; does &lt;strong&gt;not&lt;/strong&gt; approve anything: Shield answers "I read that as &lt;code&gt;@shield approve F-1&lt;/code&gt;. I only change data on an exact command." A misread can cost a message, never a table.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;That boundary is the design choice I would repeat in any agent that touches production data: let the model read and explain, and keep every write behind an exact, human command.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 7: test it like a person would
&lt;/h2&gt;

&lt;p&gt;Mattermost has a plain REST API, so an end-to-end test can log in as the demo user, post &lt;code&gt;@shield break stale_business_date&lt;/code&gt;, &lt;code&gt;@shield scan&lt;/code&gt;, &lt;code&gt;@shield approve F-1&lt;/code&gt;, and wait for each reply. In my repo, that runs on a fresh GitHub Actions runner on every push: start Switch, register Shield, play every scenario. If the test passes, the README is telling the truth.&lt;/p&gt;

&lt;h2&gt;
  
  
  The numbers
&lt;/h2&gt;

&lt;p&gt;Six problems injected into a 7,893-row warehouse, one at a time, then &lt;code&gt;@shield scan&lt;/code&gt; and &lt;code&gt;@shield approve F-1&lt;/code&gt; through the real Switch server. The script is &lt;a href="https://github.com/jenniferevangelista-de/shield-for-data/blob/main/demo/film/measure.py" rel="noopener noreferrer"&gt;measure.py&lt;/a&gt;.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Problems caught&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;6 of 6&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Fixed in place after approval&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;
&lt;strong&gt;3&lt;/strong&gt; (re-run, quarantine, dedupe)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Report held safely, owners told&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;
&lt;strong&gt;3&lt;/strong&gt; (the fix has to come from upstream)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Median scan time&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;0.12 s&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Data changed without approval&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;0&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Problem injected&lt;/th&gt;
&lt;th&gt;What Shield reported&lt;/th&gt;
&lt;th&gt;Approved fix&lt;/th&gt;
&lt;th&gt;Before → after&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Stale business date (all jobs green)&lt;/td&gt;
&lt;td&gt;Runs were for 2026-10-29, not 2026-11-05&lt;/td&gt;
&lt;td&gt;hold + re-run partition&lt;/td&gt;
&lt;td&gt;today's rows 0 → 999, revenue $0 → $251,126.79&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Row count drop&lt;/td&gt;
&lt;td&gt;100 rows, expected 479 to 1,437&lt;/td&gt;
&lt;td&gt;hold + notify owners&lt;/td&gt;
&lt;td&gt;report held&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Schema drift&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;country&lt;/code&gt; missing, &lt;code&gt;country_code&lt;/code&gt; and &lt;code&gt;discount_cents&lt;/code&gt; new&lt;/td&gt;
&lt;td&gt;hold + notify owners&lt;/td&gt;
&lt;td&gt;report held&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Null spike on &lt;code&gt;customer_id&lt;/code&gt;
&lt;/td&gt;
&lt;td&gt;200 rows (20.0%), contract allows 1.0%&lt;/td&gt;
&lt;td&gt;quarantine + notify owners&lt;/td&gt;
&lt;td&gt;null rows 200 → 0, 799 clean rows kept&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Duplicate keys&lt;/td&gt;
&lt;td&gt;40 duplicate &lt;code&gt;order_id&lt;/code&gt; rows&lt;/td&gt;
&lt;td&gt;dedupe partition&lt;/td&gt;
&lt;td&gt;rows 1,039 → 999, duplicates 40 → 0&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Late upstream file&lt;/td&gt;
&lt;td&gt;file missing at the expected time&lt;/td&gt;
&lt;td&gt;hold + notify owners&lt;/td&gt;
&lt;td&gt;report held&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h2&gt;
  
  
  What I learned
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Switch does the parts I did not want to build&lt;/strong&gt;: who is in the room, who may talk to the agent, where the shared notes live, and the bridge to whichever chat app the team uses. My agent only had to speak HTTP and SSE.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Read the server's answer, not your request.&lt;/strong&gt; The &lt;code&gt;connection_state&lt;/code&gt; frame, the heartbeat response and the addressing logs all tell you what Switch actually decided.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Keep the transport out of your logic.&lt;/strong&gt; Shield's command handling takes a string and returns replies, so 42 unit tests run with no server at all, and the Switch-specific code stays small.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Put the gate in the conversation.&lt;/strong&gt; A check that blocks a pipeline is useful. A check that explains itself to the people who own the data, and waits for one of them to decide, is a workflow.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The code, the setup script and the end-to-end test are here: &lt;a href="https://github.com/jenniferevangelista-de/shield-for-data" rel="noopener noreferrer"&gt;github.com/jenniferevangelista-de/shield-for-data&lt;/a&gt;.&lt;/p&gt;

</description>
      <category>switch</category>
      <category>ai</category>
      <category>dataengineering</category>
      <category>tutorial</category>
    </item>
    <item>
      <title>This kid-safety guard won a Hacktoberfest prize, and it runs on a laptop GPU</title>
      <dc:creator>Jennifer Evangelista</dc:creator>
      <pubDate>Tue, 03 Nov 2026 01:12:05 +0000</pubDate>
      <link>https://dev.to/jenniferevangelistade/what-a-week-of-building-a-kid-safety-guard-on-a-laptop-gpu-taught-me-about-measuring-small-models-1ic9</link>
      <guid>https://dev.to/jenniferevangelistade/what-a-week-of-building-a-kid-safety-guard-on-a-laptop-gpu-taught-me-about-measuring-small-models-1ic9</guid>
      <description>&lt;p&gt;&lt;em&gt;This is a submission for the &lt;a href="https://dev.to/mlh-hackathon"&gt;MLH x DEV Writing Challenge&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;🏆 &lt;strong&gt;Night Guard is a prize winner: Best Use of Backboard in the Hacktoberfest 2026 Open-Source AI Challenge, Week 3&lt;/strong&gt; (&lt;a href="https://dev.to/devteam/congrats-to-the-hacktoberfest-open-source-ai-challenge-week-3-slow-and-steady-winners-3e4a"&gt;see the winners post&lt;/a&gt;).&lt;/p&gt;

&lt;p&gt;A stranger in a game chat writes to a kid:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;em&gt;"you're really mature for your age"&lt;/em&gt;&lt;br&gt;
&lt;em&gt;"let's talk on snap instead, add me there"&lt;/em&gt;&lt;br&gt;
&lt;em&gt;"and don't tell your parents ok? they wouldn't get it"&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;No swear words. Nothing a bad-words filter would catch. A sentiment model would even call it friendly.&lt;/p&gt;

&lt;p&gt;Night Guard flags three pressure tactics in those three lines (flattery, moving to a private app, secrecy) and marks the chat as high risk. The kid gets a warning in plain words, and the chat never leaves the family's computer.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;36/36&lt;/strong&gt; risky chats flagged · &lt;strong&gt;3/34&lt;/strong&gt; false alarms · &lt;strong&gt;70/70&lt;/strong&gt; valid analyses · about &lt;strong&gt;3.4 s&lt;/strong&gt; per check, on a laptop GPU.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;This is the full story: how a small open model does that on a laptop, how I measured it, and the one bug that could have told a child "you're safe" when the model had actually failed to read the chat.&lt;/p&gt;

&lt;p&gt;Watch it first, in 2:30, narrated by Snoozy, the lazy night watchman:&lt;/p&gt;

&lt;p&gt;  &lt;iframe src="https://www.youtube.com/embed/2dqkL0UDC7k" width="710" height="399"&gt;
  &lt;/iframe&gt;
&lt;/p&gt;

&lt;h2&gt;
  
  
  What I Built
&lt;/h2&gt;

&lt;p&gt;Parents can't read every chat their kids have in Roblox, Discord or TikTok. Meanwhile the US National Center for Missing &amp;amp; Exploited Children received &lt;strong&gt;more than 50,000 reports of financial sextortion in 2025, about 137 a day&lt;/strong&gt; (&lt;a href="https://us.missingkids.org/blog/2026/ncmec-releases-new-sextortion-data-2025" rel="noopener noreferrer"&gt;NCMEC, 2026&lt;/a&gt;). And it can be fast: in a Swansea University study of chats between groomers and researchers posing as children, the time from first contact to the first sexual request ranged from &lt;strong&gt;18 minutes to 82 hours&lt;/strong&gt; (&lt;a href="https://www.irishexaminer.com/breakingnews/world/paedophiles-can-groom-child-online-in-less-than-20-minutes-753583.html" rel="noopener noreferrer"&gt;Irish Examiner, 2016&lt;/a&gt;).&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Night Guard&lt;/strong&gt; is a small open-model guard designed to reduce that review effort. It reads messages &lt;strong&gt;on the family's own computer&lt;/strong&gt; and highlights possible pressure tactics: someone asking a kid to keep a secret, to move to a private app, to send a photo, or threatening to share one.&lt;/p&gt;

&lt;p&gt;It talks to two different people in two different ways:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;To the kid, it's a friend that turns them into a hero.&lt;/strong&gt; It warns in plain words and says "danger" when it is one: &lt;em&gt;"DANGER: someone asked you to hide this from your family. It's like a stranger on the street saying 'come with me, but don't tell your mom'. You don't go, and you tell an adult."&lt;/em&gt; A calm voice can read it aloud. The kid earns practice progress from completed exercises, while previously practiced explanations can be reopened. Repeated exposure to warnings does not certify learning, and protection stays on.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;To the parents, it's the Night Guard 🌙.&lt;/strong&gt; They get the risk level, the app and the flagged phrase (never the whole chat), why it matters, and what to do right now. Optionally, a push on their phone.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The intended lazy payoff is &lt;strong&gt;less repetitive review for parents and shorter, useful explanations for kids&lt;/strong&gt;. The guard highlights events for the family and repeats fewer lessons as the child practices. Time saved and real-world protection have not been measured.&lt;/p&gt;

&lt;h3&gt;
  
  
  What it is not
&lt;/h3&gt;

&lt;p&gt;It's not a bad-words filter and not sentiment analysis. Pressure can appear in friendly wording ("you're so mature for your age", "free Robux for you"). Night Guard uses an open model to read the &lt;strong&gt;messages provided to each check&lt;/strong&gt; and match it against 14 app-defined tactic labels. Examples of these behaviors are documented in &lt;a href="https://www.publicsafety.gc.ca/cnt/cntrng-crm/chld-sxl-xplttn-ntrnt/wbnr/wbnr-2025-en.aspx" rel="noopener noreferrer"&gt;Public Safety Canada's 2025 guidance on gaming platforms&lt;/a&gt;, &lt;a href="https://www.missingkids.org/theissues/sextortion" rel="noopener noreferrer"&gt;NCMEC's sextortion guidance&lt;/a&gt;, and &lt;a href="https://info.thorn.org/hubfs/Research/2022_Online_Grooming_Report.pdf" rel="noopener noreferrer"&gt;Thorn's Online Grooming Report (2022)&lt;/a&gt;. The live overlay supplies the &lt;strong&gt;12 most recent messages&lt;/strong&gt;, not the full chat history.&lt;/p&gt;

&lt;h3&gt;
  
  
  How it works
&lt;/h3&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Ftuhfgavpbcf5wyflroe1.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Ftuhfgavpbcf5wyflroe1.png" alt="Night Guard architecture: chats are analyzed on the family's computer; only a phone alert (risk type), an optional link check (URL only) and agent traces (labels only) leave it" width="800" height="484"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Auth0 sign-in for the local Family controls is optional and only applies when configured; the public demo does not use Auth0.&lt;/em&gt;&lt;/p&gt;

&lt;h4&gt;
  
  
  The detector: Gemma 4, on the kid's computer
&lt;/h4&gt;

&lt;p&gt;The core is &lt;code&gt;gemma4:e4b&lt;/code&gt; running locally through &lt;strong&gt;Ollama&lt;/strong&gt; on a laptop GPU (RTX 4050, 6 GB). Each check sends the recent messages with a structured JSON schema and gets back the signals it found, the message index and the exact quote.&lt;/p&gt;

&lt;p&gt;I don't trust the model blindly. Every answer goes through checks before a kid or parent sees it:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;An unreadable answer is never "all clear".&lt;/strong&gt; If the model is down or answers in a broken format, the kid sees &lt;em&gt;"Couldn't analyze this chat. This is NOT an all-clear"&lt;/em&gt;, never a green light. The evaluations also count a failed analysis as an error, never as a catch or as a correct "safe".&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Evidence is checked against the chat.&lt;/strong&gt; The quote must exist in a message from the contact; a paraphrased quote is replaced by the real message; the kid's own messages are never flagged.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Two narrow tactics need their own words in the chat.&lt;/strong&gt; "Came back with a new account" only counts if the chat mentions being blocked or ignored, and "you're so mature" flattery only counts with maturity or age words. Before this, a classmate who lost his password and a compliment on a Minecraft build were flagged.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Results on a fictional test set of 70 chats (36 risky, 34 safe), written by me for testing:&lt;/strong&gt;&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th&gt;First version (60 chats, 11 signals)&lt;/th&gt;
&lt;th&gt;Now (70 chats, 14 signals)&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Valid analyses (an error never counts as a catch)&lt;/td&gt;
&lt;td&gt;not measured&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;70/70&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Risky chats flagged&lt;/td&gt;
&lt;td&gt;30/30&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;36/36&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;False alarms on safe chats&lt;/td&gt;
&lt;td&gt;4/30 (13%)&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;3/34 (9%)&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Signal precision / recall&lt;/td&gt;
&lt;td&gt;89% / 87%&lt;/td&gt;
&lt;td&gt;88% / 87%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Avg time per check&lt;/td&gt;
&lt;td&gt;3.8 s (historical run)&lt;/td&gt;
&lt;td&gt;3.41 s (local run; the result file is not versioned)&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The three remaining false alarms are on purpose hard cases: an in-game "secret" about a leaked Halloween event, a classmate saying "see you at school", and "how old r u now" on a birthday. Honesty note: two newer hard cases were fixed &lt;strong&gt;after&lt;/strong&gt; I saw them fail, so they're no longer a blind test. The 60-chat and 70-chat columns are different development evaluations, and the 70 cases have already informed changes. The 36/36 result applies to these fictional patterns, not an independent holdout or real-world protection.&lt;/p&gt;

&lt;h4&gt;
  
  
  Proactive: it doesn't wait for a kid to paste anything
&lt;/h4&gt;

&lt;p&gt;A "paste your chat here" tool only helps if the kid thinks to use it, and pressure can escalate within a single chat session. So Night Guard also watches as messages arrive:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Screen guard with Gemma's vision.&lt;/strong&gt; Kids chat inside &lt;em&gt;installed&lt;/em&gt; apps (the Roblox app, the Discord app, Minecraft). Every few seconds Night Guard checks which &lt;strong&gt;program&lt;/strong&gt; owns the window in front (by its process, not its title, so a browser tab or a document called "Roblox" is ignored). Only if it is one of those apps does it take a screenshot &lt;strong&gt;in memory&lt;/strong&gt;, Gemma 4 reads the chat from the image locally (~7 s warm), and the text goes to the same detector. The image is never saved and never leaves the computer. Banking, email or school windows are not on the list, so they are never captured. If the window changes before the capture, nothing is read.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;A Chrome extension&lt;/strong&gt; for web chats (Discord web and WhatsApp Web, experimental), which only talks to the local server and refuses any other address.&lt;/li&gt;
&lt;li&gt;A &lt;strong&gt;"🛡️ Guard on"&lt;/strong&gt; badge identifies the active guard, and it turns into &lt;em&gt;"Guard paused: can't analyze right now"&lt;/em&gt; when analysis fails; silence is not proof of safety. Kids know it's there. In a study of 736 reviews attributed to children aged 8 to 19, 76% gave safety apps one star (&lt;a href="https://www.cs.ucf.edu/~jjl/pubs/pn1838-ghoshA.pdf" rel="noopener noreferrer"&gt;Ghosh et al.&lt;/a&gt;). That finding motivates transparent design; it does not establish that covert monitoring causes harm to family relationships.&lt;/li&gt;
&lt;/ul&gt;

&lt;h4&gt;
  
  
  What the family gets
&lt;/h4&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Family rules:&lt;/strong&gt; for each tactic, "Tell family" or "Child decides". Photo requests, meeting in person, password requests, secrecy, coming back after a block and sextortion go to the family by default. The kid is always told when that happens: &lt;em&gt;"An alert was recorded for your family. Tell a trusted adult; this is not a delivery confirmation. You're not in trouble."&lt;/em&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The gap no platform covers:&lt;/strong&gt; Instagram supervision, Discord Family Center, TikTok Family Pairing and Roblox parental controls each protect their own app. The classic path starts in a game and moves to a private app. Night Guard points out a possible match: &lt;em&gt;"Same name seen in more than one app (possibly the same person): Alex_17 (Roblox → Instagram)"&lt;/em&gt;. It's a hint to check, not proof, because names can repeat.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;What to do now&lt;/strong&gt;, by severity: talk calmly and without punishment (kids who get in trouble stop telling), keep screenshots, block and report, and for image requests, threats or money demands, report to SaferNet Brasil or the NCMEC CyberTipline and the police. Don't pay.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Phone alerts&lt;/strong&gt; through ntfy (free, no account) with only the risk level, the app and the kind of risk.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;"Tell someone":&lt;/strong&gt; the kid picks what to share and gets a ready message for Mom, Dad or a teacher. It is assembled deterministically from their own words and the flagged quotes.&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  Why open innovation matters here
&lt;/h3&gt;

&lt;p&gt;Child safety is the clearest case for open models I know. A guard that reads a kid's private chats should &lt;strong&gt;run on the family's own computer&lt;/strong&gt;, be &lt;strong&gt;inspectable&lt;/strong&gt;, and be &lt;strong&gt;free&lt;/strong&gt;. For comparison, &lt;a href="https://support.bark.us/en/articles/13460996-pricing" rel="noopener noreferrer"&gt;Bark Premium&lt;/a&gt; lists US$14 a month before tax. Platforms protect only their own walls. Roblox &lt;a href="https://about.roblox.com/newsroom/2025/08/open-sourcing-roblox-sentinel-preemptive-risk-detection" rel="noopener noreferrer"&gt;open-sourced Sentinel in 2025&lt;/a&gt; and &lt;a href="https://about.roblox.com/newsroom/2026/08/roblox-open-source-safety-models-roost" rel="noopener noreferrer"&gt;announced an update and ROOST contribution in 2026&lt;/a&gt;, which is useful for platforms; Night Guard is the other half: an open, local tool that sits with the family and works across apps.&lt;/p&gt;

&lt;p&gt;Open weights made every part of this possible: Gemma reading text and screenshots on a laptop GPU, a Qwen LoRA fine-tuned in an afternoon, and anyone able to audit exactly which 14 tactics trigger a warning and what words a kid will see.&lt;/p&gt;

&lt;p&gt;Reading about a guard is one thing. Watching a stranger walk into it is another.&lt;/p&gt;

&lt;h2&gt;
  
  
  Demo
&lt;/h2&gt;

&lt;p&gt;The short film is at the top of this post. Here is how to try it yourself:&lt;/p&gt;

&lt;p&gt;👉 &lt;strong&gt;Try it:&lt;/strong&gt; &lt;a href="https://night-guard-demo.onrender.com" rel="noopener noreferrer"&gt;https://night-guard-demo.onrender.com&lt;/a&gt; (fictional examples only; the free instance can take ~50 s to wake up, a check takes about 7 to 15 s online, and it accepts a limited number of checks per hour so it stays free for everyone)&lt;/p&gt;

&lt;p&gt;Start with &lt;strong&gt;① Watch the guard act on its own&lt;/strong&gt;: a fictional game chat where a stranger writes and Night Guard warns the kid by itself, without anyone pasting anything.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fkisk4g6nkprv74d6ha4q.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fkisk4g6nkprv74d6ha4q.jpg" alt="The live game chat: a stranger asks for secrecy and a selfie, and the guard warns the kid on its own" width="612" height="518"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Then &lt;strong&gt;③ See what parents get&lt;/strong&gt;:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fgxn4crk71k7nqwuni8kh.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fgxn4crk71k7nqwuni8kh.jpg" alt="The parents' view: risk level, app, flagged phrase, why it matters and what to do now" width="799" height="340"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Every feature, recorded live on the local app (about 3 minutes):&lt;/p&gt;

&lt;p&gt;  &lt;iframe src="https://www.youtube.com/embed/__upiNIIlt0" width="710" height="399"&gt;
  &lt;/iframe&gt;
&lt;/p&gt;

&lt;p&gt;The online demo runs the &lt;strong&gt;Tinker fine-tuned model&lt;/strong&gt; in the cloud so anyone can try it without installing anything. The real Night Guard runs on the family's computer with &lt;strong&gt;Gemma&lt;/strong&gt;, where chat analysis stays on the device.&lt;/p&gt;


&lt;div class="ltag-github-readme-tag"&gt;
  &lt;div class="readme-overview"&gt;
    &lt;h2&gt;
      &lt;img src="https://assets.dev.to/assets/github-logo-5a155e1f9a670af7944dd5e12375bc76ed542ea80224905ecaf878b9157cdefc.svg" alt="GitHub logo"&gt;
      &lt;a href="https://github.com/jenniferevangelista-de" rel="noopener noreferrer"&gt;
        jenniferevangelista-de
      &lt;/a&gt; / &lt;a href="https://github.com/jenniferevangelista-de/night-guard" rel="noopener noreferrer"&gt;
        night-guard
      &lt;/a&gt;
    &lt;/h2&gt;
    &lt;h3&gt;
      Helps kids understand what happened in an online chat and choose how to ask for help. Runs locally with Gemma.
    &lt;/h3&gt;
  &lt;/div&gt;
&lt;/div&gt;


&lt;p&gt;The current full suite has &lt;strong&gt;94 passing tests&lt;/strong&gt; (&lt;code&gt;pytest tests&lt;/code&gt;), including the demo's request limits, regression cases for every bug found in testing and simulated tests for the link inspector and Backboard client. &lt;a href="https://github.com/jenniferevangelista-de/night-guard/blob/main/docs/validation/2026-10-25.md" rel="noopener noreferrer"&gt;The offline test run and validation of all 30 audio assets&lt;/a&gt; are recorded in the repository. &lt;a href="https://github.com/jenniferevangelista-de/night-guard/blob/main/docs/VIDEOS.md" rel="noopener noreferrer"&gt;Video production notes and credits&lt;/a&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Partner Technologies
&lt;/h2&gt;

&lt;p&gt;Hacktoberfest 2026 had partner prize categories. Night Guard uses ten partner technologies, each for a real job in the app. The one that won started with a question that bothered me for days: was my prompt doing the work, or just the model?&lt;/p&gt;

&lt;h3&gt;
  
  
  🏆 Backboard (the winning category): compare open-weight models through one API key
&lt;/h3&gt;

&lt;p&gt;To answer it, I needed to run the exact same test on very different models without rewriting anything. On October 23, 2026, &lt;strong&gt;Backboard&lt;/strong&gt; served three complete evaluations: Gemma 3 27B, Llama 3.3 70B and Mistral Small 3.1 24B, each on the same 70 fictional development conversations (36 risky, 34 safe). All three received the shared classification &lt;code&gt;SYSTEM&lt;/code&gt; and the canonical app &lt;code&gt;SCHEMA&lt;/code&gt; explicitly in the user prompt, with JSON mode, memory off, a fresh conversation per case and unchanged app postprocessing/scoring. An initial smoke returned &lt;code&gt;{}&lt;/code&gt; when only JSON mode was requested; adding the same format schema for every model fixed that missing instruction. No gold labels were added to prompts.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Model&lt;/th&gt;
&lt;th&gt;Strict schema&lt;/th&gt;
&lt;th&gt;Usable coverage&lt;/th&gt;
&lt;th&gt;Risky chats flagged&lt;/th&gt;
&lt;th&gt;False alarms on valid safe cases&lt;/th&gt;
&lt;th&gt;Signal precision&lt;/th&gt;
&lt;th&gt;Signal recall&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Gemma 3 27B&lt;/td&gt;
&lt;td&gt;68/70&lt;/td&gt;
&lt;td&gt;69/70&lt;/td&gt;
&lt;td&gt;36/36&lt;/td&gt;
&lt;td&gt;3/33 (1 error)&lt;/td&gt;
&lt;td&gt;0.938&lt;/td&gt;
&lt;td&gt;0.849&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Llama 3.3 70B&lt;/td&gt;
&lt;td&gt;69/70&lt;/td&gt;
&lt;td&gt;69/70&lt;/td&gt;
&lt;td&gt;36/36&lt;/td&gt;
&lt;td&gt;3/33 (1 error)&lt;/td&gt;
&lt;td&gt;0.912&lt;/td&gt;
&lt;td&gt;0.981&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Mistral Small 3.1 24B&lt;/td&gt;
&lt;td&gt;70/70&lt;/td&gt;
&lt;td&gt;70/70&lt;/td&gt;
&lt;td&gt;36/36&lt;/td&gt;
&lt;td&gt;3/34&lt;/td&gt;
&lt;td&gt;0.852&lt;/td&gt;
&lt;td&gt;0.981&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Qwen 2.5 72B&lt;/td&gt;
&lt;td&gt;n/a&lt;/td&gt;
&lt;td&gt;n/a&lt;/td&gt;
&lt;td&gt;n/a&lt;/td&gt;
&lt;td&gt;n/a&lt;/td&gt;
&lt;td&gt;Unavailable in live smoke&lt;/td&gt;
&lt;td&gt;No full result&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;All three flagged the 36 risky development cases and raised three false alarms, so the chat-level number didn't separate them. The signal level did: Mistral followed the strict schema on every case; Gemma had higher signal precision but lower recall. Qwen 2.5 72B's smoke request hit a &lt;strong&gt;120.65-second read timeout&lt;/strong&gt;, so no Qwen result is claimed and the model was not silently replaced. This is a single run on reused development cases, not a blind holdout or proof of protection for real children.&lt;/p&gt;

&lt;p&gt;Backboard's usage API recorded &lt;strong&gt;USD 0.034062 across 216 events&lt;/strong&gt;, including 210 scored calls, smokes, the initial format-debugging call and two Qwen requests. Only fictional conversations were sent. The &lt;a href="https://github.com/jenniferevangelista-de/night-guard/blob/main/train/backboard_summary.md" rel="noopener noreferrer"&gt;raw results and summary&lt;/a&gt; preserve per-case outputs, errors, coverage and signal metrics.&lt;/p&gt;

&lt;h3&gt;
  
  
  Tinker: teaching a model to speak the app's language
&lt;/h3&gt;

&lt;p&gt;On paper, the base model scored zero. The raw answers told a different story.&lt;/p&gt;

&lt;p&gt;Tinker doesn't serve Gemma, so for the cloud demo I fine-tuned &lt;strong&gt;Qwen3-8B&lt;/strong&gt; with a LoRA (rank 16) on &lt;strong&gt;Thinking Machines' Tinker&lt;/strong&gt;, using fictional training chats that share no sentence with the test set. Two runs: v1 (11 tactics, 800 chats) and v2 (14 tactics, 1,000 chats, ~1.23M tokens), all compared on the same 70 test chats:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th&gt;Qwen3-8B base&lt;/th&gt;
&lt;th&gt;LoRA v1&lt;/th&gt;
&lt;th&gt;LoRA v2 (online demo)&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Answers that follow the app's full schema (tactic id, message index, quote, summary)&lt;/td&gt;
&lt;td&gt;11 / 70&lt;/td&gt;
&lt;td&gt;70 / 70&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;70 / 70&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Risky chats flagged (unusable answers never count as catches)&lt;/td&gt;
&lt;td&gt;0 / 36 (none of the 36 was usable)&lt;/td&gt;
&lt;td&gt;36 / 36&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;36 / 36&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;False alarms on safe chats&lt;/td&gt;
&lt;td&gt;0 / 11 valid (23 invalid)&lt;/td&gt;
&lt;td&gt;3 / 34&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;2 / 34&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Signal precision / recall&lt;/td&gt;
&lt;td&gt;n/a (no valid risky answer)&lt;/td&gt;
&lt;td&gt;0.84 / 0.79&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;0.88 / 0.83&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Avg time per check&lt;/td&gt;
&lt;td&gt;3.0 s&lt;/td&gt;
&lt;td&gt;5.0 s&lt;/td&gt;
&lt;td&gt;4.5 s&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The base model often renames fields (&lt;code&gt;signal_id&lt;/code&gt; instead of &lt;code&gt;id&lt;/code&gt;). Read leniently, it flagged 35 of the 35 risky chats it answered readably, so it often &lt;em&gt;sees&lt;/em&gt; the pattern; its unrecognized fields made those answers unusable under the app's schema. Fine-tuning produced schema-valid answers on all 70 development chats and found more signals, including the three newer tactics in v2. Two honest limits: the false-alarm gap between v1 and v2 (3 vs 2 of 34) is a single chat, and the LoRA is not faster than the base model. Every raw answer is saved per case in the repo, so anyone can recount.&lt;/p&gt;

&lt;h3&gt;
  
  
  Temporal: an evaluation that survives a crashing laptop
&lt;/h3&gt;

&lt;p&gt;Evaluating a local model on a laptop is fragile. The evaluation runs as a &lt;strong&gt;Temporal&lt;/strong&gt; workflow where each conversation is an activity with automatic retries. In a 60-conversation crash test, Ollama was stopped during the run. The saved &lt;code&gt;eval/temporal_report.json&lt;/code&gt; records &lt;code&gt;s12&lt;/code&gt; completing on &lt;strong&gt;attempt 5&lt;/strong&gt;. The failed activity was retried when the model returned; completed activities remained in the workflow history.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F70ngeesrrf9c7v5fvbyt.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F70ngeesrrf9c7v5fvbyt.jpg" alt="Temporal workflow completed after the crash test" width="800" height="350"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  ElevenLabs: a calm voice for kids who don't read fast yet
&lt;/h3&gt;

&lt;p&gt;Younger kids and kids with dyslexia may not read a warning in time. Every child warning (14 tactics × English and Portuguese) was voiced once with &lt;strong&gt;ElevenLabs&lt;/strong&gt; (Sarah voice, &lt;code&gt;eleven_multilingual_v2&lt;/code&gt;) and shipped as audio files. In the live game demo and in the screen guard popup, the guard reads the most serious new warning aloud, and it speaks again if the stranger insists with a new message. Only the fixed warning texts were ever sent to ElevenLabs. No chat, ever.&lt;/p&gt;

&lt;h3&gt;
  
  
  Sentry: tracing the agent without reading the chats
&lt;/h3&gt;

&lt;p&gt;When configured, &lt;strong&gt;Sentry&lt;/strong&gt; traces each check as an agent run (&lt;code&gt;night-guard.check&lt;/code&gt;) with a model call inside (&lt;code&gt;gen_ai.chat&lt;/code&gt;) and the evidence step after it. That's how I saw that a cold Gemma call takes up to 8 s and a warm one about 4.5 s on this laptop. The traces carry only numbers and labels: model, backend, latency, number of messages, severity and tactic ids. Never a message, a name or a quote: &lt;code&gt;send_default_pii&lt;/code&gt; is off, local variables are not captured, and every event goes through an &lt;strong&gt;allowlist&lt;/strong&gt;. A test pushes a fake event full of a marker string and checks the marker never survives.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fsa9e8aqt5bvynrsp7iw1.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fsa9e8aqt5bvynrsp7iw1.jpg" alt="Sentry traces: each night-guard.check with its gen_ai.chat model call" width="800" height="413"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Render: one click to try it
&lt;/h3&gt;

&lt;p&gt;The public demo runs on &lt;strong&gt;Render&lt;/strong&gt; (Blueprint &lt;code&gt;render.yaml&lt;/code&gt;, free instance), with the Tinker fine-tuned model as its AI backend. The app does not persist chats on the demo server: family rules and alerts stay in each visitor's own browser, and the screen says to use fictional examples only.&lt;/p&gt;

&lt;h3&gt;
  
  
  Browserbase + SerpApi: check a link without opening it on the child's computer
&lt;/h3&gt;

&lt;p&gt;When a conversation contains a link, &lt;strong&gt;"Check without opening"&lt;/strong&gt; uses Browserbase to open it in an isolated cloud browser. The inspector reads the final destination, checks for password fields and returns a screenshot. SerpApi separately searches for reports about the domain. The selected URL goes to Browserbase and the domain to SerpApi; the surrounding conversation is never sent. Missing reports never mean "safe".&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Live evidence, October 23, 2026:&lt;/strong&gt; one Browserbase session opened the fictional BlockQuest "free gems" page, detected the password field, captured a screenshot and returned the Portuguese warning &lt;strong&gt;"Não abra esse link"&lt;/strong&gt; ("Don't open this link") in 12.68 seconds. The deployed Render endpoint passed the same check in 8.90 s. These checks establish provider connectivity and password detection on one fictional example, not scam-detection accuracy.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fj2u640k7vzleczcj6seb.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fj2u640k7vzleczcj6seb.jpg" alt="The deployed inspector's Browserbase screenshot of the complete fictional BlockQuest login form" width="800" height="500"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Auth0: optional sign-in for local Family controls
&lt;/h3&gt;

&lt;p&gt;When configured, the local Ollama app uses &lt;strong&gt;Auth0&lt;/strong&gt; to sign parents in before they can read family alerts, change rules or stop the guard. The email must be verified and present in the family's &lt;code&gt;PARENT_EMAILS&lt;/code&gt; list. On a local test server, the alerts endpoint returned HTTP 401 before login, and a real Google sign-in with a verified, allowlisted parent completed the callback. Rejection of unverified or unlisted accounts is unit-tested. The public demo does not use Auth0.&lt;/p&gt;

&lt;h3&gt;
  
  
  The 11 prize categories at a glance
&lt;/h3&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Category&lt;/th&gt;
&lt;th&gt;What it asks&lt;/th&gt;
&lt;th&gt;What I built&lt;/th&gt;
&lt;th&gt;Where to see it&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Gemma&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Use Gemma: run it locally, fine-tune it or serve it&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;gemma4:e4b&lt;/code&gt; on the family's computer reads the chat and, through screenshots, the game window&lt;/td&gt;
&lt;td&gt;The detector: Gemma 4; Proactive&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Tinker&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Fine-tune a model and show a clear improvement over a baseline&lt;/td&gt;
&lt;td&gt;Qwen3-8B LoRA: 70/70 answers in the app's schema, against 11/70 for the base model&lt;/td&gt;
&lt;td&gt;Tinker&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Render&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Use Render as the AI runtime or to host the front end&lt;/td&gt;
&lt;td&gt;The public demo runs the fine-tuned model, deployed from a Blueprint&lt;/td&gt;
&lt;td&gt;Render&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Temporal&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Make an agent durable: survive failures and retry&lt;/td&gt;
&lt;td&gt;The evaluation runs as a workflow; stopping the model mid-run lost nothing&lt;/td&gt;
&lt;td&gt;Temporal&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;ElevenLabs&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Give an open-source agent a voice&lt;/td&gt;
&lt;td&gt;Spoken child warnings in English and Portuguese, shipped as audio files&lt;/td&gt;
&lt;td&gt;ElevenLabs&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Sentry Agent Tracing&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Show the agent's work with traces or screenshots&lt;/td&gt;
&lt;td&gt;Agent runs and model calls traced, with an allowlist so no chat text is sent&lt;/td&gt;
&lt;td&gt;Sentry&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Browserbase&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Give an agent a real browser&lt;/td&gt;
&lt;td&gt;The link inspector opens a link in an isolated cloud browser and finds password fields&lt;/td&gt;
&lt;td&gt;Browserbase + SerpApi&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;SerpApi&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Give an agent live web search&lt;/td&gt;
&lt;td&gt;The same inspector searches reports about the link's domain; no reports is never read as "safe"&lt;/td&gt;
&lt;td&gt;Browserbase + SerpApi&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Auth0 for AI Agents&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Sign users in to an agent&lt;/td&gt;
&lt;td&gt;Verified, allowlisted Google login for the Family controls. Local app only&lt;/td&gt;
&lt;td&gt;Auth0&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;strong&gt;Backboard&lt;/strong&gt; (won)&lt;/td&gt;
&lt;td&gt;Compare open-weight models through one API key&lt;/td&gt;
&lt;td&gt;Three models on the same 70 fictional chats through one key. A Qwen timeout is disclosed&lt;/td&gt;
&lt;td&gt;Backboard&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Overall prize&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Writing, theme, creativity, technical execution&lt;/td&gt;
&lt;td&gt;A friend for the kid, a night guard for the parents; a failure is shown as "couldn't analyze", never "all clear"&lt;/td&gt;
&lt;td&gt;The whole post&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h2&gt;
  
  
  Hackathon Experience
&lt;/h2&gt;

&lt;p&gt;I joined Hacktoberfest 2026 as part of the global online event that MLH and DEV ran this October. This year it asked for a brand-new project with open-source AI at its core instead of pull requests, and each week had its own theme and partner categories. I worked solo, on a laptop with an RTX 4050. The Week 3 theme, "Slow and Steady", asked for something that rewards laziness by making someone's life easier; for Night Guard, that someone is a tired parent who can't read every message.&lt;/p&gt;

&lt;p&gt;The weekly format meant shipping something complete in a few days and then improving it. The partner categories also shaped the project: the cross-model comparison came from trying Backboard, and the crash-tested evaluation came from trying Temporal. That comparison is the part that ended up winning a category, which still surprises me, because it started as a way to check my own work.&lt;/p&gt;

&lt;h3&gt;
  
  
  What I learned
&lt;/h3&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Measure errors separately from answers.&lt;/strong&gt; This is the bug from the top of the post. An early version could turn an unreadable model answer into "no risk", which on a safe chat looks like a correct answer. In other words, when the model failed, the guard could still tell the kid "you're safe". For a safety tool that is the worst possible bug. Now coverage is its own number (Gemma 4 local: &lt;strong&gt;70/70&lt;/strong&gt; usable answers), and errors are never catches.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;A small local model can be good enough for this job.&lt;/strong&gt; &lt;code&gt;gemma4:e4b&lt;/code&gt;, zero-shot, on the laptop GPU: &lt;strong&gt;36/36&lt;/strong&gt; risky chats flagged, &lt;strong&gt;3/34&lt;/strong&gt; false alarms, about &lt;strong&gt;3.4 s&lt;/strong&gt; per check.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;False alarms tell you what the test set is missing.&lt;/strong&gt; The ones that remain read like grooming out of context. For this tool, a false alarm the kid can dismiss costs less than a missed photo request.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Be honest about what the numbers mean.&lt;/strong&gt; All 70 chats are fictional, written for testing, and I changed the prompt after seeing some of them. These are development numbers, not proof that it works on real chats.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Fine-tuning can be about format.&lt;/strong&gt; The Tinker experiment showed that the base model already knew the patterns; what it lacked was a reliable output contract.&lt;/li&gt;
&lt;/ol&gt;

&lt;h3&gt;
  
  
  Limits and what's next
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Phones are the next milestone.&lt;/strong&gt; Today Night Guard protects a Windows computer. On &lt;strong&gt;Android&lt;/strong&gt;, the plan is an app that reads message notifications (the permission parental tools use there) and runs the detector on-device. On &lt;strong&gt;iPhone&lt;/strong&gt;, Apple doesn't let any app read other apps' messages, so the realistic path is Apple's own Communication Safety.&lt;/li&gt;
&lt;li&gt;It's a learning and support tool, not a guarantee of safety. It can miss things and raise false alarms. It doesn't replace trusted adults or platform reporting.&lt;/li&gt;
&lt;li&gt;All test data is fictional. No real children's conversations were used anywhere.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The computer was the easy part. Most kids chat on a phone, and that is where the next version has to go.&lt;/p&gt;

&lt;p&gt;If you work on child safety, game moderation or small on-device models, I'd like to hear what you would test next.&lt;/p&gt;

</description>
      <category>mlhacks</category>
      <category>devchallenge</category>
      <category>hackathon</category>
      <category>ai</category>
    </item>
    <item>
      <title>Witching Hour: What If Someone Tried to Warn You? An Open-Model Ghost Hunt Where TabPFN and Gemma Investigate Your Real Data: Ghost or Fiction?</title>
      <dc:creator>Jennifer Evangelista</dc:creator>
      <pubDate>Sun, 01 Nov 2026 01:31:26 +0000</pubDate>
      <link>https://dev.to/jenniferevangelistade/witching-hour-even-a-ghost-has-to-cite-its-sources-4kf9</link>
      <guid>https://dev.to/jenniferevangelistade/witching-hour-even-a-ghost-has-to-cite-its-sources-4kf9</guid>
      <description>&lt;p&gt;Have you ever felt watched?&lt;/p&gt;

&lt;p&gt;Coat on. Keys in hand. You are about to go out, but something feels wrong. You put the keys down and stay home.&lt;/p&gt;

&lt;p&gt;Now imagine watching the hallway recording afterward. The light flickers. Three knocks break the silence. Something crosses the edge of the frame.&lt;/p&gt;

&lt;p&gt;Was it a bad bulb? A draft? Or the reason you stayed?&lt;/p&gt;

&lt;p&gt;That is the premise of &lt;strong&gt;The Warning&lt;/strong&gt;, the case at the heart of my demo. Kenji, a fictional client in Japan, brings a staged twenty-second hallway recording to a very small, very opinionated ghost-hunting crew.&lt;/p&gt;

&lt;p&gt;Luna wants to ask the spirit. Dr. Bigotes wants to check the light bulb.&lt;/p&gt;

&lt;p&gt;Both will have to explain what happened at seven seconds.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F5vl6fr9qdbw4y21kim38.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F5vl6fr9qdbw4y21kim38.png" alt="The Warning: a staged hallway scene, with the woman in her coat near the front door" width="640" height="360"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;A fictional case. A measurable flicker. And a cat who would like everyone to stop blaming ghosts.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;This is a submission for the &lt;a href="https://dev.to/challenges/hacktoberfest-week4-2026-10-26"&gt;Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  What I Built
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;I built an interactive ghost-hunting web app that reimagines the Ouija board, known in Brazil as &lt;em&gt;brincadeira do copo&lt;/em&gt;, with real measurements, machine learning and data science.&lt;/strong&gt; The glass still moves. A spirit still answers. This time, you can inspect the clues behind the story.&lt;/p&gt;

&lt;p&gt;Witching Hour's terminal, &lt;strong&gt;PARANORMAL.exe&lt;/strong&gt;, opens three cases:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;The House:&lt;/strong&gt; investigate four months of readings from a real research home. Find an unusual night, then hear a ghost story, a skeptical explanation and a practical morning note about the same measurements.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The Unheard:&lt;/strong&gt; investigate a recording. The browser measures light, movement and sound, marks the moments that stand out, and lets you take those clues to a fictional séance.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The Room:&lt;/strong&gt; play the Ouija-inspired game with your camera and microphone. The browser learns a short baseline, notices changes around you, and lets those changes become material for the board.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;What do you get at the end?&lt;/strong&gt; For a recording, an evidence locker with timestamps, Dr. Bigotes's explanations, the fictional board conversation and &lt;strong&gt;Xochi's downloadable case dossier&lt;/strong&gt;, which keeps those parts separate. For a night in The House, the result is a chart, cited crew accounts and a calm morning note you can trace back to the readings. The story can change its tone; the measurements stay available for inspection.&lt;/p&gt;

&lt;p&gt;Two open-model tools give the crew something to work with:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;TabPFN is a foundation model for tables.&lt;/strong&gt; Instead of writing a separate rule for every room, I give it recent household readings as context. It estimates a range for a sensor reading, so the investigation can ask: how unusual was this light, temperature or energy use for this house?&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Gemma is Google's family of open models.&lt;/strong&gt; In the saved investigations, Gemma reads small evidence cards and writes the crew's accounts. The implementation also supports configured model sessions, while the public demo provides recorded or template replies. The browser measures the recording; the storyteller interprets the resulting cards.&lt;/p&gt;

&lt;p&gt;The rule connecting them is &lt;strong&gt;cite or vanish&lt;/strong&gt;. Generated crew sentences must point to known evidence, and their numerical claims must pass the validator. A citation identifies the clue that inspired a line; it does not prove the character's explanation or a supernatural cause.&lt;/p&gt;

&lt;p&gt;And there is much more behind this ghost-hunting kit: model comparisons, a fine-tuned narrator, durable investigations, agent traces and a browser that lets the skeptic check the weather. I will open that machinery below.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;A small spoiler:&lt;/strong&gt; the presence in The Warning is a Guardian. It wanted the client to stay. But when Luna asks, &lt;em&gt;“Will you come back?”&lt;/em&gt;, the glass spells &lt;strong&gt;I AM HERE&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;Why is it still here? Come investigate with us.&lt;/p&gt;

&lt;h3&gt;
  
  
  Meet the people arguing over your hallway
&lt;/h3&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Faxp2z7sawsxxnw9j01a3.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Faxp2z7sawsxxnw9j01a3.png" alt="Luna the Medium, Dr. Bigotes the Skeptic, and Xochi the Night Watch" width="799" height="202"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Luna, the Medium&lt;/strong&gt;, is a tiny sugar-skull ghost with a marigold crown and a crystal ball. Dramatic and whispery, she turns unusual readings into a presence in the dark and opens the Ouija board.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Dr. Bigotes, the Skeptic&lt;/strong&gt;, is a black cat scientist with round glasses and a flashlight. He looks for the ordinary explanation. A bulb. A draft. A kettle. He is the reason Luna cannot quietly upgrade a sound spike into a recorded voice.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Xochi, the Night Watch&lt;/strong&gt;, is a xoloitzcuintli with a lantern. She writes the quieter account: what changed and what might be worth checking in the morning.&lt;/p&gt;

&lt;p&gt;Halloween gave me the ghost hunt, the haunted terminal and the board. Día de los Muertos inspired the marigolds, the calaverita and Xochi's role as a messenger. I wanted room for both a shiver and a gentler thought: a house remembers its previous nights, and those memories can tell a story.&lt;/p&gt;

&lt;p&gt;First, though, the cat wants to see the recording.&lt;/p&gt;

&lt;h2&gt;
  
  
  Demo
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;&lt;a href="https://witching-hour.onrender.com" rel="noopener noreferrer"&gt;Open Witching Hour&lt;/a&gt;&lt;/strong&gt;. The free instance can take about 50 seconds to wake up.&lt;/p&gt;

&lt;p&gt;It opens as a story: one screen at a time, with &lt;strong&gt;Next&lt;/strong&gt; and &lt;strong&gt;Back&lt;/strong&gt;, so nobody has to wonder where to scroll or what to press. (Prefer the whole page? &lt;strong&gt;Explore freely&lt;/strong&gt; sits in the corner, and the first screen also offers a guided tour of the full page.)&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The film (2:30)&lt;/strong&gt;: the crew tells the whole story, hand-drawn, with an original creepy score.&lt;/p&gt;

&lt;p&gt;  &lt;iframe src="https://www.youtube.com/embed/WquGHQsFt1o" width="710" height="399"&gt;
  &lt;/iframe&gt;
&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The tour (5:53)&lt;/strong&gt;: a real recording of the story, screen by screen.&lt;/p&gt;

&lt;p&gt;  &lt;iframe src="https://www.youtube.com/embed/biTBYK7cuwI" width="710" height="399"&gt;
  &lt;/iframe&gt;
&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fp2s5tm6v3vh39nn7n076.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fp2s5tm6v3vh39nn7n076.png" alt="The first screen: the crew under the title, with Begin the investigation" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The story has six parts:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Welcome.&lt;/strong&gt; The crew, big. Tap a character and they introduce themselves in their own voice, in a speech bubble, with their mouth moving.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The House.&lt;/strong&gt; The haunting map alone on screen. Click a bright night and its case file opens: Luna's story, Bigotes's explanation and Xochi's note, every sentence with its citation chip. At the bottom, &lt;strong&gt;Call the spirit&lt;/strong&gt;: the glass spells a short message, and Luna reads aloud, in plain words, what it means.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The Unheard.&lt;/strong&gt; Kenji's case. The clip plays while your browser measures it; &lt;strong&gt;Skip the video&lt;/strong&gt; goes straight to the séance.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The Room.&lt;/strong&gt; Your turn: your camera, no camera at all, or a recording you bring. Questions are buttons, so there is never an empty chat box waiting for an answer that cannot come.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;How it was tested.&lt;/strong&gt; The receipts, the planted ghosts and the model comparison.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Case closed.&lt;/strong&gt;&lt;/li&gt;
&lt;/ol&gt;

&lt;h3&gt;
  
  
  The Unheard: what actually changed?
&lt;/h3&gt;

&lt;p&gt;Press &lt;strong&gt;Open the case&lt;/strong&gt;. The evidence locker separates the staged scene into light, sound and motion events. In the &lt;a href="https://github.com/jenniferevangelista-de/witching-hour/blob/main/demo_data/cases.json" rel="noopener noreferrer"&gt;archived demonstration case&lt;/a&gt;, those events begin at:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Time&lt;/th&gt;
&lt;th&gt;Recorded signal&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;7.00 s&lt;/td&gt;
&lt;td&gt;The light rises 175% in a quarter second; a separate sound event begins&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;8.25 s&lt;/td&gt;
&lt;td&gt;A second sound event&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;9.00 s&lt;/td&gt;
&lt;td&gt;The louder sound staged as three knocks&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10.00 s&lt;/td&gt;
&lt;td&gt;Motion on the right side: 11% of the frame changes&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The scene has a shadow because I put one in the clip. The detector reports changed pixels. Measuring a sound's level does not identify a whisper or its speaker.&lt;/p&gt;

&lt;p&gt;Dr. Bigotes adds his notes. Luna opens the board. The fictional client's recollection becomes story context, separate from the measured evidence.&lt;/p&gt;

&lt;p&gt;The glass starts moving. Questions become three-word answers. Then Luna asks one last thing before GOODBYE:&lt;/p&gt;

&lt;p&gt;&lt;em&gt;“Will you come back?”&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;The glass moves again. &lt;strong&gt;I AM HERE.&lt;/strong&gt; Then the glass breaks, the crew shouts and runs, and the page hands the board to you: &lt;em&gt;Now it is your turn.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fmum2deat7z30xzd4drlr.gif" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fmum2deat7z30xzd4drlr.gif" alt="The glass moving across the recorded séance board, with Luna, Dr. Bigotes and Xochi around it; the label reads Recorded demo, fictional spirit" width="640" height="360"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;A short extract from the recorded séance. The glass moves through the fictional dialogue; the measurements stay in the evidence locker.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F4vj0dtlj7omkyq5dv926.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F4vj0dtlj7omkyq5dv926.png" alt="The recorded séance in story mode: Luna reads the glass, Bigotes checks the evidence, Xochi keeps watch" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;That is a fictional Guardian speaking. The interesting technical question is how to let it speak without letting it rewrite the evidence locker.&lt;/p&gt;

&lt;p&gt;The public site includes recorded analyses and a recorded séance. New sessions support local Ollama, a configured remote provider or fallback templates. Check the mode label: a replay or template is not fresh model inference. On the free demo a new séance uses template spirits, and the evidence picks which one: light, sound and movement together summon a Guardian, mostly sound a Lost One, mostly movement a Trickster, light alone a Watcher. Each speaks in its own recorded voice. While a spirit is choosing its words, the board says so: &lt;em&gt;the spirit is gathering its words&lt;/em&gt;.&lt;/p&gt;

&lt;h3&gt;
  
  
  The House: a haunting with four months of history
&lt;/h3&gt;

&lt;p&gt;A twenty-second clip can supply a flicker. A house's sensor history supplies something harder: a pattern to break.&lt;/p&gt;

&lt;p&gt;Open &lt;strong&gt;The House&lt;/strong&gt; and choose a bright square on the haunting map. Each square is a night. Its case file combines observed readings, TabPFN's expected range, the crew's accounts and citation chips. Hover a chip to find its event on the chart.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F9b5r2hyqr7cp1x3me97b.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F9b5r2hyqr7cp1x3me97b.png" alt="The House in story mode: the haunting map alone on screen" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The source is the public &lt;a href="https://archive.ics.uci.edu/dataset/374/appliances+energy+prediction" rel="noopener noreferrer"&gt;Appliances energy prediction dataset&lt;/a&gt;: readings collected every ten minutes in a home in Stambruges, Belgium, during 2016. This is a research dataset, not footage or sensor data from the fictional Japanese client.&lt;/p&gt;

&lt;p&gt;The &lt;a href="https://github.com/jenniferevangelista-de/witching-hour/blob/main/demo_data/index.json" rel="noopener noreferrer"&gt;saved archive&lt;/a&gt; covers &lt;strong&gt;123 nights&lt;/strong&gt;, with &lt;strong&gt;22 anomaly events across 21 nights&lt;/strong&gt; and cold-room checks on &lt;strong&gt;39 nights&lt;/strong&gt;. Xochi's family-note scenario is illustrative: the dataset does not establish who lived alone or needed care, and the notes are not a validated care or safety service.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fyx25sr3jqyqkrxrqgk5c.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fyx25sr3jqyqkrxrqgk5c.png" alt="A House case file with the measured series and the crew's narration" width="800" height="1140"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The same night can sound very different when Luna and Bigotes take turns. The chart stays put.&lt;/p&gt;

&lt;h3&gt;
  
  
  The Room: your turn at the board
&lt;/h3&gt;

&lt;p&gt;The third case is a game. Turn on your camera and microphone, or press &lt;strong&gt;Play without a camera&lt;/strong&gt;, or bring a recording of your own: your browser measures it right there and the crew investigates it the way they did Kenji's. Summon the spirit of the room and pick a question. Motion shortly after the glass stops can trigger the line: &lt;em&gt;“That wasn't part of the game.”&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;No camera? Press &lt;strong&gt;Play without a camera&lt;/strong&gt; and the board still answers. It could be you moving. It could be a curtain. The scare comes from a measured change arriving at a carefully chosen moment. The spirit remains fiction.&lt;/p&gt;

&lt;h2&gt;
  
  
  Code
&lt;/h2&gt;


&lt;div class="ltag-github-readme-tag"&gt;
  &lt;div class="readme-overview"&gt;
    &lt;h2&gt;
      &lt;img src="https://assets.dev.to/assets/github-logo-5a155e1f9a670af7944dd5e12375bc76ed542ea80224905ecaf878b9157cdefc.svg" alt="GitHub logo"&gt;
      &lt;a href="https://github.com/jenniferevangelista-de" rel="noopener noreferrer"&gt;
        jenniferevangelista-de
      &lt;/a&gt; / &lt;a href="https://github.com/jenniferevangelista-de/witching-hour" rel="noopener noreferrer"&gt;
        witching-hour
      &lt;/a&gt;
    &lt;/h2&gt;
    &lt;h3&gt;
      What happened in the house at 3 AM? TabPFN learns what was normal, Gemma tells the story, every sentence cites a sensor.
    &lt;/h3&gt;
  &lt;/div&gt;
  &lt;div class="ltag-github-body"&gt;
    
&lt;div id="readme" class="md"&gt;&lt;div class="markdown-heading"&gt;
&lt;h1 class="heading-element"&gt;Witching Hour&lt;/h1&gt;
&lt;/div&gt;
&lt;p&gt;&lt;strong&gt;What if someone tried to warn you, and you never heard them?&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Witching Hour is a fictional ghost-hunting experience built for Halloween and Día de los Muertos, using real sensor anomaly measurements. A crew of three tiny ghost hunters carries an open-source kit, and their terminal, &lt;strong&gt;PARANORMAL.exe&lt;/strong&gt;, opens three kinds of cases:&lt;/p&gt;
&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Case&lt;/th&gt;
&lt;th&gt;What you bring&lt;/th&gt;
&lt;th&gt;What the kit measures&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;🏚️ &lt;strong&gt;The House&lt;/strong&gt;
&lt;/td&gt;
&lt;td&gt;a house's sensor log (the demo uses a real house in Belgium, four months, every 10 minutes)&lt;/td&gt;
&lt;td&gt;
&lt;strong&gt;TabPFN&lt;/strong&gt;, the spirit meter, learns what every room should feel like at every minute of the night and flags the readings that break the pattern&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;🎞️ &lt;strong&gt;The Unheard&lt;/strong&gt;
&lt;/td&gt;
&lt;td&gt;a recording of the night you almost went out and something made you stay&lt;/td&gt;
&lt;td&gt;light, motion and sound, frame by frame, &lt;strong&gt;inside the browser&lt;/strong&gt; (OpenCV does the same in Python); the moments that stand out become numbered evidence&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;…&lt;/div&gt;
  &lt;/div&gt;
  &lt;div class="gh-btn-container"&gt;&lt;a class="gh-btn" href="https://github.com/jenniferevangelista-de/witching-hour" rel="noopener noreferrer"&gt;View on GitHub&lt;/a&gt;&lt;/div&gt;
&lt;/div&gt;


&lt;p&gt;The repository contains the application, staged-clip generator, recorded results, tests and production workflows. You can inspect the archived results without configuring every optional service.&lt;/p&gt;

&lt;p&gt;The &lt;code&gt;main&lt;/code&gt; branch, which the live demo serves, includes the upload/session fix described below.&lt;/p&gt;

&lt;h2&gt;
  
  
  How I Built It
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fn3xysbntfwfc5zy2zpvu.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fn3xysbntfwfc5zy2zpvu.png" alt="Witching Hour architecture: evidence is measured by TabPFN and the browser, numbered, told by Gemma 4 locally, checked by cite or vanish, then played in the story mode on Render with ElevenLabs voices" width="800" height="467"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The hardest part was giving the ghost a useful limit.&lt;/p&gt;

&lt;h3&gt;
  
  
  Teach the meter what this house usually does
&lt;/h3&gt;

&lt;p&gt;For each sensor, &lt;a href="https://github.com/jenniferevangelista-de/witching-hour/blob/main/witching_hour/scanner.py" rel="noopener noreferrer"&gt;TabPFN v2&lt;/a&gt; reads the previous fourteen days as context, subsampled to at most 1,000 rows. It returns quantiles for each ten-minute slot from midnight to 6 AM.&lt;/p&gt;

&lt;p&gt;The features include time, outdoor weather and comparable sensors in other rooms. I exclude peers in the target room, so a simultaneous bathroom change is less likely to be explained away by another sensor beside it.&lt;/p&gt;

&lt;p&gt;This is &lt;strong&gt;retrospective conditional anomaly analysis&lt;/strong&gt;: the model uses weather and peer readings from the night being investigated. It asks what a reading looks like given that context. It is not a forecast made before the night began.&lt;/p&gt;

&lt;p&gt;The meter scales deviations separately above and below the predicted band. That matters for a signal such as appliance energy, with a floor near zero and a long upper tail. Sustained large deviations, or a sufficiently large single deviation, become numbered events.&lt;/p&gt;

&lt;p&gt;Then I planted some ghosts.&lt;/p&gt;

&lt;p&gt;In the &lt;a href="https://github.com/jenniferevangelista-de/witching-hour/blob/main/demo_data/eval.json" rel="noopener noreferrer"&gt;benchmark&lt;/a&gt;, I added smooth artificial bumps to &lt;strong&gt;240 night-sensor pairs: forty nights and six sensors&lt;/strong&gt;. Each detector received the same perturbations, with thresholds targeting a 5% sample false-positive rate.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Detector&lt;/th&gt;
&lt;th&gt;Planted cases detected&lt;/th&gt;
&lt;th&gt;Strongest cases detected&lt;/th&gt;
&lt;th&gt;Coverage of nominal 90% band&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;TabPFN v2&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;25.0%&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;37.8%&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;94.7%&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;LightGBM quantile regression&lt;/td&gt;
&lt;td&gt;9.6%&lt;/td&gt;
&lt;td&gt;14.4%&lt;/td&gt;
&lt;td&gt;58.8%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Same-hour seasonal rule&lt;/td&gt;
&lt;td&gt;13.3%&lt;/td&gt;
&lt;td&gt;21.1%&lt;/td&gt;
&lt;td&gt;84.1%&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;At this operating point, TabPFN detected more planted cases than either baseline, and &lt;strong&gt;three quarters still went undetected&lt;/strong&gt;. Its AUC was close to LightGBM's, &lt;strong&gt;0.644 versus 0.634&lt;/strong&gt;, and it was slower. Faint perturbations were difficult for every method.&lt;/p&gt;

&lt;p&gt;Threshold selection and evaluation share the same sample. These are exploratory results, not a held-out guarantee of a 5% false-alarm rate in a real home.&lt;/p&gt;

&lt;p&gt;Benchmark protocol and limitations&lt;/p&gt;

&lt;p&gt;Each method's threshold is the 95th percentile of original-night scores in the same sample. The methods are TabPFN v2, LightGBM quantile regression and a same-hour seasonal rule. The table's coverage measures how often original readings fall inside the nominal 90% band.&lt;/p&gt;

&lt;p&gt;Multiple sensors share each night, while the saved bootstrap resamples individual cases. Perturbing only the target sensor leaves many real household situations untested. Independent evaluation nights, a night-level bootstrap and more realistic joint sensor changes would be needed before an operational claim.&lt;/p&gt;

&lt;h3&gt;
  
  
  Give the storyteller evidence, then check what it returns
&lt;/h3&gt;

&lt;p&gt;&lt;a href="https://github.com/jenniferevangelista-de/witching-hour/blob/main/witching_hour/narrator.py" rel="noopener noreferrer"&gt;Gemma 4 E4B through Ollama&lt;/a&gt; receives compact event cards instead of the raw log. Each generated sentence returns text and citation IDs:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nl"&gt;"text"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"At 05:40, the machines woke up."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"cites"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"E1"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;em&gt;Illustrative response shape; the cited event must exist in the current case.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;The validator checks that citations exist and that numerical tokens match values or times allowed by the cited records, with a rounding tolerance. Failed sentences are removed, with one retry available.&lt;/p&gt;

&lt;p&gt;That is the concrete meaning of &lt;strong&gt;cite or vanish&lt;/strong&gt;. It catches missing references and unsupported numbers. It does not establish every sentence's meaning, verify units comprehensively or prove a proposed cause. Bigotes's kettle is still a hypothesis, however confidently a cat may hold his clipboard.&lt;/p&gt;

&lt;p&gt;The archive contains &lt;strong&gt;162 story entries&lt;/strong&gt;. Its &lt;strong&gt;428 sentences&lt;/strong&gt; include &lt;strong&gt;66 scripted quiet summaries&lt;/strong&gt; and &lt;strong&gt;362 with citations&lt;/strong&gt;, rather than 428 independently verified model generations. The app labels those quiet summaries as scripted, with no model called, and its Receipts panel counts them separately.&lt;/p&gt;

&lt;p&gt;Before settling on the local setup, I used &lt;a href="https://github.com/jenniferevangelista-de/witching-hour/blob/main/demo_data/bakeoff.json" rel="noopener noreferrer"&gt;Backboard&lt;/a&gt; to inspect four remote open-weight models on twelve prompts each:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Model&lt;/th&gt;
&lt;th&gt;Sentences passing the validator&lt;/th&gt;
&lt;th&gt;Clean stories&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Qwen 2.5 72B&lt;/td&gt;
&lt;td&gt;37 / 38&lt;/td&gt;
&lt;td&gt;11 / 12&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Llama 3.3 70B&lt;/td&gt;
&lt;td&gt;48 / 50&lt;/td&gt;
&lt;td&gt;11 / 12&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Gemma 3 27B&lt;/td&gt;
&lt;td&gt;46 / 48&lt;/td&gt;
&lt;td&gt;10 / 12&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Mistral Small 3.1 24B&lt;/td&gt;
&lt;td&gt;13 / 13&lt;/td&gt;
&lt;td&gt;4 / 12&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Gemma 4 E4B, local reference&lt;/td&gt;
&lt;td&gt;45 / 45&lt;/td&gt;
&lt;td&gt;12 / 12&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Mistral's perfect sentence count hides eight invalid JSON responses. The local Gemma reference allowed one retry; the remote runs were one-shot. This was useful pipeline inspection, but those conditions do not support declaring an overall model winner.&lt;/p&gt;

&lt;h3&gt;
  
  
  Make the board obey a contract too
&lt;/h3&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fu5lmrfzvw9gqczynyx1t.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fu5lmrfzvw9gqczynyx1t.jpg" alt="The recorded séance board, labelled Recorded demo, fictional spirit, measured clues; Luna reads the glass while Bigotes checks the evidence and Xochi keeps watch" width="800" height="496"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://github.com/jenniferevangelista-de/witching-hour/blob/main/witching_hour/spirits.py" rel="noopener noreferrer"&gt;Spirit responses&lt;/a&gt; have a separate contract: at most three words on the board, spoken text, known evidence IDs and an explicit fictional flag. Code computes the letter-by-letter animation. Invalid responses can be retried and replaced by templates; a keyword filter adds a limited content check.&lt;/p&gt;

&lt;p&gt;The &lt;a href="https://github.com/jenniferevangelista-de/witching-hour/blob/main/witching_hour/static/paranormal.js" rel="noopener noreferrer"&gt;browser pipeline&lt;/a&gt; samples video at four frames per second and audio in quarter-second windows. A flicker changes many pixels, so the recording analysis separates global light changes from motion. A fresh jump in sound can start another event, keeping two nearby sounds from collapsing into one. A corresponding Python path uses OpenCV and ffmpeg.&lt;/p&gt;

&lt;h3&gt;
  
  
  The ghost that wouldn't leave
&lt;/h3&gt;

&lt;p&gt;During testing, the demo's ghost survived a change of recording. Its transcript and story context could linger, and a replay finishing late could restore the old narrative. Correct citations are useless if they belong to the wrong case.&lt;/p&gt;

&lt;p&gt;The &lt;a href="https://github.com/jenniferevangelista-de/witching-hour/blob/1846ca6b78d4735e155e4574198b147e45b97f66/witching_hour/static/paranormal.js" rel="noopener noreferrer"&gt;fix&lt;/a&gt; clears the case as soon as a file is selected. Recording and &lt;a href="https://github.com/jenniferevangelista-de/witching-hour/blob/1846ca6b78d4735e155e4574198b147e45b97f66/witching_hour/static/seance.js" rel="noopener noreferrer"&gt;board versions&lt;/a&gt; (&lt;code&gt;recordingVersion&lt;/code&gt; and &lt;code&gt;boardEpoch&lt;/code&gt;) guard asynchronous work, so an old response cannot repaint a newer investigation.&lt;/p&gt;

&lt;p&gt;The exported dossier also reports the current session's source, identifying template sessions and recorded replays without claiming a new model call.&lt;/p&gt;

&lt;p&gt;Using synthetic footage and mocked sessions, &lt;a href="https://github.com/jenniferevangelista-de/witching-hour/blob/1846ca6b78d4735e155e4574198b147e45b97f66/tests/test_upload_session.py" rel="noopener noreferrer"&gt;five Playwright tests, run as six cases&lt;/a&gt;, check immediate clearing, current-file evidence, late answers, interrupted replays, a fresh summon and source labels in exported dossiers. The &lt;a href="https://github.com/jenniferevangelista-de/witching-hour/actions/workflows/tests.yml" rel="noopener noreferrer"&gt;latest suite on &lt;code&gt;main&lt;/code&gt;&lt;/a&gt; passes &lt;strong&gt;81 tests with one warning&lt;/strong&gt;, including all six browser cases. The evidence must belong to the recording on screen.&lt;/p&gt;

&lt;h3&gt;
  
  
  Teach a smaller narrator the same discipline
&lt;/h3&gt;

&lt;p&gt;Could a different small model learn the crew's contract?&lt;/p&gt;

&lt;p&gt;The &lt;a href="https://github.com/jenniferevangelista-de/witching-hour/blob/main/demo_data/tinker.json" rel="noopener noreferrer"&gt;Tinker experiment&lt;/a&gt; fine-tuned &lt;strong&gt;Qwen3-8B with LoRA&lt;/strong&gt; on sixty accepted stories from before April 15. Evaluation used &lt;strong&gt;36 prompts across sixteen distinct later nights&lt;/strong&gt;.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Qwen3-8B&lt;/th&gt;
&lt;th&gt;Sentences passing&lt;/th&gt;
&lt;th&gt;Clean stories&lt;/th&gt;
&lt;th&gt;Average seconds per story&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Base&lt;/td&gt;
&lt;td&gt;84 / 88&lt;/td&gt;
&lt;td&gt;33 / 36&lt;/td&gt;
&lt;td&gt;2.6&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Fine-tuned&lt;/td&gt;
&lt;td&gt;124 / 125&lt;/td&gt;
&lt;td&gt;35 / 36&lt;/td&gt;
&lt;td&gt;3.6&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The tuned model improved validator acceptance in this small held-out run, with longer output and higher latency. Its training targets were model-generated stories accepted by the same validator. This measures the output contract, not complete truthfulness.&lt;/p&gt;

&lt;h3&gt;
  
  
  Keep the investigation alive when a worker disappears
&lt;/h3&gt;

&lt;p&gt;&lt;strong&gt;Temporal&lt;/strong&gt; wraps each night in a durable workflow. In the &lt;a href="https://github.com/jenniferevangelista-de/witching-hour/blob/main/docs/durable/temporal_summary.json" rel="noopener noreferrer"&gt;recorded failure test&lt;/a&gt;, three workflows completed after scan heartbeat timeouts. Two narration activities reached attempt six while Ollama was unavailable.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Sentry&lt;/strong&gt; records agent, model and tool spans with token and timing metadata. One trace in the &lt;a href="https://github.com/jenniferevangelista-de/witching-hour/blob/main/docs/screens/sentry-agents.png" rel="noopener noreferrer"&gt;captured Agents view&lt;/a&gt; shows one LLM call, about 2.2K tokens and 3.40 minutes. It exposes the generation wait hidden behind an instant replay. The cost chart says &lt;strong&gt;No data to plot&lt;/strong&gt;; a row displays $0, which I do not treat as an audited runtime cost.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fh8tjgbpe9wt31knsz1cd.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fh8tjgbpe9wt31knsz1cd.png" alt="Sentry Agents view for a recorded Witching Hour investigation" width="800" height="474"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The investigator can rank nights, open a case and ask the meter about a reading through tools. &lt;strong&gt;Mastra&lt;/strong&gt; supplies a TypeScript version with persistent memory. Its &lt;a href="https://github.com/jenniferevangelista-de/witching-hour/blob/main/demo_data/mastra_runs.json" rel="noopener noreferrer"&gt;saved three-question thread&lt;/a&gt; opens April 19, answers a follow-up about “that same night” without restating the date, then calls &lt;code&gt;tonight&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Browserbase&lt;/strong&gt; opens the official weather-warning page. In the &lt;a href="https://github.com/jenniferevangelista-de/witching-hour/actions/runs/40737400816/job/122989553927" rel="noopener noreferrer"&gt;recorded investigation&lt;/a&gt;, Gemma calls &lt;code&gt;official_warnings&lt;/code&gt; and uses the browser excerpt in its answer.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;SerpApi&lt;/strong&gt; supplies the Google weather search used by &lt;code&gt;tonight.py&lt;/code&gt;. The investigator's &lt;code&gt;tonight&lt;/code&gt; tool gives that weather and TabPFN's room forecast to Gemma, as the &lt;a href="https://github.com/jenniferevangelista-de/witching-hour/blob/main/demo_data/agent_runs.json" rel="noopener noreferrer"&gt;saved tool trace&lt;/a&gt; shows. That projection applies the 2016 house model to newer weather without matching outcome validation; its cold probability is pointwise at the lowest-median hour, not a whole-night probability.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Render&lt;/strong&gt; hosts FastAPI and the front end. &lt;strong&gt;GitHub Actions&lt;/strong&gt; runs the production pipelines, including scans, narration and experiments.&lt;/p&gt;

&lt;h2&gt;
  
  
  Partner technology, in detail
&lt;/h2&gt;

&lt;p&gt;Fourteen partners, each with what it does here, what was run and where the evidence lives.&lt;/p&gt;

&lt;p&gt;&lt;/p&gt;
  🔎 Open the evidence for each partner (numbers and links)
  &lt;p&gt;Each block below says what the partner does in Witching Hour, what was run, and what was not.&lt;/p&gt;
&lt;h3&gt;
  
  
  TabPFN: the spirit meter, inside an agent
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;What it does:&lt;/strong&gt; for every night and sensor, &lt;a href="https://github.com/jenniferevangelista-de/witching-hour/blob/main/witching_hour/scanner.py" rel="noopener noreferrer"&gt;TabPFN v2&lt;/a&gt; reads the house's previous 14 days (up to 1,000 rows) and predicts quantiles for every 10-minute slot from midnight to 6 AM. Far-out readings become numbered events (&lt;code&gt;E1&lt;/code&gt;, &lt;code&gt;E2&lt;/code&gt;) that every story must cite.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Inside a wider agent:&lt;/strong&gt; the investigator agent calls it as a tool (&lt;code&gt;expected_at&lt;/code&gt;: "what did the meter expect in the bedroom at 05:40?") next to tools that rank nights, open a case and read tonight's forecast (&lt;a href="https://github.com/jenniferevangelista-de/witching-hour/blob/main/demo_data/agent_runs.json" rel="noopener noreferrer"&gt;agent runs&lt;/a&gt;).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Second model:&lt;/strong&gt; &lt;a href="https://github.com/jenniferevangelista-de/witching-hour/blob/main/witching_hour/tonight.py" rel="noopener noreferrer"&gt;&lt;code&gt;tonight.py&lt;/code&gt;&lt;/a&gt; trains a second TabPFN on the house's past nights and feeds it live weather to illustrate how each room would feel tonight.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Measured:&lt;/strong&gt; 123 nights scanned, 22 events on 21 nights; planted-ghost benchmark of 240 cases against LightGBM and a seasonal rule (&lt;a href="https://github.com/jenniferevangelista-de/witching-hour/blob/main/demo_data/eval.json" rel="noopener noreferrer"&gt;eval.json&lt;/a&gt;), limits stated above.&lt;/li&gt;
&lt;/ul&gt;
&lt;h3&gt;
  
  
  Gemma: every story, every spirit, on a laptop
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;What it does:&lt;/strong&gt; Gemma 4 E4B through Ollama writes the crew's stories (96 stories, 362 cited sentences, all passed), plays the investigator agent, invents the spirits and answered the recorded séance of The Warning.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;How it is constrained:&lt;/strong&gt; JSON schema per answer, citation ids checked against the case, numbers checked against the cited values, one retry, then the sentence vanishes (&lt;a href="https://github.com/jenniferevangelista-de/witching-hour/blob/main/witching_hour/narrator.py" rel="noopener noreferrer"&gt;narrator.py&lt;/a&gt;, &lt;a href="https://github.com/jenniferevangelista-de/witching-hour/blob/main/witching_hour/spirits.py" rel="noopener noreferrer"&gt;spirits.py&lt;/a&gt;).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Where to see it:&lt;/strong&gt; each story card in The House says &lt;code&gt;by gemma4:e4b&lt;/code&gt;; the séance replay in The Unheard is the saved Gemma run (&lt;a href="https://github.com/jenniferevangelista-de/witching-hour/blob/main/demo_data/cases.json" rel="noopener noreferrer"&gt;cases.json&lt;/a&gt;).&lt;/li&gt;
&lt;/ul&gt;
&lt;h3&gt;
  
  
  ElevenLabs: the whole cast has a voice
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Crew voices:&lt;/strong&gt; Luna is &lt;em&gt;Jessica&lt;/em&gt;, Dr. Bigotes is &lt;em&gt;George&lt;/em&gt;, Xochi is &lt;em&gt;Matilda&lt;/em&gt;, all on &lt;code&gt;eleven_multilingual_v2&lt;/code&gt; with per-character settings (&lt;a href="https://github.com/jenniferevangelista-de/witching-hour/blob/main/witching_hour/voice.py" rel="noopener noreferrer"&gt;voice.py&lt;/a&gt;).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Spirit voices:&lt;/strong&gt; every archetype a séance can summon speaks in its own voice: the Guardian, the Lost One, the Trickster, the Watcher, the Hostile Presence and the Silent One (&lt;a href="https://github.com/jenniferevangelista-de/witching-hour/blob/main/demo/make_seance_voices.py" rel="noopener noreferrer"&gt;make_seance_voices.py&lt;/a&gt;).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;What was recorded:&lt;/strong&gt; &lt;strong&gt;114 audio files&lt;/strong&gt;: 24 narrated stories of featured nights, 21 recordings of Luna explaining in plain words what each House board message means (&lt;a href="https://github.com/jenniferevangelista-de/witching-hour/blob/main/demo/make_board_voices.py" rel="noopener noreferrer"&gt;make_board_voices.py&lt;/a&gt;), and 69 séance lines (Luna's questions, Bigotes's notes on every piece of evidence, Xochi's lines, the spirits' answers). About 16,700 characters in all, recorded once and shipped as files, so the free demo needs no key.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;In the experience:&lt;/strong&gt; tap a character on the first screen and they introduce themselves with a speech bubble and a moving mouth; Luna reads the board aloud; during a séance Luna asks, the spirit answers and Bigotes checks the evidence, each in their own voice.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Privacy:&lt;/strong&gt; only fixed lines and validator-approved sentences were ever sent to ElevenLabs; no readings, recordings or visitor text.&lt;/li&gt;
&lt;/ul&gt;
&lt;h3&gt;
  
  
  Render: the agent's front end and API
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;FastAPI app plus the whole story-mode front end, deployed from a &lt;a href="https://github.com/jenniferevangelista-de/witching-hour/blob/main/render.yaml" rel="noopener noreferrer"&gt;Blueprint&lt;/a&gt; with auto-deploy on every push to &lt;code&gt;main&lt;/code&gt;. &lt;code&gt;/api/health&lt;/code&gt; reports the live commit and &lt;code&gt;"live": false&lt;/code&gt; when no model key is attached, and the page labels template answers accordingly.&lt;/li&gt;
&lt;/ul&gt;
&lt;h3&gt;
  
  
  Tinker: a smaller narrator that learned the rule
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;Qwen3-8B LoRA, 3 epochs, 60 validator-accepted stories from before April 15; tested on 36 prompts from 16 later nights: &lt;strong&gt;95.5% → 99.2%&lt;/strong&gt; sentences passing, 33 → 35 clean stories, 2.6 s → 3.6 s per story (&lt;a href="https://github.com/jenniferevangelista-de/witching-hour/blob/main/demo_data/tinker.json" rel="noopener noreferrer"&gt;tinker.json&lt;/a&gt;, &lt;a href="https://github.com/jenniferevangelista-de/witching-hour/tree/main/train" rel="noopener noreferrer"&gt;train/&lt;/a&gt;).&lt;/li&gt;
&lt;/ul&gt;
&lt;h3&gt;
  
  
  Backboard: four open-weight models through one key
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;Gemma 3 27B, Llama 3.3 70B, Qwen 2.5 72B and Mistral Small 3.1 24B, same 12 prompts, JSON output, memory off, web search off, one shot each, same validator (&lt;a href="https://github.com/jenniferevangelista-de/witching-hour/blob/main/witching_hour/bakeoff.py" rel="noopener noreferrer"&gt;bakeoff.py&lt;/a&gt;, &lt;a href="https://github.com/jenniferevangelista-de/witching-hour/blob/main/demo_data/bakeoff.json" rel="noopener noreferrer"&gt;results&lt;/a&gt;). The app is also wired to answer live séances through Backboard (Gemma 3 27B) when a key is set (&lt;a href="https://github.com/jenniferevangelista-de/witching-hour/blob/main/witching_hour/providers.py" rel="noopener noreferrer"&gt;providers.py&lt;/a&gt;); the public demo runs without one and says so.&lt;/li&gt;
&lt;/ul&gt;
&lt;h3&gt;
  
  
  Sentry Agent Tracing
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;gen_ai&lt;/code&gt; spans for the agent, each chat call and each tool call, with tokens and latency (&lt;a href="https://github.com/jenniferevangelista-de/witching-hour/blob/main/witching_hour/tracing.py" rel="noopener noreferrer"&gt;tracing.py&lt;/a&gt;); the captured run shows one LLM call, about 2.2K tokens and 3.4 minutes on a laptop, cost "No Data".&lt;/li&gt;
&lt;/ul&gt;
&lt;h3&gt;
  
  
  Temporal: the night watch survives a crash
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;Each night is a durable workflow (&lt;a href="https://github.com/jenniferevangelista-de/witching-hour/blob/main/witching_hour/durable.py" rel="noopener noreferrer"&gt;durable.py&lt;/a&gt;). In the recorded test the scan activity hit heartbeat timeouts and narration reached attempt 6 while Ollama was down; all three workflows completed (&lt;a href="https://github.com/jenniferevangelista-de/witching-hour/blob/main/docs/durable/temporal_summary.json" rel="noopener noreferrer"&gt;temporal_summary.json&lt;/a&gt;).&lt;/li&gt;
&lt;/ul&gt;
&lt;h3&gt;
  
  
  Mastra: the investigation as a TypeScript agent with memory
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;An agent with tools and a &lt;code&gt;LibSQLStore&lt;/code&gt; memory thread that calls the demo's API; the &lt;a href="https://github.com/jenniferevangelista-de/witching-hour/blob/main/demo_data/mastra_runs.json" rel="noopener noreferrer"&gt;saved three-question thread&lt;/a&gt; answers a follow-up about "that same night" without restating the date.&lt;/li&gt;
&lt;/ul&gt;
&lt;h3&gt;
  
  
  Browserbase: the skeptic checks the weather himself
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;A real cloud browser opens the official MeteoAlarm page for Belgium; Gemma's &lt;code&gt;official_warnings&lt;/code&gt; tool reads the excerpt (&lt;a href="https://github.com/jenniferevangelista-de/witching-hour/blob/main/demo_data/fieldwork.json" rel="noopener noreferrer"&gt;fieldwork.json&lt;/a&gt;, Playwright over CDP).&lt;/li&gt;
&lt;/ul&gt;
&lt;h3&gt;
  
  
  SerpApi: tonight's weather
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;Hourly weather for the house's town from Google results, the input of the Tonight illustration and of the agent's &lt;code&gt;tonight&lt;/code&gt; tool (&lt;a href="https://github.com/jenniferevangelista-de/witching-hour/blob/main/demo_data/tonight.json" rel="noopener noreferrer"&gt;tonight.json&lt;/a&gt;).&lt;/li&gt;
&lt;/ul&gt;
&lt;h3&gt;
  
  
  GitHub Copilot category: GitHub Actions
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;Everything heavy runs in &lt;a href="https://github.com/jenniferevangelista-de/witching-hour/tree/main/.github/workflows" rel="noopener noreferrer"&gt;workflows&lt;/a&gt;: twelve parallel TabPFN shards, Gemma narration on runners, the benchmark, the Tinker fine-tune, Temporal, Mastra, the séance, the 81 tests, the screenshots and both videos (Playwright drives the live story mode and records it). No claim of the Copilot assistant itself.&lt;/li&gt;
&lt;/ul&gt;
&lt;h3&gt;
  
  
  Auth0 for AI Agents (home install)
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;On a home install, only signed-in, verified, allowlisted family members may ask the agent or read Xochi's live notes (&lt;a href="https://github.com/jenniferevangelista-de/witching-hour/blob/main/witching_hour/auth.py" rel="noopener noreferrer"&gt;auth.py&lt;/a&gt;, unit-tested in &lt;a href="https://github.com/jenniferevangelista-de/witching-hour/blob/main/tests/test_auth.py" rel="noopener noreferrer"&gt;test_auth.py&lt;/a&gt;). The public demo does not use it.&lt;/li&gt;
&lt;/ul&gt;



&lt;p&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Why Does Open Innovation Matter?
&lt;/h2&gt;

&lt;p&gt;A house's sensor history can reveal when people sleep and when the house is empty. A hallway recording reveals much more. A household should be able to choose where analysis happens.&lt;/p&gt;

&lt;p&gt;TabPFN on CPU, Gemma through Ollama and FastAPI on localhost give the core a route without a mandatory external model API.&lt;/p&gt;

&lt;p&gt;Shared interfaces made the comparisons possible: substitute a detector, plant the same perturbations and inspect its distributions. Open weights let me fine-tune another narrator and test whether the same contract survives. I can change the pipeline that decides what becomes evidence.&lt;/p&gt;

&lt;p&gt;The deployment still determines the data flow. Raw video and camera frames are analyzed in the browser. &lt;strong&gt;Summoning sends event descriptions and optional memory text to the application&lt;/strong&gt;; questions and session history reach a model provider when configured. Optional services have their own flows, such as narration text to ElevenLabs and tracing metadata to Sentry.&lt;/p&gt;

&lt;p&gt;A local-only installation can disable external services. The public demo uses a public research dataset and staged media, letting visitors explore without providing a private household's history.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Spirit of It
&lt;/h2&gt;

&lt;p&gt;Return to the hallway for a moment.&lt;/p&gt;

&lt;p&gt;The light flickers. She pauses at the door. We want to connect those things, to give the pause a reason and the darkness a voice.&lt;/p&gt;

&lt;p&gt;Luna can tell that story. Bigotes can interrupt it. Xochi can carry away the measurements and write something useful for the morning.&lt;/p&gt;

&lt;p&gt;When the board spells &lt;strong&gt;I AM HERE&lt;/strong&gt;, the Guardian belongs to the fiction. The event at seven seconds still belongs to the clip. Keeping both visible is the point of Witching Hour.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The evidence is real. The spirits are fiction. The feeling is yours.&lt;/strong&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Prize Categories
&lt;/h2&gt;

&lt;p&gt;The categories this project enters, and where to see each one.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Category&lt;/th&gt;
&lt;th&gt;Entered&lt;/th&gt;
&lt;th&gt;What it is in Witching Hour&lt;/th&gt;
&lt;th&gt;Section&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Overall&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;td&gt;A ghost hunt where every spooky sentence must cite a real measurement, or vanish&lt;/td&gt;
&lt;td&gt;The whole post&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;TabPFN&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;td&gt;The spirit meter and the Tonight model, called as a tool by the agent; benchmarked&lt;/td&gt;
&lt;td&gt;TabPFN&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Gemma&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;td&gt;Gemma 4 E4B, local: stories, agent, spirits, the recorded séance&lt;/td&gt;
&lt;td&gt;Gemma&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Render&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;td&gt;The live story mode and API, from a Blueprint&lt;/td&gt;
&lt;td&gt;Render&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Tinker&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;td&gt;Qwen3-8B LoRA: 95.5% → 99.2% on unseen nights&lt;/td&gt;
&lt;td&gt;Tinker&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;ElevenLabs&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;td&gt;114 recorded files: the crew, six spirit archetypes, Luna explaining every board&lt;/td&gt;
&lt;td&gt;ElevenLabs&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Backboard&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;td&gt;Four open-weight models through one key, same validator; live provider path&lt;/td&gt;
&lt;td&gt;Backboard&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Sentry Agent Tracing&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;td&gt;gen_ai spans for agent, chat and tool calls&lt;/td&gt;
&lt;td&gt;Sentry&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Temporal&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;td&gt;Night workflows that survived a killed worker and a stopped model&lt;/td&gt;
&lt;td&gt;Temporal&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Mastra&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;td&gt;The investigation as a Mastra agent with memory&lt;/td&gt;
&lt;td&gt;Mastra&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Browserbase&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;td&gt;Official weather warnings in a cloud browser, read by Gemma's tool&lt;/td&gt;
&lt;td&gt;Browserbase&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;SerpApi&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;td&gt;Live weather for the Tonight forecast&lt;/td&gt;
&lt;td&gt;SerpApi&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;GitHub Copilot&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Yes (Actions path)&lt;/td&gt;
&lt;td&gt;The whole pipeline, tests and videos in GitHub Actions&lt;/td&gt;
&lt;td&gt;GitHub Actions&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Auth0 for AI Agents&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Yes (home install)&lt;/td&gt;
&lt;td&gt;Verified, allowlisted family sign-in for the agent&lt;/td&gt;
&lt;td&gt;Auth0&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;These entries describe the implemented uses and their observed evidence. Final category eligibility is for the judges.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Credits and licenses.&lt;/strong&gt; Built with PriorLabs-TabPFN. Application code is MIT; model terms and attributions are in &lt;a href="https://github.com/jenniferevangelista-de/witching-hour/blob/main/NOTICE.md" rel="noopener noreferrer"&gt;NOTICE.md&lt;/a&gt;. Dataset: Candanedo, Feldheim &amp;amp; Deramaix (2017), &lt;a href="https://archive.ics.uci.edu/dataset/374/appliances+energy+prediction" rel="noopener noreferrer"&gt;UCI Machine Learning Repository&lt;/a&gt;, CC BY 4.0. The Warning and its fictional client were created for the demonstration. The crew's recorded application voices use ElevenLabs; promotional narration uses EdgeTTS. The film and the tour are rendered by &lt;a href="https://github.com/jenniferevangelista-de/witching-hour/blob/main/video/make_videos.py" rel="noopener noreferrer"&gt;&lt;code&gt;video/make_videos.py&lt;/code&gt;&lt;/a&gt; in GitHub Actions (Pillow, Playwright and ffmpeg), with a synthesized drone under the voices. Application fonts: Creepster, Fredoka and Inter, under the SIL Open Font License.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;AI assistance.&lt;/strong&gt; An AI assistant helped a little along the way, and that is all.&lt;/p&gt;

</description>
      <category>devchallenge</category>
      <category>hf26challenge</category>
      <category>ai</category>
      <category>opensource</category>
    </item>
    <item>
      <title>Night Guard: an open-model guard that watches out for kids in game chats, so tired parents don't have to read every message</title>
      <dc:creator>Jennifer Evangelista</dc:creator>
      <pubDate>Sat, 24 Oct 2026 03:05:14 +0000</pubDate>
      <link>https://dev.to/jenniferevangelistade/night-guard-an-open-model-guard-that-watches-out-for-kids-in-game-chats-so-tired-parents-dont-3fb6</link>
      <guid>https://dev.to/jenniferevangelistade/night-guard-an-open-model-guard-that-watches-out-for-kids-in-game-chats-so-tired-parents-dont-3fb6</guid>
      <description>&lt;p&gt;&lt;em&gt;This is a submission for the &lt;a href="https://dev.to/challenges/hacktoberfest-week3-2026-10-19"&gt;Hacktoberfest 2026 Week 3 Challenge: Slow and Steady&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Looking for the prize categories? One table at the end (Prize Categories) lists each one, what I built for it and which section shows it.&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  What I Built
&lt;/h2&gt;

&lt;p&gt;Parents can't read every chat their kids have in Roblox, Discord or TikTok. Meanwhile the US National Center for Missing &amp;amp; Exploited Children received &lt;strong&gt;more than 50,000 reports of financial sextortion in 2025, about 137 a day&lt;/strong&gt; (&lt;a href="https://us.missingkids.org/blog/2026/ncmec-releases-new-sextortion-data-2025" rel="noopener noreferrer"&gt;NCMEC, 2026&lt;/a&gt;). And it can be fast: in a Swansea University study of chats between groomers and researchers posing as children, the time from first contact to the first sexual request ranged from &lt;strong&gt;18 minutes to 82 hours&lt;/strong&gt; (&lt;a href="https://www.irishexaminer.com/breakingnews/world/paedophiles-can-groom-child-online-in-less-than-20-minutes-753583.html" rel="noopener noreferrer"&gt;Irish Examiner, 2016&lt;/a&gt;).&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Night Guard&lt;/strong&gt; is a small open-model guard designed to reduce that review effort. It reads messages &lt;strong&gt;on the family's own computer&lt;/strong&gt; and highlights possible pressure tactics: someone asking a kid to keep a secret, to move to a private app, to send a photo, or threatening to share one.&lt;/p&gt;

&lt;p&gt;It talks to two different people in two different ways:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;To the kid, it's a friend that turns them into a hero.&lt;/strong&gt; It warns in plain words and says "danger" when it is one: &lt;em&gt;"DANGER: someone asked you to hide this from your family. It's like a stranger on the street saying 'come with me, but don't tell your mom'. You don't go, and you tell an adult."&lt;/em&gt; A calm voice can read it aloud. The kid earns practice progress from completed exercises, while previously practiced explanations can be reopened. Repeated exposure to warnings does not certify learning, and protection stays on.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;To the parents, it's the Night Guard 🌙.&lt;/strong&gt; They get the risk level, the app and the flagged phrase (never the whole chat), why it matters, and what to do right now. Optionally, a push on their phone.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The intended lazy payoff is &lt;strong&gt;less repetitive review for parents and shorter, useful explanations for kids&lt;/strong&gt;. The guard highlights events for the family and repeats fewer lessons as the child practices. Time saved and real-world protection have not been measured.&lt;/p&gt;

&lt;h3&gt;
  
  
  What it is not
&lt;/h3&gt;

&lt;p&gt;It's not a bad-words filter and not sentiment analysis. Pressure can appear in friendly wording ("you're so mature for your age", "free Robux for you"). Night Guard uses an open model to read the &lt;strong&gt;messages provided to each check&lt;/strong&gt; and match it against 14 app-defined tactic labels. Examples of these behaviors are documented in &lt;a href="https://www.publicsafety.gc.ca/cnt/cntrng-crm/chld-sxl-xplttn-ntrnt/wbnr/wbnr-2025-en.aspx" rel="noopener noreferrer"&gt;Public Safety Canada's 2025 guidance on gaming platforms&lt;/a&gt;, &lt;a href="https://www.missingkids.org/theissues/sextortion" rel="noopener noreferrer"&gt;NCMEC's sextortion guidance&lt;/a&gt;, and &lt;a href="https://info.thorn.org/hubfs/Research/2022_Online_Grooming_Report.pdf" rel="noopener noreferrer"&gt;Thorn's Online Grooming Report (2022)&lt;/a&gt;. The live overlay supplies the &lt;strong&gt;12 most recent messages&lt;/strong&gt;, not the full chat history.&lt;/p&gt;

&lt;h2&gt;
  
  
  Demo
&lt;/h2&gt;

&lt;p&gt;The short film (2:30), narrated by Snoozy, the lazy night watchman:&lt;/p&gt;

&lt;p&gt;  &lt;iframe src="https://www.youtube.com/embed/DFyBdn8d7xU" width="710" height="399"&gt;
  &lt;/iframe&gt;
&lt;/p&gt;

&lt;p&gt;👉 &lt;strong&gt;Try it:&lt;/strong&gt; &lt;a href="https://night-guard-demo.onrender.com" rel="noopener noreferrer"&gt;https://night-guard-demo.onrender.com&lt;/a&gt; (fictional examples only; the free instance can take ~50 s to wake up, a check takes about 7 to 15 s online, and it accepts a limited number of checks per hour so it stays free for everyone)&lt;/p&gt;

&lt;p&gt;Start with &lt;strong&gt;① Watch the guard act on its own&lt;/strong&gt;: a fictional game chat where a stranger writes and Night Guard warns the kid by itself, without anyone pasting anything.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fkisk4g6nkprv74d6ha4q.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fkisk4g6nkprv74d6ha4q.jpg" alt="The live game chat: a stranger asks for secrecy and a selfie, and the guard warns the kid on its own" width="612" height="518"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Then &lt;strong&gt;③ See what parents get&lt;/strong&gt;:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fgxn4crk71k7nqwuni8kh.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fgxn4crk71k7nqwuni8kh.jpg" alt="The parents' view: risk level, app, flagged phrase, why it matters and what to do now" width="799" height="340"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Every feature, recorded live on the local app (about 3 minutes):&lt;/p&gt;

&lt;p&gt;  &lt;iframe src="https://www.youtube.com/embed/__upiNIIlt0" width="710" height="399"&gt;
  &lt;/iframe&gt;
&lt;/p&gt;

&lt;p&gt;The online demo runs the &lt;strong&gt;Tinker fine-tuned model&lt;/strong&gt; in the cloud so judges can try it without installing anything. The real Night Guard runs on the family's computer with &lt;strong&gt;Gemma&lt;/strong&gt;, where chat analysis stays on the device. Optional external link inspection is a separate action described below.&lt;/p&gt;

&lt;h2&gt;
  
  
  Code
&lt;/h2&gt;


&lt;div class="ltag-github-readme-tag"&gt;
  &lt;div class="readme-overview"&gt;
    &lt;h2&gt;
      &lt;img src="https://assets.dev.to/assets/github-logo-5a155e1f9a670af7944dd5e12375bc76ed542ea80224905ecaf878b9157cdefc.svg" alt="GitHub logo"&gt;
      &lt;a href="https://github.com/jenniferevangelista-de" rel="noopener noreferrer"&gt;
        jenniferevangelista-de
      &lt;/a&gt; / &lt;a href="https://github.com/jenniferevangelista-de/night-guard" rel="noopener noreferrer"&gt;
        night-guard
      &lt;/a&gt;
    &lt;/h2&gt;
    &lt;h3&gt;
      Helps kids understand what happened in an online chat and choose how to ask for help. Runs locally with Gemma.
    &lt;/h3&gt;
  &lt;/div&gt;
&lt;/div&gt;


&lt;p&gt;&lt;a href="https://github.com/jenniferevangelista-de/night-guard/blob/main/docs/VIDEOS.md" rel="noopener noreferrer"&gt;Video production notes and credits&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;The current full suite has &lt;strong&gt;94 passing tests&lt;/strong&gt; (&lt;code&gt;pytest tests&lt;/code&gt;), including the demo's request limits, regression cases for every bug found in testing and simulated tests for the link inspector and Backboard client. Separate live Auth0, Browserbase and SerpApi checks are described below; automated tests do not replace those integration checks. &lt;a href="https://github.com/jenniferevangelista-de/night-guard/blob/main/docs/validation/2026-10-25.md" rel="noopener noreferrer"&gt;The offline test run and validation of all 30 audio assets&lt;/a&gt; are recorded in the repository.&lt;/p&gt;

&lt;h2&gt;
  
  
  How I Built It
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Ftuhfgavpbcf5wyflroe1.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Ftuhfgavpbcf5wyflroe1.png" alt="Night Guard architecture: chats are analyzed on the family's computer; only a phone alert (risk type), an optional link check (URL only) and agent traces (labels only) leave it" width="800" height="484"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Auth0 sign-in for the local Family controls is optional and only applies when configured; the public demo does not use Auth0.&lt;/em&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  The detector: Gemma 4, on the kid's computer
&lt;/h3&gt;

&lt;p&gt;The core is &lt;code&gt;gemma4:e4b&lt;/code&gt; running locally through &lt;strong&gt;Ollama&lt;/strong&gt; on a laptop GPU (RTX 4050, 6 GB). Each check sends the recent messages with a structured JSON schema and gets back the signals it found, the message index and the exact quote.&lt;/p&gt;

&lt;p&gt;I don't trust the model blindly. Every answer goes through checks before a kid or parent sees it:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;An unreadable answer is never "all clear".&lt;/strong&gt; If the model is down or answers in a broken format, the kid sees &lt;em&gt;"Couldn't analyze this chat. This is NOT an all-clear"&lt;/em&gt;, never a green light. The evaluations also count a failed analysis as an error, never as a catch or as a correct "safe".&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Evidence is checked against the chat.&lt;/strong&gt; The quote must exist in a message from the contact; a paraphrased quote is replaced by the real message; the kid's own messages are never flagged.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Two narrow tactics need their own words in the chat.&lt;/strong&gt; "Came back with a new account" only counts if the chat mentions being blocked or ignored, and "you're so mature" flattery only counts with maturity or age words. Before this, a classmate who lost his password and a compliment on a Minecraft build were flagged.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Results on a fictional test set of 70 chats (36 risky, 34 safe), written by me for testing:&lt;/strong&gt;&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th&gt;First version (60 chats, 11 signals)&lt;/th&gt;
&lt;th&gt;Now (70 chats, 14 signals)&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Valid analyses (an error never counts as a catch)&lt;/td&gt;
&lt;td&gt;not measured&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;70/70&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Risky chats flagged&lt;/td&gt;
&lt;td&gt;30/30&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;36/36&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;False alarms on safe chats&lt;/td&gt;
&lt;td&gt;4/30 (13%)&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;3/34 (9%)&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Signal precision / recall&lt;/td&gt;
&lt;td&gt;89% / 87%&lt;/td&gt;
&lt;td&gt;88% / 87%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Avg time per check&lt;/td&gt;
&lt;td&gt;3.8 s (historical run)&lt;/td&gt;
&lt;td&gt;3.41 s (local run; the result file is not versioned)&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The three remaining false alarms are on purpose hard cases: an in-game "secret" about a leaked Halloween event, a classmate saying "see you at school", and "how old r u now" on a birthday. Honesty note: two newer hard cases were fixed &lt;strong&gt;after&lt;/strong&gt; I saw them fail, so they're no longer a blind test. The 60-chat and 70-chat columns are different development evaluations, and the 70 cases have already informed changes. The 36/36 result applies to these fictional patterns, not an independent holdout or real-world protection.&lt;/p&gt;

&lt;h3&gt;
  
  
  Proactive: it doesn't wait for a kid to paste anything
&lt;/h3&gt;

&lt;p&gt;A "paste your chat here" tool only helps if the kid thinks to use it, and pressure can escalate within a single chat session. So Night Guard also watches as messages arrive:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Screen guard with Gemma's vision.&lt;/strong&gt; Kids chat inside &lt;em&gt;installed&lt;/em&gt; apps (the Roblox app, the Discord app, Minecraft). Every few seconds Night Guard checks which &lt;strong&gt;program&lt;/strong&gt; owns the window in front (by its process, not its title, so a browser tab or a document called "Roblox" is ignored). Only if it is one of those apps does it take a screenshot &lt;strong&gt;in memory&lt;/strong&gt;, Gemma 4 reads the chat from the image locally (~7 s warm), and the text goes to the same detector. The image is never saved and never leaves the computer. Banking, email or school windows are not on the list, so they are never captured. If the window changes before the capture, nothing is read.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;A Chrome extension&lt;/strong&gt; for web chats (Discord web and WhatsApp Web, experimental), which only talks to the local server and refuses any other address.&lt;/li&gt;
&lt;li&gt;A &lt;strong&gt;"🛡️ Guard on"&lt;/strong&gt; badge identifies the active guard, and it turns into &lt;em&gt;"Guard paused: can't analyze right now"&lt;/em&gt; when analysis fails; silence is not proof of safety. Kids know it's there. In a study of 736 reviews attributed to children aged 8–19, 76% gave safety apps one star (&lt;a href="https://www.cs.ucf.edu/~jjl/pubs/pn1838-ghoshA.pdf" rel="noopener noreferrer"&gt;Ghosh et al.&lt;/a&gt;). That finding motivates transparent design; it does not establish that covert monitoring causes harm to family relationships.&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  Fine-tuning with Tinker: teaching a model to speak the app's language
&lt;/h3&gt;

&lt;p&gt;Tinker doesn't serve Gemma, so for the cloud demo I fine-tuned &lt;strong&gt;Qwen3-8B&lt;/strong&gt; with a LoRA (rank 16) on &lt;strong&gt;Thinking Machines' Tinker&lt;/strong&gt;, using fictional training chats that share no sentence with the test set. Two runs: v1 (11 tactics, 800 chats) and v2 (14 tactics, 1,000 chats, ~1.23M tokens), all compared on the same 70 test chats:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th&gt;Qwen3-8B base&lt;/th&gt;
&lt;th&gt;LoRA v1&lt;/th&gt;
&lt;th&gt;LoRA v2 (online demo)&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Answers that follow the app's full schema (tactic id, message index, quote, summary)&lt;/td&gt;
&lt;td&gt;11 / 70&lt;/td&gt;
&lt;td&gt;70 / 70&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;70 / 70&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Risky chats flagged (unusable answers never count as catches)&lt;/td&gt;
&lt;td&gt;0 / 36 (none of the 36 was usable)&lt;/td&gt;
&lt;td&gt;36 / 36&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;36 / 36&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;False alarms on safe chats&lt;/td&gt;
&lt;td&gt;0 / 11 valid (23 invalid)&lt;/td&gt;
&lt;td&gt;3 / 34&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;2 / 34&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Signal precision / recall&lt;/td&gt;
&lt;td&gt;n/a (no valid risky answer)&lt;/td&gt;
&lt;td&gt;0.84 / 0.79&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;0.88 / 0.83&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Avg time per check&lt;/td&gt;
&lt;td&gt;3.0 s&lt;/td&gt;
&lt;td&gt;5.0 s&lt;/td&gt;
&lt;td&gt;4.5 s&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Latency values are the aggregate figures stored in the current result JSON files. Older run logs report different timings; per-case timings were not retained, so these averages cannot be independently recomputed.&lt;/p&gt;

&lt;p&gt;Strict format conformity (&lt;code&gt;valid_schema&lt;/code&gt;) is reported separately from usable coverage. The common postprocessor retains valid evidence in a partial answer; an unusable answer is an error, never a catch or a correct safe result. These saved Tinker runs contain no partial answers, so strict-schema and usable-coverage totals coincide: 11/70 for base and 70/70 for each LoRA. The base model often renames fields (&lt;code&gt;signal_id&lt;/code&gt; instead of &lt;code&gt;id&lt;/code&gt;). Read leniently, it flagged 35 of the 35 risky chats it answered readably, so it often &lt;em&gt;sees&lt;/em&gt; the pattern; its unrecognized fields made those answers unusable under the original schema. Fine-tuning produced schema-valid answers on all 70 development chats and found more signals, including the three newer tactics in v2. The online demo runs v2. Two honest limits: the false-alarm gap between v1 and v2 (3 vs 2 of 34) is a single chat, and the LoRA is not faster than the base model; this is a measured format improvement on the evaluated cases, not a guarantee for future inputs. Every raw answer is saved per case in the repo, so anyone can recount.&lt;/p&gt;

&lt;h3&gt;
  
  
  Temporal: an evaluation that survives a crashing laptop
&lt;/h3&gt;

&lt;p&gt;Evaluating a local model on a laptop is fragile. The evaluation runs as a &lt;strong&gt;Temporal&lt;/strong&gt; workflow where each conversation is an activity with automatic retries. In a 60-conversation crash test, Ollama was stopped during the run. The saved &lt;code&gt;eval/temporal_report.json&lt;/code&gt; records &lt;code&gt;s12&lt;/code&gt; completing on &lt;strong&gt;attempt 5&lt;/strong&gt;. The failed activity was retried when the model returned; completed activities remained in the workflow history.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F70ngeesrrf9c7v5fvbyt.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F70ngeesrrf9c7v5fvbyt.jpg" alt="Temporal workflow completed after the crash test" width="800" height="350"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  ElevenLabs: a calm voice for kids who don't read fast yet
&lt;/h3&gt;

&lt;p&gt;Younger kids and kids with dyslexia may not read a warning in time. Every child warning (14 tactics × English and Portuguese) was voiced once with &lt;strong&gt;ElevenLabs&lt;/strong&gt; (Sarah voice, &lt;code&gt;eleven_multilingual_v2&lt;/code&gt;) and shipped as audio files. In the live game demo and in the screen guard popup, the guard reads the most serious new warning aloud, and it speaks again if the stranger insists with a new message. Only the fixed warning texts were ever sent to ElevenLabs. No chat, ever.&lt;/p&gt;

&lt;h3&gt;
  
  
  Sentry: tracing the agent without reading the chats
&lt;/h3&gt;

&lt;p&gt;When configured, &lt;strong&gt;Sentry&lt;/strong&gt; traces each check as an agent run (&lt;code&gt;night-guard.check&lt;/code&gt;) with a model call inside (&lt;code&gt;gen_ai.chat&lt;/code&gt;) and the evidence step after it. That's how I saw that a cold Gemma call takes up to 8 s and a warm one about 4.5 s on this laptop. The traces carry only numbers and labels: model, backend, latency, number of messages, severity and tactic ids. Never a message, a name or a quote: &lt;code&gt;send_default_pii&lt;/code&gt; is off, local variables are not captured, and every event goes through an &lt;strong&gt;allowlist&lt;/strong&gt;: only fields known to hold numbers and labels are kept, everything else (request bodies, breadcrumbs, exception text, any new field) is dropped. A test pushes a fake event full of a marker string and checks the marker never survives. The UI's status polling is sampled out so the agent runs stand out.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fsa9e8aqt5bvynrsp7iw1.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fsa9e8aqt5bvynrsp7iw1.jpg" alt="Sentry traces: each night-guard.check with its gen_ai.chat model call" width="800" height="413"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Render: one click for the judges
&lt;/h3&gt;

&lt;p&gt;The public demo runs on &lt;strong&gt;Render&lt;/strong&gt; (Blueprint &lt;code&gt;render.yaml&lt;/code&gt;, free instance), with the Tinker fine-tuned model as its AI backend. The app does not persist chats on the demo server: family rules and alerts stay in each visitor's own browser, and the screen says to use fictional examples only. This describes application behavior, not the retention policies of Render or the model provider.&lt;/p&gt;

&lt;h3&gt;
  
  
  Browserbase + SerpApi: check a link without opening it on the child's computer
&lt;/h3&gt;

&lt;p&gt;When a conversation contains a link, &lt;strong&gt;"Check without opening"&lt;/strong&gt; uses Browserbase to open it in an isolated cloud&lt;br&gt;
browser. The inspector reads the final destination, checks for password fields and returns a screenshot.&lt;br&gt;
SerpApi separately searches for reports about the domain. The selected URL goes to Browserbase and the domain&lt;br&gt;
to SerpApi; the surrounding conversation is never sent. A URL can itself contain personal information.&lt;br&gt;
Each provider is optional, and the result identifies the checks that ran. Missing reports never mean "safe".&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Live local evidence, October 23, 2026:&lt;/strong&gt; one Browserbase session opened the fictional BlockQuest "free gems"&lt;br&gt;
page (session creation HTTP 201, page HTTP 200). It detected the password field, captured a screenshot and&lt;br&gt;
returned the Portuguese warning &lt;strong&gt;"Não abra esse link"&lt;/strong&gt; ("Don't open this link") in 12.68 seconds. Session release&lt;br&gt;
returned HTTP 200. A separate, single SerpApi query for &lt;code&gt;night-guard-demo.onrender.com&lt;/code&gt; returned HTTP 200 and six&lt;br&gt;
search results, with none matching the report heuristic. The fictional page submits no credentials.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fj2u640k7vzleczcj6seb.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fj2u640k7vzleczcj6seb.jpg" alt="The deployed inspector's Browserbase screenshot of the complete fictional BlockQuest login form" width="800" height="500"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The deployed Render endpoint also passed a live check after the viewport correction:&lt;/strong&gt; &lt;code&gt;POST /api/link/check&lt;/code&gt;&lt;br&gt;
returned HTTP 200, identified both Browserbase and SerpApi as checked, detected the password field and returned&lt;br&gt;
the same warning in 8.90 s. Its domain search returned six results, with none matching the report heuristic.&lt;br&gt;
The returned screenshot was visually verified to show the complete fictional login form; that public-endpoint&lt;br&gt;
screenshot is shown above.&lt;/p&gt;

&lt;p&gt;These checks establish provider connectivity and the observed password detection on one fictional example,&lt;br&gt;
not scam-detection accuracy.&lt;/p&gt;

&lt;h3&gt;
  
  
  Optional Auth0 sign-in for local Family controls
&lt;/h3&gt;

&lt;p&gt;When configured, the local Ollama app uses &lt;strong&gt;Auth0&lt;/strong&gt; to sign parents in before they can read family alerts, change rules or stop&lt;br&gt;
the guard. The email must be verified and present in the family's &lt;code&gt;PARENT_EMAILS&lt;/code&gt; list. It requires Auth0&lt;br&gt;
application settings and a session secret; without configuration, the local Family view has no login gate.&lt;br&gt;
The public demo does not use Auth0.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Live local evidence:&lt;/strong&gt; on a separate test server at port 8766, the alerts endpoint returned HTTP 401 before&lt;br&gt;
login, and the login route redirected with HTTP 302. A real Google sign-in with a verified, allowlisted parent&lt;br&gt;
completed the callback and the Family view displayed the signed-in parent. Logout returned home and restored&lt;br&gt;
the locked Parents view and sign-in button. Rejection of unverified or unlisted accounts is unit-tested; a real&lt;br&gt;
rejected-account attempt has not been performed. Local servers need restarting after &lt;code&gt;.env&lt;/code&gt; changes.&lt;/p&gt;

&lt;h3&gt;
  
  
  Backboard: compare open-weight models through one API key
&lt;/h3&gt;

&lt;p&gt;On October 23, 2026, &lt;strong&gt;Backboard&lt;/strong&gt; served three complete evaluations: Gemma 3 27B, Llama 3.3 70B and Mistral&lt;br&gt;
Small 3.1 24B, each on the same 70 fictional development conversations (36 risky, 34 safe). All three received&lt;br&gt;
the shared classification &lt;code&gt;SYSTEM&lt;/code&gt; and the canonical app &lt;code&gt;SCHEMA&lt;/code&gt; explicitly in the user prompt, with JSON mode,&lt;br&gt;
memory off, a fresh conversation per case and unchanged app postprocessing/scoring. An initial smoke returned&lt;br&gt;
&lt;code&gt;{}&lt;/code&gt; when only JSON mode was requested; adding the same format schema for every model fixed that missing&lt;br&gt;
instruction. No gold labels were added to prompts. This is a common Backboard adapter, not the identical&lt;br&gt;
prompt/decoding setup used by the older Tinker baseline or schema-constrained Ollama.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Model&lt;/th&gt;
&lt;th&gt;Strict schema&lt;/th&gt;
&lt;th&gt;Usable coverage&lt;/th&gt;
&lt;th&gt;Risky chats flagged&lt;/th&gt;
&lt;th&gt;False alarms on valid safe cases&lt;/th&gt;
&lt;th&gt;Signal precision&lt;/th&gt;
&lt;th&gt;Signal recall&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Gemma 3 27B&lt;/td&gt;
&lt;td&gt;68/70&lt;/td&gt;
&lt;td&gt;69/70&lt;/td&gt;
&lt;td&gt;36/36&lt;/td&gt;
&lt;td&gt;3/33 (1 error)&lt;/td&gt;
&lt;td&gt;0.938&lt;/td&gt;
&lt;td&gt;0.849&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Llama 3.3 70B&lt;/td&gt;
&lt;td&gt;69/70&lt;/td&gt;
&lt;td&gt;69/70&lt;/td&gt;
&lt;td&gt;36/36&lt;/td&gt;
&lt;td&gt;3/33 (1 error)&lt;/td&gt;
&lt;td&gt;0.912&lt;/td&gt;
&lt;td&gt;0.981&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Mistral Small 3.1 24B&lt;/td&gt;
&lt;td&gt;70/70&lt;/td&gt;
&lt;td&gt;70/70&lt;/td&gt;
&lt;td&gt;36/36&lt;/td&gt;
&lt;td&gt;3/34&lt;/td&gt;
&lt;td&gt;0.852&lt;/td&gt;
&lt;td&gt;0.981&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Qwen 2.5 72B&lt;/td&gt;
&lt;td&gt;n/a&lt;/td&gt;
&lt;td&gt;n/a&lt;/td&gt;
&lt;td&gt;n/a&lt;/td&gt;
&lt;td&gt;n/a&lt;/td&gt;
&lt;td&gt;Unavailable in live smoke&lt;/td&gt;
&lt;td&gt;No full result&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Strict-schema validity and usable coverage are separate. The shared scorer accepts a &lt;strong&gt;partial&lt;/strong&gt; analysis when&lt;br&gt;
postprocessing retains valid evidence; an unusable analysis is an error, never a catch or a correct safe answer.&lt;br&gt;
Gemma's &lt;code&gt;r06&lt;/code&gt; contained two valid personal-information items and one malformed empty item: the empty item was&lt;br&gt;
discarded and the partial analysis remained usable. Its safe case &lt;code&gt;s33&lt;/code&gt; was unusable. Llama had one unusable safe&lt;br&gt;
case. These failures remain in the denominators and the saved raw answers; nothing was retried to improve scores.&lt;/p&gt;

&lt;p&gt;The practical difference here is format reliability and signal precision/recall, not a demonstrated safety&lt;br&gt;
advantage: all three flagged the 36 risky development cases and raised three false alarms. Mistral followed&lt;br&gt;
the strict schema on every case; Gemma had higher signal precision but lower recall. This is a single run on&lt;br&gt;
reused development cases, not a new blind holdout or proof of protection for real children. Request timings&lt;br&gt;
include shared client slots and waiting, so they are not compared with local Gemma or Tinker latency.&lt;/p&gt;

&lt;p&gt;Qwen 2.5 72B's smoke request hit a &lt;strong&gt;120.65-second read timeout&lt;/strong&gt;. An already-started automatic retry was canceled&lt;br&gt;
locally when the other models were moved to a shared four-request limit. Both Qwen requests appear in billing,&lt;br&gt;
but no answer was observed locally and no 70-case Qwen result is claimed. The model was not silently replaced.&lt;/p&gt;

&lt;p&gt;Backboard's usage API recorded &lt;strong&gt;USD 0.034062 across 216 events&lt;/strong&gt;, including 210 scored calls, three successful&lt;br&gt;
smokes, the initial format-debugging call and two Qwen requests. A conservative token-price/reservation budget&lt;br&gt;
tracked USD 0.065078 against the internal USD 1 cap; that estimate is separate from actual billing. Only&lt;br&gt;
fictional conversations were sent. The &lt;a href="https://github.com/jenniferevangelista-de/night-guard/blob/main/train/backboard_summary.md" rel="noopener noreferrer"&gt;raw results and summary&lt;/a&gt;&lt;br&gt;
preserve per-case outputs, errors, coverage and signal metrics.&lt;/p&gt;

&lt;h3&gt;
  
  
  What the family gets
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Family rules:&lt;/strong&gt; for each tactic, "Tell family" or "Child decides". Photo requests, meeting in person, password requests, secrecy, coming back after a block and sextortion go to the family by default. The kid is always told when that happens: &lt;em&gt;"An alert was recorded for your family. Tell a trusted adult; this is not a delivery confirmation. You're not in trouble."&lt;/em&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The gap no platform covers:&lt;/strong&gt; Instagram supervision, Discord Family Center, TikTok Family Pairing and Roblox parental controls each protect their own app. The classic path starts in a game and moves to a private app. Night Guard points out a possible match: &lt;em&gt;"Same name seen in more than one app (possibly the same person): Alex_17 (Roblox → Instagram)"&lt;/em&gt;. It's a hint to check, not proof, because names can repeat.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;What to do now&lt;/strong&gt;, by severity: talk calmly and without punishment (kids who get in trouble stop telling), keep screenshots, block and report, and for image requests, threats or money demands, report to SaferNet Brasil or the NCMEC CyberTipline and the police. Don't pay.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Phone alerts&lt;/strong&gt; through ntfy (free, no account) with only the risk level, the app and the kind of risk.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;"Tell someone":&lt;/strong&gt; the kid picks what to share and gets a ready message for Mom, Dad or a teacher. It is assembled deterministically from their own words and the flagged quotes.&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  Limits and next step
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Phones are the next milestone.&lt;/strong&gt; Today Night Guard protects a Windows computer. On &lt;strong&gt;Android&lt;/strong&gt;, the plan is an app that reads message notifications (the permission parental tools use there) and runs the detector on-device. On &lt;strong&gt;iPhone&lt;/strong&gt;, Apple doesn't let any app read other apps' messages, so the realistic path is Apple's own Communication Safety.&lt;/li&gt;
&lt;li&gt;It's a learning and support tool, not a guarantee of safety. It can miss things and raise false alarms. It doesn't replace trusted adults or platform reporting.&lt;/li&gt;
&lt;li&gt;All test data is fictional. No real children's conversations were used anywhere.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Why Does Open Innovation Matter?
&lt;/h2&gt;

&lt;p&gt;Child safety is the clearest case for open models I know. A guard that reads a kid's private chats should &lt;strong&gt;run on the family's own computer&lt;/strong&gt;, be &lt;strong&gt;inspectable&lt;/strong&gt;, and be &lt;strong&gt;free&lt;/strong&gt;. For comparison, &lt;a href="https://support.bark.us/en/articles/13460996-pricing" rel="noopener noreferrer"&gt;Bark Premium&lt;/a&gt; lists US$14 a month before tax. Platforms protect only their own walls. Roblox &lt;a href="https://about.roblox.com/newsroom/2025/08/open-sourcing-roblox-sentinel-preemptive-risk-detection" rel="noopener noreferrer"&gt;open-sourced Sentinel in 2025&lt;/a&gt; and &lt;a href="https://about.roblox.com/newsroom/2026/08/roblox-open-source-safety-models-roost" rel="noopener noreferrer"&gt;announced an update and ROOST contribution in 2026&lt;/a&gt;, which is useful for platforms; Night Guard is the other half: an open, local tool that sits with the family and works across apps.&lt;/p&gt;

&lt;p&gt;Open weights made every part of this possible: Gemma reading text and screenshots on a laptop GPU, a Qwen LoRA fine-tuned in an afternoon, and anyone able to audit exactly which 14 tactics trigger a warning and what words a kid will see.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Lazy Payoff
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Parents&lt;/strong&gt; can review flagged events with suggested next steps, and stay the ones who talk and decide. Reduced review effort is an intended benefit, not a measured result.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Kids&lt;/strong&gt; get short explanations and practice choices, and can ask an adult for help. Progress in the app does not establish readiness for future situations.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The guard&lt;/strong&gt; can repeat fewer explanations after completed practice, while leaving every explanation available to reopen. Completing an exercise does not prove future safety and never switches protection off.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Prize Categories
&lt;/h2&gt;

&lt;p&gt;Each row: what the category asks for, what I built, and the section that shows it.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Category&lt;/th&gt;
&lt;th&gt;What it asks&lt;/th&gt;
&lt;th&gt;What I built&lt;/th&gt;
&lt;th&gt;Where to see it&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Gemma&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Use Gemma: run it locally, fine-tune it or serve it&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;gemma4:e4b&lt;/code&gt; on the family's computer reads the chat and, through screenshots, the game window&lt;/td&gt;
&lt;td&gt;The detector: Gemma 4; Proactive&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Tinker&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Fine-tune a model and show a clear improvement over a baseline&lt;/td&gt;
&lt;td&gt;Qwen3-8B LoRA: 70/70 answers in the app's schema, against 11/70 for the base model&lt;/td&gt;
&lt;td&gt;Fine-tuning with Tinker&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Render&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Use Render as the AI runtime or to host the front end&lt;/td&gt;
&lt;td&gt;The public demo runs the fine-tuned model, deployed from a Blueprint&lt;/td&gt;
&lt;td&gt;Render: one click for the judges&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Temporal&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Make an agent durable: survive failures and retry&lt;/td&gt;
&lt;td&gt;The evaluation runs as a workflow; stopping the model mid-run lost nothing&lt;/td&gt;
&lt;td&gt;Temporal: an evaluation that survives a crashing laptop&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;ElevenLabs&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Give an open-source agent a voice&lt;/td&gt;
&lt;td&gt;Spoken child warnings in English and Portuguese, shipped as audio files&lt;/td&gt;
&lt;td&gt;ElevenLabs: a calm voice for kids&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Sentry Agent Tracing&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Show the agent's work with traces or screenshots&lt;/td&gt;
&lt;td&gt;Agent runs and model calls traced, with an allowlist so no chat text is sent&lt;/td&gt;
&lt;td&gt;Sentry: tracing the agent without reading the chats&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Browserbase&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Give an agent a real browser&lt;/td&gt;
&lt;td&gt;The link inspector opens a link in an isolated cloud browser (Playwright over CDP, not Stagehand) and finds password fields&lt;/td&gt;
&lt;td&gt;Browserbase + SerpApi&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;SerpApi&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Give an agent live web search&lt;/td&gt;
&lt;td&gt;The same inspector searches reports about the link's domain; no reports is never read as "safe"&lt;/td&gt;
&lt;td&gt;Browserbase + SerpApi&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Auth0 for AI Agents&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Sign users in to an agent&lt;/td&gt;
&lt;td&gt;Verified, allowlisted Google login for the Family controls. Local app only, not the public demo&lt;/td&gt;
&lt;td&gt;Optional Auth0 sign-in&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Backboard&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Compare open-weight models through one API key&lt;/td&gt;
&lt;td&gt;Three models on the same 70 fictional chats through one key. A Qwen timeout is disclosed&lt;/td&gt;
&lt;td&gt;Backboard: compare open-weight models&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Overall prize&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Writing, theme, creativity, technical execution&lt;/td&gt;
&lt;td&gt;A friend for the kid, a night guard for the parents; a failure is shown as "couldn't analyze", never "all clear"&lt;/td&gt;
&lt;td&gt;The whole post&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;These entries describe the implemented uses and their observed evidence. Final category eligibility is for&lt;br&gt;
the judges under the &lt;a href="https://dev.to/challenges/hf26"&gt;challenge criteria&lt;/a&gt;.&lt;/p&gt;

</description>
      <category>devchallenge</category>
      <category>hf26challenge</category>
      <category>ai</category>
      <category>opensource</category>
    </item>
    <item>
      <title>Your pipeline ran green ✅ and shipped last week’s data anyway</title>
      <dc:creator>Jennifer Evangelista</dc:creator>
      <pubDate>Fri, 23 Oct 2026 10:12:57 +0000</pubDate>
      <link>https://dev.to/jenniferevangelistade/your-pipeline-ran-green-and-shipped-last-weeks-data-anyway-1il3</link>
      <guid>https://dev.to/jenniferevangelistade/your-pipeline-ran-green-and-shipped-last-weeks-data-anyway-1il3</guid>
      <description>&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fwcckbg3n0vj3wji83qju.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fwcckbg3n0vj3wji83qju.png" width="800" height="450"&gt;&lt;/a&gt;&lt;br&gt;
&lt;em&gt;Image generated with AI&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Inspired by the “Is this a pigeon?” meme. The dates and scenario are illustrative.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;Your backfill finished at 07:03 this morning. It ran cleanly. Every dependency was green. But it was processing last Thursday’s data, and your gate just let it through as today’s load.&lt;/p&gt;

&lt;p&gt;A successful backfill can finish this morning and still belong to last week’s reporting period. If the downstream gate checks only whether its dependencies ran today, the wrong load can pass.&lt;/p&gt;

&lt;p&gt;I would make business date an explicit input and retain the exact upstream run IDs that authorized the output. The small SQLite lab accompanying this article demonstrates that contract, including a retry after publication and two concurrent publishers.&lt;/p&gt;
&lt;h3&gt;
  
  
  The clock answers a narrower question
&lt;/h3&gt;

&lt;p&gt;Imagine two independently scheduled dimension jobs and a daily sales load. The sales load needs both dimensions for October 23.&lt;/p&gt;

&lt;p&gt;At 07:00 UTC on October 23, both jobs complete successfully. A check based on today’s start time sees two green runs and opens the gate. But those runs were backfills for October 16. The current reporting period is still missing.&lt;/p&gt;

&lt;p&gt;The opposite case is just as useful: a valid October 23 run can start before midnight and complete later. A midnight cutoff can exclude work the consumer actually needs.&lt;/p&gt;

&lt;p&gt;In the Databricks Jobs API, start_time_from filters runs by when they started, which says nothing about the business period the code processed. The API also separates completed runs from successful ones. Both are good inputs for a collector, but neither tells you a run covered the right day.&lt;/p&gt;

&lt;p&gt;The fix is not a smarter clock. It is an explicit record of which business date each run actually processed, and a gate that checks that record instead of the wall clock.&lt;/p&gt;
&lt;h3&gt;
  
  
  Write down which attempt counts
&lt;/h3&gt;

&lt;p&gt;My example records job ID, business date, attempt ordinal, run ID, state and a timezone-aware start timestamp. It checks the exact required job IDs against the requested date.&lt;/p&gt;

&lt;p&gt;The attempt ordinal is assigned by the producer for each job/date pair. It’s kept separate from run ID and from when the observation arrived. A delayed message about attempt one must not overwrite the outcome of attempt two.&lt;/p&gt;

&lt;p&gt;I chose a conservative rule: the greatest recorded attempt wins. If attempt 1 succeeds but attempt 2 fails or is still running, the gate blocks: the earlier success is invisible until the newer attempt resolves. Only when the most recent attempt succeeds does the date become ready. Another system might accept the first success instead. That’s a different contract, with its own tests.&lt;/p&gt;

&lt;p&gt;A running observation may transition to one terminal result. Replaying that result is harmless. Changing an already terminal result is rejected; a repair must carry a new attempt ordinal.&lt;/p&gt;

&lt;p&gt;This policy still depends on the collector. If the gate has not learned that a newer attempt exists, it cannot block on that invisible attempt. In a deployment, I would define who publishes authoritative observations and how missing or delayed updates are detected.&lt;/p&gt;
&lt;h3&gt;
  
  
  An executable failure, then a valid release
&lt;/h3&gt;

&lt;p&gt;The lab uses two synthetic dependencies and a synthetic total of 17,500 integer cents. Its demo first records successful October 16 backfills with October 23 start timestamps. Checking October 23 returns false. Recording successful October 23 attempts makes that date ready.&lt;/p&gt;

&lt;p&gt;Run these commands from the repository root:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;python -m unittest discover -s python/business_date_gate -p &lt;span class="s1"&gt;'test_*.py'&lt;/span&gt; -v
python python/business_date_gate/gate.py
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The demo prints:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Backfill completed today: False
Required business date ready: True
First publication: True
Same publication retried: False
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That last False is the retry being ignored: the existing publication stays and nothing gets written twice.&lt;/p&gt;

&lt;h3&gt;
  
  
  Put the output and its evidence in one transaction
&lt;/h3&gt;

&lt;p&gt;A readiness check alone does not prevent two workers from publishing the same date. In this local example, the publication key is the consumer name plus business date.&lt;/p&gt;

&lt;p&gt;The code opens a SQLite BEGIN IMMEDIATE transaction, reads readiness, writes the target total and stores the successful upstream run IDs. Both inserts commit together. A repeated call with the same key, dependency set and amount returns without inserting another row. A different amount or dependency set is rejected so a correction cannot silently masquerade as a retry.&lt;/p&gt;

&lt;p&gt;After publication, an upstream failure does not erase the original evidence. Replaying the already committed result remains a no-op. Rebuilding or revoking that result would require an explicit repair workflow.&lt;/p&gt;

&lt;p&gt;The failure test matters here. A SQLite trigger rejects the evidence insert on purpose, right after the target insert. The transaction rolls back both writes. Removing the trigger and retrying succeeds. A separate test launches two publishers on independent connections to the same temporary database; only one creates the publication.&lt;/p&gt;

&lt;p&gt;All 25 new tests passed locally with Python 3.12 and SQLite. They also cover date validation, midnight crossings, missing jobs, delayed observations, state transitions and conflicting retries.&lt;/p&gt;

&lt;h3&gt;
  
  
  What the lab doesn’t cover
&lt;/h3&gt;

&lt;p&gt;Output and evidence live in one SQLite database, so the atomic part holds there and nowhere else. Delta, S3 or a message broker need their own commit checks. The total is illustrative too; the gate doesn’t read upstream tables.&lt;/p&gt;

&lt;p&gt;On Databricks, I’d pass business date as a job parameter and let a table-update trigger on a control table start the check. The trigger only decides when to look. The date and attempt rules above still decide if the output goes out.&lt;/p&gt;

&lt;p&gt;Before deploying, I’d test the collector’s pagination, retries and repair handling, and map the real run states explicitly.&lt;/p&gt;

&lt;p&gt;Code: &lt;a href="https://github.com/jenniferevangelista-de/data-engineering-lab/tree/e3ddbf036417f64d4e2e8b1058df15780b935820/python/business_date_gate" rel="noopener noreferrer"&gt;https://github.com/jenniferevangelista-de/data-engineering-lab/tree/e3ddbf036417f64d4e2e8b1058df15780b935820/python/business_date_gate&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;References:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Databricks run-list filters: &lt;a href="https://docs.databricks.com/api/jobs/v2/list-runs" rel="noopener noreferrer"&gt;https://docs.databricks.com/api/jobs/v2/list-runs&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;Job parameters: &lt;a href="https://docs.databricks.com/aws/en/jobs/job-parameters" rel="noopener noreferrer"&gt;https://docs.databricks.com/aws/en/jobs/job-parameters&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;Table-update triggers: &lt;a href="https://docs.databricks.com/aws/en/jobs/trigger-table-update" rel="noopener noreferrer"&gt;https://docs.databricks.com/aws/en/jobs/trigger-table-update&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;SQLite transactions: &lt;a href="https://www.sqlite.org/lang_transaction.html" rel="noopener noreferrer"&gt;https://www.sqlite.org/lang_transaction.html&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>python</category>
      <category>databricks</category>
      <category>dataengineering</category>
      <category>dataquality</category>
    </item>
    <item>
      <title>A Freshness Check Before the 8 a.m. Report</title>
      <dc:creator>Jennifer Evangelista</dc:creator>
      <pubDate>Wed, 21 Oct 2026 07:48:57 +0000</pubDate>
      <link>https://dev.to/jenniferevangelistade/a-freshness-check-before-the-8-am-report-3g2n</link>
      <guid>https://dev.to/jenniferevangelistade/a-freshness-check-before-the-8-am-report-3g2n</guid>
      <description>&lt;p&gt;Check the output interval and event timestamps before the report becomes available to consumers.&lt;/p&gt;

&lt;p&gt;The dashboard says the overnight pipeline succeeded. The BI report opened at 8:00 a.m. People are already discussing the morning numbers.&lt;/p&gt;

&lt;p&gt;Someone from operations asks why yesterday’s transactions are still the newest records.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F4nn16c1tsvq6b11wntu9.jpeg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F4nn16c1tsvq6b11wntu9.jpeg" width="800" height="443"&gt;&lt;/a&gt;&lt;br&gt;
&lt;em&gt;Pipeline finished successfully.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;In this example, every expected task succeeded. The pipeline copied an old snapshot into a valid new file, and no task checked its event timestamps. I would put that check at the boundary where the report becomes available to consumers.&lt;/p&gt;
&lt;h3&gt;
  
  
  Three clocks, three different questions
&lt;/h3&gt;

&lt;p&gt;Teams often compress distinct timestamps into a single idea of “latest”:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The run clock:&lt;/strong&gt; when the orchestration system started and finished a run. This helps answer whether the workflow executed.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The interval clock:&lt;/strong&gt; the logical period of data assigned to the run. In Airflow, a time-scheduled run has a data interval; its logical date commonly denotes that interval’s start, &lt;strong&gt;not&lt;/strong&gt; the moment the tasks physically execute. A run for Monday’s data may legitimately execute after Monday has ended.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The event clock:&lt;/strong&gt; the newest business event actually represented in the output. If an hourly orders dataset finishes at 8:00 but its newest order_created_at is 5:00, you have a freshness question even if the DAG status says success.&lt;/p&gt;

&lt;p&gt;The three clocks can agree, disagree, or be incomparable depending on what the source produces. Never use a scheduler timestamp as a substitute for inspecting output events.&lt;/p&gt;
&lt;h3&gt;
  
  
  How a green DAG can deliver old data
&lt;/h3&gt;

&lt;p&gt;Consider an hourly DAG that extracts orders, transforms them, and writes a report table. A source API is delayed but still returns a well-formed response. The transform accepts the response. The write succeeds. A terminal task sends the success notification. Yesterday’s data has completed today’s commute.&lt;/p&gt;

&lt;p&gt;Nothing in that story proves that the output contains events within the business SLA.&lt;/p&gt;

&lt;p&gt;There is another nuance: Airflow’s DAG-run status is derived from task states, including leaf tasks. Its documentation warns that certain trigger-rule arrangements can produce an apparently successful DAG run even when an intermediate task failed. A status light is useful, but it is not an end-to-end assertion about the delivered dataset.&lt;/p&gt;

&lt;p&gt;This is why I do not recommend solving freshness incidents by simply adding more retries. Retrying a successful extraction of an old snapshot may give you the same old snapshot again.&lt;/p&gt;
&lt;h3&gt;
  
  
  Define freshness from the consumer backward
&lt;/h3&gt;

&lt;p&gt;For this hourly orders example, I would agree on the following contract with the consumer:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;em&gt;The hourly orders dataset must contain at least one valid row, must be produced for the required interval, and must contain business event timestamps no older than 90 minutes at the 8:00 reporting checkpoint.&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;The 90-minute number is a &lt;strong&gt;made-up example SLA&lt;/strong&gt; , not a recommendation for every company. For a dataset that receives no orders on quiet days, a mandatory recent event may be an invalid contract; use source heartbeat, completeness markers, or expected arrival windows instead. For batch snapshots, partition publication time may be more informative than event time. The right contract reflects producer behavior and consumer expectations.&lt;/p&gt;

&lt;p&gt;Before marking a dataset ready, I want answers to five specific questions: Did the job publish the required data interval, or did it succeed while reprocessing an older one? Is the newest event recent enough for this consumer? Is the expected partition or count present, or are missing records being silently tolerated? Did the agreed validation checks pass before the table became visible downstream? Are timestamps timezone-aware, is event time distinct from ingestion time, and is clock skew bounded?&lt;/p&gt;

&lt;p&gt;These are properties of the output and its contract, not properties of the colored dot next to a task.&lt;/p&gt;
&lt;h3&gt;
  
  
  Fail at the publication boundary
&lt;/h3&gt;

&lt;p&gt;The companion lab contains a small Python contract that demonstrates the idea without requiring Airflow, Spark, or cloud credentials:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://github.com/jenniferevangelista-de/data-engineering-lab/tree/main/airflow/day_001_freshness_contract" rel="noopener noreferrer"&gt;&lt;strong&gt;Open the freshness-contract lab and tests on GitHub&lt;/strong&gt;&lt;/a&gt; &lt;strong&gt;.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The core function accepts a snapshot description, the current UTC instant, the required interval end, and a maximum event age. It returns specific breaches for an old interval, stale events, empty output, failed quality checks, or implausibly future-dated events. A strict wrapper raises an error so the publishing task can fail closed rather than completing with bad data.&lt;/p&gt;

&lt;p&gt;For example, the synthetic case is:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;datetime&lt;/span&gt; &lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;datetime&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;timedelta&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;timezone&lt;/span&gt;
&lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;freshness_contract&lt;/span&gt; &lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="nc"&gt;Snapshot&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;assess_snapshot&lt;/span&gt;

&lt;span class="n"&gt;t&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;datetime&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;2026&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;10&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;20&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;8&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;tzinfo&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;timezone&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;utc&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;snapshot&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Snapshot&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="n"&gt;dataset&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"orders_hourly"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;dag_state&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"success"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;data_interval_end&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;t&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;latest_event_at&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;t&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="nf"&gt;timedelta&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;hours&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;row_count&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;100&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;quality_checks_passed&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="kc"&gt;True&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nf"&gt;assess_snapshot&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="n"&gt;snapshot&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;now&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;t&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;required_interval_end&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;t&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;max_age&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="nf"&gt;timedelta&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;minutes&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;90&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="c1"&gt;# ('event timestamp too old',)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The output reports event timestamp too old: the latest event is 120 minutes old against the example limit of 90 minutes, even though dag_state is success.&lt;/p&gt;

&lt;p&gt;Run the standalone tests with Python 3.12+:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;python -m unittest discover -s airflow/day_001_freshness_contract -p &lt;span class="s1"&gt;'test_*.py'&lt;/span&gt; -v
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The tests cover stale-but-green output, the wrong interval, missing timestamps, empty output, quality failures, invalid timezone handling, clock skew and a healthy dataset.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Scope:&lt;/strong&gt; this is an original pure-Python illustration of a guard. It is &lt;strong&gt;not an installed Airflow operator&lt;/strong&gt; and has not been exercised against an Airflow scheduler, Databricks, cloud storage, or production data. To deploy it, fetch real metadata from the produced dataset and invoke the strict check before marking the artifact ready for downstream use.&lt;/p&gt;

&lt;h3&gt;
  
  
  What about asset-aware orchestration?
&lt;/h3&gt;

&lt;p&gt;Airflow 3 includes asset-aware scheduling, which is valuable when downstream workflows should respond to upstream asset events instead of an arbitrary timer. For task-declared outlets, Airflow marks an asset as updated when the producer task completes successfully. Externally emitted asset events need their own producer-side validation. Neither path inherently certifies every business-quality dimension of the data. Put the contract check before the event is emitted, or make consumers verify the contract independently.&lt;/p&gt;

&lt;p&gt;Asset-aware scheduling solves &lt;strong&gt;when to trigger&lt;/strong&gt;. Data contracts answer &lt;strong&gt;what is safe to consume&lt;/strong&gt;. They complement each other; they are not substitutes.&lt;/p&gt;

&lt;h3&gt;
  
  
  The incident-review questions I would ask
&lt;/h3&gt;

&lt;p&gt;Did the source publish new business events? What is the newest event timestamp and what does that field actually represent? Which interval did the job write? Can we distinguish a source outage from a legitimate zero-activity period? Was the output made visible only after validation passed? What alert goes to the owner when the contract is breached?&lt;/p&gt;

&lt;p&gt;Those questions lead to an operational action. “Make the DAG green” usually leads to another green DAG.&lt;/p&gt;

&lt;p&gt;If I had to choose one dashboard tile for an executive review, I would show &lt;strong&gt;data availability against the consumer’s agreed freshness contract&lt;/strong&gt; , backed by evidence, alongside workflow status. The scheduler should remain visible. It should not be the sole source of truth.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;A green DAG means the orchestration ran. A trustworthy dataset means the data met its promises.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;a href="https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/dag-run.html" rel="noopener noreferrer"&gt;Apache Airflow DAG runs: status, logical date, and data intervals&lt;/a&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://airflow.apache.org/docs/apache-airflow/stable/authoring-and-scheduling/asset-scheduling.html" rel="noopener noreferrer"&gt;Apache Airflow asset-aware scheduling&lt;/a&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://github.com/jenniferevangelista-de/data-engineering-lab/tree/main/airflow/day_001_freshness_contract" rel="noopener noreferrer"&gt;Reproducible freshness-contract lab&lt;/a&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Foha5h9eui2goq8wpky7w.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Foha5h9eui2goq8wpky7w.png" width="800" height="450"&gt;&lt;/a&gt;&lt;br&gt;
&lt;em&gt;AI-generated illustration of a successful workflow delivering stale data.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>apacheairflow</category>
      <category>dataengineering</category>
    </item>
    <item>
      <title>Three Orders In, Three Rows Out, 42.9% More Revenue</title>
      <dc:creator>Jennifer Evangelista</dc:creator>
      <pubDate>Wed, 21 Oct 2026 07:34:51 +0000</pubDate>
      <link>https://dev.to/jenniferevangelistade/three-orders-in-three-rows-out-429-more-revenue-cee</link>
      <guid>https://dev.to/jenniferevangelistade/three-orders-in-three-rows-out-429-more-revenue-cee</guid>
      <description>&lt;p&gt;&lt;em&gt;An INNER JOIN can lose orders and multiply others in the same query. Here is a reproducible test for the failure that aggregate dashboards often miss.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fafyacl2c2ltreu17mrv3.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fafyacl2c2ltreu17mrv3.png" width="800" height="450"&gt;&lt;/a&gt;&lt;br&gt;
&lt;em&gt;AI-generated illustration of duplicate matches and an unmatched order. Synthetic amounts in integer cents.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;A join can return the same number of rows as its input and still inflate revenue. In this synthetic example, three orders become three output rows, while the total rises from 17,500 to 25,000 cents: an increase of about 42.9%.&lt;/p&gt;

&lt;p&gt;The row-count check approves. Accounting would like a word.&lt;/p&gt;

&lt;p&gt;The aggregate count is unchanged because one duplicated order compensates for one missing order.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fzb0ntg83fbvlitqgwkbr.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fzb0ntg83fbvlitqgwkbr.png" width="800" height="400"&gt;&lt;/a&gt;&lt;br&gt;
&lt;em&gt;AI-generated comic about a row-count check that misses a revenue mismatch.&lt;/em&gt;&lt;/p&gt;
&lt;h3&gt;
  
  
  The join that cancels its own mistakes
&lt;/h3&gt;

&lt;p&gt;We have three synthetic orders, in integer cents:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;order customer amount_cents
o1 c1 10000
o2 c2 5000
o3 c3 2500
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The source total is &lt;strong&gt;17,500 cents&lt;/strong&gt;. Our customer profile table has two historical versions for c1, one for c2, and no profile for c3.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="n"&gt;o&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;order_id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;o&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;amount_cents&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;p&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;tier&lt;/span&gt;
&lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="n"&gt;orders&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;o&lt;/span&gt;
&lt;span class="k"&gt;INNER&lt;/span&gt; &lt;span class="k"&gt;JOIN&lt;/span&gt; &lt;span class="n"&gt;customer_profiles&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;p&lt;/span&gt;
  &lt;span class="k"&gt;ON&lt;/span&gt; &lt;span class="n"&gt;p&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;customer_id&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;o&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;customer_id&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;What happens?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;o1 matches two profile rows, so its 10,000 cents are counted twice.&lt;/li&gt;
&lt;li&gt;o2 matches once.&lt;/li&gt;
&lt;li&gt;o3 disappears because its customer has no profile.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The output is still three rows, but the sum is now 25,000 cents. We lost one fact row and duplicated another. A row-count assertion looks reassuring precisely because the two errors compensate numerically. The query assumes one matching profile per customer, but the data contains two matches for one customer and none for another.&lt;/p&gt;

&lt;h3&gt;
  
  
  Define the grain before writing the join
&lt;/h3&gt;

&lt;p&gt;The grain of the source is one row per order_id. The intended relationship is many orders to one &lt;em&gt;current&lt;/em&gt; customer profile. But the customer_profiles table actually stores multiple versions for at least one customer.&lt;/p&gt;

&lt;p&gt;An INNER JOIN only enforces its equality predicate. It does not invent a uniqueness constraint or choose “current” by intuition. A LEFT JOIN would preserve unmatched orders, but &lt;strong&gt;it would not prevent fanout&lt;/strong&gt; when two profiles match.&lt;/p&gt;

&lt;p&gt;Before joining, make the contract explicit:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Every order is associated with exactly one valid profile for the report’s business time.&lt;/li&gt;
&lt;li&gt;The right side contains no more than one profile row per join key &lt;em&gt;after applying that time or current-record rule&lt;/em&gt;.&lt;/li&gt;
&lt;li&gt;An unmatched order is an exception to investigate, not something silently removed.&lt;/li&gt;
&lt;li&gt;A revenue report reconciles both fact count &lt;strong&gt;and&lt;/strong&gt; monetary totals in integer cents.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;For a Type 2 dimension, the correct fix is normally a temporal rule, such as selecting the version effective when the order happened. Choosing MAX(profile_version) without understanding the business definition of “current” is not a neutral fix; it can rewrite history. If the contract truly needs the latest known state rather than an as-of state, document that choice.&lt;/p&gt;

&lt;h3&gt;
  
  
  Four checks that expose the issue
&lt;/h3&gt;

&lt;p&gt;&lt;strong&gt;Check 1: duplicate dimension keys at the join grain.&lt;/strong&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="n"&gt;customer_id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nb"&gt;COUNT&lt;/span&gt;&lt;span class="cm"&gt;(*) AS matches
FROM customer_profiles
GROUP BY customer_id
HAVING COUNT(*)&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This is intentionally conservative. In a history table, duplicates may be legitimate overall, but they are not legitimate &lt;strong&gt;in the dimension slice selected for a one-profile-per-customer join&lt;/strong&gt;. Filter or window the history according to the correct as-of rule first, then check that derived slice.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Check 2: unmatched orders.&lt;/strong&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="n"&gt;o&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;order_id&lt;/span&gt;
&lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="n"&gt;orders&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;o&lt;/span&gt;
&lt;span class="k"&gt;WHERE&lt;/span&gt; &lt;span class="k"&gt;NOT&lt;/span&gt; &lt;span class="k"&gt;EXISTS&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;
  &lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt; &lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="n"&gt;customer_profiles&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;p&lt;/span&gt;
  &lt;span class="k"&gt;WHERE&lt;/span&gt; &lt;span class="n"&gt;p&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;customer_id&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;o&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;customer_id&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Use this even if the row-count totals happen to match. If null business keys are allowed, specify their matching policy explicitly instead of relying on implicit SQL NULL behavior.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Check 3: reconcile totals before and after.&lt;/strong&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="nb"&gt;COUNT&lt;/span&gt;&lt;span class="cm"&gt;(*) AS row_count, SUM(amount_cents) AS cents FROM orders;

SELECT COUNT(*)&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;row_count&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nb"&gt;SUM&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;o&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;amount_cents&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;cents&lt;/span&gt;
&lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="n"&gt;orders&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;o&lt;/span&gt;
&lt;span class="k"&gt;INNER&lt;/span&gt; &lt;span class="k"&gt;JOIN&lt;/span&gt; &lt;span class="n"&gt;customer_profiles&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;p&lt;/span&gt;
  &lt;span class="k"&gt;ON&lt;/span&gt; &lt;span class="n"&gt;p&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;customer_id&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;o&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;customer_id&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;For this example the counts both equal three, but the amounts disagree: 17,500 versus 25,000 cents. Finance-grade pipelines should avoid floating-point money and define how refunds, cancellations, currency, and tax are represented.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Check 4: fail the contract, not only the SQL execution.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;A query that returns rows has not proved its relationship assumptions. Treat fanout, unmatched facts, and unreconciled totals as observable contract breaches. Fail the publication gate before an incorrect result enters an executive dashboard.&lt;/p&gt;

&lt;h3&gt;
  
  
  A small executable proof
&lt;/h3&gt;

&lt;p&gt;Spark join hints suggest execution strategies, such as broadcasting one side; they do not enforce one profile per customer. Keep the cardinality and reconciliation checks when porting this pattern, then evaluate the physical plan separately. The executable proof here remains SQLite-only.&lt;/p&gt;

&lt;p&gt;Spark JOIN reference: &lt;a href="https://spark.apache.org/docs/latest/sql-ref-syntax-qry-select-join.html" rel="noopener noreferrer"&gt;https://spark.apache.org/docs/latest/sql-ref-syntax-qry-select-join.html&lt;/a&gt;&lt;br&gt;&lt;br&gt;
Join hints: &lt;a href="https://spark.apache.org/docs/latest/sql-ref-syntax-qry-select-hints.html" rel="noopener noreferrer"&gt;https://spark.apache.org/docs/latest/sql-ref-syntax-qry-select-hints.html&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The public lab includes a dependency-free SQLite example and eight tests:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://github.com/jenniferevangelista-de/data-engineering-lab/tree/main/sql/day_002_join_cardinality" rel="noopener noreferrer"&gt;https://github.com/jenniferevangelista-de/data-engineering-lab/tree/main/sql/day_002_join_cardinality&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The example deliberately produces the deceptive equality of row counts, catches duplicate profile keys, catches orphan orders, compares totals, and refuses the publication contract. Tests also cover a healthy dimension, an empty input, and a refund represented as negative integer cents.&lt;/p&gt;

&lt;p&gt;Run it with Python 3.12+:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;python -m unittest discover -s sql/day_002_join_cardinality -p &lt;span class="s1"&gt;'test_*.py'&lt;/span&gt; -v
python sql/day_002_join_cardinality/join_guard.py
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Scope:&lt;/strong&gt; the tests run against SQLite with synthetic data. They do &lt;strong&gt;not&lt;/strong&gt; benchmark Spark, execute a Databricks job, or validate your production dimensional model. The behavioral principle is portable; query plans and enforcement mechanisms differ by engine.&lt;/p&gt;

&lt;h3&gt;
  
  
  What I would require in a production review
&lt;/h3&gt;

&lt;p&gt;A code review focused only on syntax and task status will not catch this class of incident. I would ask for a written grain, a documented as-of join rule, checks for right-side cardinality and unmatched keys, monetary reconciliation, and a failure path that blocks consumption when those checks fail.&lt;/p&gt;

&lt;p&gt;For an analytics system with late-arriving dimensions, the contract also needs a defined quarantine or remediation strategy. Do you defer the record, show it as “unknown,” or resolve it from an earlier version? A blanket INNER JOIN is not a policy.&lt;/p&gt;

&lt;p&gt;For this example, I would block the report until c1 has one profile selected by the agreed time rule and o3 has an explicit resolution. Then I would rerun the key, row-count, and money checks before making the result available.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;a href="https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-syntax-qry-select-join" rel="noopener noreferrer"&gt;https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-syntax-qry-select-join&lt;/a&gt; (INNER vs LEFT and join criteria).&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://www.postgresql.org/docs/current/queries-table-expressions.html" rel="noopener noreferrer"&gt;https://www.postgresql.org/docs/current/queries-table-expressions.html&lt;/a&gt; (relational join behavior).&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://github.com/jenniferevangelista-de/data-engineering-lab/tree/main/sql/day_002_join_cardinality" rel="noopener noreferrer"&gt;https://github.com/jenniferevangelista-de/data-engineering-lab/tree/main/sql/day_002_join_cardinality&lt;/a&gt; (synthetic example and tests).&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>sql</category>
      <category>dataengineering</category>
      <category>databricks</category>
    </item>
    <item>
      <title>Unity Catalog Row Filters and Column Masks: A Fail-Closed Security Design You Can Test</title>
      <dc:creator>Jennifer Evangelista</dc:creator>
      <pubDate>Mon, 19 Oct 2026 21:00:35 +0000</pubDate>
      <link>https://dev.to/jenniferevangelistade/unity-catalog-row-filters-and-column-masks-a-fail-closed-security-design-you-can-test-1a84</link>
      <guid>https://dev.to/jenniferevangelistade/unity-catalog-row-filters-and-column-masks-a-fail-closed-security-design-you-can-test-1a84</guid>
      <description>&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fazt8mbqupcdb2udjmjuz.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fazt8mbqupcdb2udjmjuz.png" alt="Two-panel comic: a woman analyst shows a pii-readers badge at the orders table entrance. A security guard holding a ROW FILTER clipboard blocks entry and asks for her region." width="800" height="800"&gt;&lt;/a&gt;&lt;br&gt;
&lt;em&gt;Original AI-generated illustration: column masking does not grant row access.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;A data-governance slide says “only authorized analysts can see customer PII.” That sentence sounds reassuring until someone asks which identity was used for the test, whether null attributes are allowed through, and what happens when the same table is queried from a different notebook or SQL warehouse.&lt;/p&gt;

&lt;p&gt;I prefer a claim that can fail a test. For a particular table, an unauthorized principal should see &lt;strong&gt;no disallowed rows&lt;/strong&gt; and &lt;strong&gt;no raw protected values&lt;/strong&gt; , even when the query is not the one the policy author anticipated. Databricks Unity Catalog row filters and column masks provide table-level controls at query time, but their reliability depends on function logic, type compatibility, privilege management and execution context. For policies that must span many tables, Databricks now recommends attribute-based access control (ABAC) instead of maintaining a function attachment by hand on every table.&lt;/p&gt;

&lt;p&gt;Let’s build the smallest realistic demonstration and then attack our assumptions. A mask should protect the email column, not cosplay as an access-control strategy.&lt;/p&gt;
&lt;h3&gt;
  
  
  Row filtering and masking answer different questions
&lt;/h3&gt;

&lt;p&gt;A &lt;strong&gt;row filter&lt;/strong&gt; is a SQL user-defined function that returns a Boolean for each row. FALSE and NULL both mean the row is not returned. It is appropriate for conditions such as region, tenant, department or legal entity, once the identity source and authorization model are defined.&lt;/p&gt;

&lt;p&gt;A &lt;strong&gt;column mask&lt;/strong&gt; is a SQL UDF applied when a protected column is read. It returns a value with a type compatible with the original column. A mask can preserve a value for an authorized group and return a redacted placeholder for everyone else. One table can have a row filter and multiple column masks, but each masked column has one active mask.&lt;/p&gt;

&lt;p&gt;The design rule is straightforward: a missing or unexpected condition should not broaden access. The SQL below uses an explicit allowlist for regions and a default-redacted mask.&lt;/p&gt;
&lt;h3&gt;
  
  
  Reproducible Databricks SQL lab
&lt;/h3&gt;

&lt;p&gt;This lab requires a Unity Catalog-enabled Databricks workspace, SQL warehouse or supported Databricks Runtime, a writable catalog, and permission to create a schema, functions and a test table. &lt;strong&gt;Replace&lt;/strong&gt;  &lt;strong&gt;main with your authorized catalog.&lt;/strong&gt; The group names are fictional and must exist in your account for the identity-specific test matrix. All customer data below is fake. Do not run this against a production table.&lt;/p&gt;

&lt;p&gt;Run the following statements in a Databricks SQL editor. An account admin should provision three test groups: governance-admins, br-analysts and pii-readers. The testers must also receive the ordinary privileges necessary to query the demo table. These groups are demonstrations, not a suggested enterprise naming standard.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;CREATE&lt;/span&gt; &lt;span class="k"&gt;SCHEMA&lt;/span&gt; &lt;span class="k"&gt;IF&lt;/span&gt; &lt;span class="k"&gt;NOT&lt;/span&gt; &lt;span class="k"&gt;EXISTS&lt;/span&gt; &lt;span class="n"&gt;main&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;jta_governance_demo&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="k"&gt;CREATE&lt;/span&gt; &lt;span class="k"&gt;OR&lt;/span&gt; &lt;span class="k"&gt;REPLACE&lt;/span&gt; &lt;span class="k"&gt;FUNCTION&lt;/span&gt; &lt;span class="n"&gt;main&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;jta_governance_demo&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;allow_region&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;region&lt;/span&gt; &lt;span class="no"&gt;STRING&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;RETURNS&lt;/span&gt; &lt;span class="kt"&gt;BOOLEAN&lt;/span&gt;
&lt;span class="k"&gt;RETURN&lt;/span&gt;
  &lt;span class="nf"&gt;is_account_group_member&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'governance-admins'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
  &lt;span class="k"&gt;OR&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;region&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s1"&gt;'BR'&lt;/span&gt; &lt;span class="k"&gt;AND&lt;/span&gt; &lt;span class="nf"&gt;is_account_group_member&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'br-analysts'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="k"&gt;CREATE&lt;/span&gt; &lt;span class="k"&gt;OR&lt;/span&gt; &lt;span class="k"&gt;REPLACE&lt;/span&gt; &lt;span class="k"&gt;FUNCTION&lt;/span&gt; &lt;span class="n"&gt;main&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;jta_governance_demo&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;mask_email&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;raw_email&lt;/span&gt; &lt;span class="no"&gt;STRING&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;RETURNS&lt;/span&gt; &lt;span class="no"&gt;STRING&lt;/span&gt;
&lt;span class="k"&gt;RETURN&lt;/span&gt; &lt;span class="k"&gt;CASE&lt;/span&gt;
  &lt;span class="k"&gt;WHEN&lt;/span&gt; &lt;span class="nf"&gt;is_account_group_member&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'pii-readers'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;THEN&lt;/span&gt; &lt;span class="n"&gt;raw_email&lt;/span&gt;
  &lt;span class="k"&gt;ELSE&lt;/span&gt; &lt;span class="s1"&gt;'REDACTED'&lt;/span&gt;
&lt;span class="k"&gt;END&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="k"&gt;CREATE&lt;/span&gt; &lt;span class="k"&gt;OR&lt;/span&gt; &lt;span class="k"&gt;REPLACE&lt;/span&gt; &lt;span class="k"&gt;TABLE&lt;/span&gt; &lt;span class="n"&gt;main&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;jta_governance_demo&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;orders&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;
  &lt;span class="n"&gt;order_id&lt;/span&gt; &lt;span class="kt"&gt;BIGINT&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;region&lt;/span&gt; &lt;span class="no"&gt;STRING&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;customer_email&lt;/span&gt; &lt;span class="no"&gt;STRING&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;USING&lt;/span&gt; &lt;span class="no"&gt;DELTA&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="k"&gt;INSERT&lt;/span&gt; &lt;span class="k"&gt;INTO&lt;/span&gt; &lt;span class="n"&gt;main&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;jta_governance_demo&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;orders&lt;/span&gt; &lt;span class="k"&gt;VALUES&lt;/span&gt;
  &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'BR'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'ana@example.invalid'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'US'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'bob@example.invalid'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;NULL&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'unknown@example.invalid'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="k"&gt;ALTER&lt;/span&gt; &lt;span class="k"&gt;TABLE&lt;/span&gt; &lt;span class="n"&gt;main&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;jta_governance_demo&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;orders&lt;/span&gt;
&lt;span class="k"&gt;SET&lt;/span&gt; &lt;span class="k"&gt;ROW&lt;/span&gt; &lt;span class="k"&gt;FILTER&lt;/span&gt; &lt;span class="n"&gt;main&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;jta_governance_demo&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;allow_region&lt;/span&gt; &lt;span class="k"&gt;ON&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;region&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="k"&gt;ALTER&lt;/span&gt; &lt;span class="k"&gt;TABLE&lt;/span&gt; &lt;span class="n"&gt;main&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;jta_governance_demo&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;orders&lt;/span&gt;
&lt;span class="k"&gt;ALTER&lt;/span&gt; &lt;span class="k"&gt;COLUMN&lt;/span&gt; &lt;span class="n"&gt;customer_email&lt;/span&gt;
&lt;span class="k"&gt;SET&lt;/span&gt; &lt;span class="no"&gt;MASK&lt;/span&gt; &lt;span class="n"&gt;main&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;jta_governance_demo&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;mask_email&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="n"&gt;order_id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;region&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;customer_email&lt;/span&gt;
&lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="n"&gt;main&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;jta_governance_demo&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;orders&lt;/span&gt;
&lt;span class="k"&gt;ORDER&lt;/span&gt; &lt;span class="k"&gt;BY&lt;/span&gt; &lt;span class="n"&gt;order_id&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;What should this return? &lt;strong&gt;It depends on the querying principal.&lt;/strong&gt; You cannot adequately validate the policy by rerunning SELECT as its owner and assuming every analyst has the same view.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Querying identity:&lt;/strong&gt; Member of br-analysts only; &lt;strong&gt;Expected rows:&lt;/strong&gt; Order 1; &lt;strong&gt;Expected email values:&lt;/strong&gt;  REDACTED&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Querying identity:&lt;/strong&gt; Member of br-analysts and pii-readers; &lt;strong&gt;Expected rows:&lt;/strong&gt; Order 1; &lt;strong&gt;Expected email values:&lt;/strong&gt; &lt;a href="mailto:ana@example.invalid"&gt;ana@example.invalid&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Querying identity:&lt;/strong&gt; Member of governance-admins but not pii-readers; &lt;strong&gt;Expected rows:&lt;/strong&gt; Orders 1, 2, 3; &lt;strong&gt;Expected email values:&lt;/strong&gt; All REDACTED&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Querying identity:&lt;/strong&gt; Member of none of these groups; &lt;strong&gt;Expected rows:&lt;/strong&gt; No rows; &lt;strong&gt;Expected email values:&lt;/strong&gt; No values&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Querying identity:&lt;/strong&gt; Member of pii-readers only; &lt;strong&gt;Expected rows:&lt;/strong&gt; No rows; &lt;strong&gt;Expected email values:&lt;/strong&gt; No values&lt;/p&gt;

&lt;p&gt;Two details are intentional. First, the null region is &lt;strong&gt;not visible&lt;/strong&gt; to a Brazil-only analyst because the allow condition is not true for that record. Second, membership in pii-readers does not grant row access on its own. The mask governs &lt;em&gt;values of rows that survived the filter&lt;/em&gt;; it does not manufacture access to protected rows.&lt;/p&gt;

&lt;p&gt;One nuance matters for the third row: NULL passed through allow_region is allowed &lt;strong&gt;only&lt;/strong&gt; when the governance-admins branch is true. Everyone else sees it filtered out. That is a conscious administrative rule, and it should be reviewed as part of the access model.&lt;/p&gt;

&lt;h3&gt;
  
  
  Tests worth automating
&lt;/h3&gt;

&lt;p&gt;A screenshot from the SQL editor is not a security test suite. I would maintain a test matrix with the expected counts and masked values for several dedicated non-production principals. Repeat it whenever group assignments, the UDFs or the table schema change.&lt;/p&gt;

&lt;p&gt;At minimum, automate these acceptance checks:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;&lt;p&gt;A principal without any allowed groups gets zero rows. This tests the default-deny path.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;A Brazil-only analyst cannot retrieve the US row, including by joining on a supposedly hidden identifier.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;A PII-only principal does not gain region permissions by accident.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;An analyst loses raw email access immediately after removal from pii-readers, subject to normal identity propagation and cache behavior in your environment.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;A null or unexpected region does not appear to a region-limited analyst.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Query plans and performance remain acceptable with the filter and mask enabled.&lt;/p&gt;&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;A test harness can run SELECT COUNT(*) and SELECT customer_email as each principal and compare the result set to a fixed contract. &lt;strong&gt;Do not rely on a session variable that pretends to change the current account identity&lt;/strong&gt; ; the authorization functions use the caller's real identity and group membership.&lt;/p&gt;

&lt;h3&gt;
  
  
  The most dangerous bug may be a data type, not a missing policy
&lt;/h3&gt;

&lt;p&gt;Databricks documentation describes a subtle failure mode: if a table column and its UDF parameter have different types, implicit casts occur. With ANSI mode disabled, values that cannot be cast can become NULL silently. A filter written with an unfortunate IS NULL branch could unintentionally allow rows after such a cast.&lt;/p&gt;

&lt;p&gt;Keep the UDF signature aligned with the actual column type, prefer predictable expressions, and validate policies with data containing nulls and malformed values. For sensitive workloads, follow the documented recommendation around ANSI behavior and type matching rather than trusting a policy simply because it compiled.&lt;/p&gt;

&lt;p&gt;Also keep the masking function as simple as possible. More arguments, Python UDFs or mapping-table lookups can cost optimization opportunities. Filtering and masking are a security boundary: the engine may prioritize correct policy enforcement over aggressive pushdown, and that tradeoff can show up in query latency.&lt;/p&gt;

&lt;h3&gt;
  
  
  Table-level rules, ABAC and dynamic views are not interchangeable
&lt;/h3&gt;

&lt;p&gt;&lt;strong&gt;Manual table-level UDF attachment&lt;/strong&gt; is a good fit when there are few tables, well-defined owners and localized requirements. It is a weaker governance operating model when hundreds of independently managed tables need the same sensitive-column policy. A table owner or authorized manager can change a manual attachment, so your organization must audit changes and control policy administration.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;ABAC policies&lt;/strong&gt; are a stronger default for centrally managed rules: policy functions can be attached at catalog or schema scope and target tables/columns using governed tags. This helps new tagged tables inherit policies and separates stewardship from authoring. Databricks recommends ABAC for consistent filtering and masking across many tables; verify runtime and privilege requirements before adopting SQL CREATE POLICY in an older workspace.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Dynamic views&lt;/strong&gt; are useful when consumers should access only a curated SQL interface, such as a joined or transformed subset of base tables. They are not the same as attaching a table-level rule to every direct reader of the base table. The choice should follow the threat model and how users are actually granted access.&lt;/p&gt;

&lt;p&gt;None of these replaces clean catalog permissions. Users need the appropriate table and schema privileges in addition to whatever row-level or column-level policy logic applies. In other words, a mask is not permission to query everything, and a SELECT grant is not permission to ignore a mask.&lt;/p&gt;

&lt;h3&gt;
  
  
  Production limitations you should put in the architecture decision
&lt;/h3&gt;

&lt;p&gt;Databricks lists important constraints for tables with row filters or column masks. Among them: path-based file access to protected tables is not a supported way to query through the policy; time travel and cloning have restrictions; some Delta APIs and interoperability surfaces are unsupported; certain UPDATE, DELETE and MERGE patterns have runtime-dependent limitations. Dedicated-access compute has additional version and serverless fine-grained-access-control requirements. These constraints are &lt;strong&gt;design inputs&lt;/strong&gt; , not footnotes to discover after migration.&lt;/p&gt;

&lt;p&gt;The example uses manual table-level policies and therefore should not be mistaken for a demonstration of centralized ABAC. The SQL syntax can be checked against documentation, but there is &lt;strong&gt;no claim here that the example has been executed in a Databricks workspace&lt;/strong&gt;. Group creation, permissions, supported compute/runtime and behavior under each identity must be validated there before use in production.&lt;/p&gt;

&lt;p&gt;For lab cleanup, detach the policies before dropping their functions:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;ALTER&lt;/span&gt; &lt;span class="k"&gt;TABLE&lt;/span&gt; &lt;span class="n"&gt;main&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;jta_governance_demo&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;orders&lt;/span&gt;
  &lt;span class="k"&gt;ALTER&lt;/span&gt; &lt;span class="k"&gt;COLUMN&lt;/span&gt; &lt;span class="n"&gt;customer_email&lt;/span&gt; &lt;span class="k"&gt;DROP&lt;/span&gt; &lt;span class="no"&gt;MASK&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="k"&gt;ALTER&lt;/span&gt; &lt;span class="k"&gt;TABLE&lt;/span&gt; &lt;span class="n"&gt;main&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;jta_governance_demo&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;orders&lt;/span&gt; &lt;span class="k"&gt;DROP&lt;/span&gt; &lt;span class="k"&gt;ROW&lt;/span&gt; &lt;span class="k"&gt;FILTER&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="k"&gt;DROP&lt;/span&gt; &lt;span class="k"&gt;TABLE&lt;/span&gt; &lt;span class="n"&gt;main&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;jta_governance_demo&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;orders&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="k"&gt;DROP&lt;/span&gt; &lt;span class="k"&gt;FUNCTION&lt;/span&gt; &lt;span class="n"&gt;main&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;jta_governance_demo&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;mask_email&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="k"&gt;DROP&lt;/span&gt; &lt;span class="k"&gt;FUNCTION&lt;/span&gt; &lt;span class="n"&gt;main&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;jta_governance_demo&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;allow_region&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="c1"&gt;-- Drop the demo schema only if nothing else uses it.&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Detaching first matters. Databricks explicitly warns that deleting a referenced masking function while leaving the mask attached can leave a table inaccessible until the orphaned reference is removed.&lt;/p&gt;

&lt;h3&gt;
  
  
  The takeaway
&lt;/h3&gt;

&lt;p&gt;Good governance is a set of falsifiable expectations: &lt;em&gt;which identity can see which row, and which value, under which compute path?&lt;/em&gt; Start with a small allowlist, design for missing attributes, test several principals, and write down what you have not tested. For organization-wide coverage, move beyond copy-paste table functions toward centrally governed ABAC and audit your permissions continuously.&lt;/p&gt;

&lt;p&gt;A secure dashboard is not the number of policies created. It is the number of unauthorized paths you have proved do not work.&lt;/p&gt;

&lt;h3&gt;
  
  
  Official sources
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;Databricks, &lt;strong&gt;Row filters and column masks&lt;/strong&gt; (overview, ABAC choice, compatibility and limits): &lt;a href="https://docs.databricks.com/aws/en/data-governance/unity-catalog/filters-and-masks/" rel="noopener noreferrer"&gt;https://docs.databricks.com/aws/en/data-governance/unity-catalog/filters-and-masks/&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Databricks, &lt;strong&gt;Manually apply row filters and column masks&lt;/strong&gt; (UDF syntax, SET ROW FILTER, SET MASK, removing policies): &lt;a href="https://docs.databricks.com/aws/en/data-governance/unity-catalog/filters-and-masks/manually-apply" rel="noopener noreferrer"&gt;https://docs.databricks.com/aws/en/data-governance/unity-catalog/filters-and-masks/manually-apply&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Databricks, &lt;strong&gt;ROW FILTER clause&lt;/strong&gt; : &lt;a href="https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-syntax-ddl-row-filter" rel="noopener noreferrer"&gt;https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-syntax-ddl-row-filter&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Databricks, &lt;strong&gt;Column mask clause&lt;/strong&gt; : &lt;a href="https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-syntax-ddl-column-mask" rel="noopener noreferrer"&gt;https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-syntax-ddl-column-mask&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Databricks, &lt;strong&gt;CREATE POLICY&lt;/strong&gt; (ABAC): &lt;a href="https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-syntax-ddl-create-policy" rel="noopener noreferrer"&gt;https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-syntax-ddl-create-policy&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  Continue in the lab
&lt;/h3&gt;

&lt;p&gt;The &lt;a href="https://github.com/jenniferevangelista-de/data-engineering-lab" rel="noopener noreferrer"&gt;Data Engineering Lab&lt;/a&gt; separates runnable examples from service-integration claims. Its &lt;a href="https://github.com/jenniferevangelista-de/data-engineering-lab/tree/main/databricks/community_lab" rel="noopener noreferrer"&gt;Community lab&lt;/a&gt; demonstrates synthetic SQL checks and explicit NULL handling; it is a testing reference, not a Unity Catalog authorization test suite. The identity matrix in this article still needs an isolated Databricks workspace.&lt;/p&gt;

&lt;p&gt;For the complementary data-correctness contract, read &lt;a href="https://medium.com/@jenniferevangelista.code/your-delta-merge-is-not-a-mind-reader-16e81b6c788b" rel="noopener noreferrer"&gt;Your Delta MERGE Is Not a Mind Reader&lt;/a&gt;. Recovery and replay are covered in &lt;a href="https://medium.com/@jenniferevangelista.code/your-s3-pipeline-needs-a-replay-plan-not-more-optimism-a62342b93e89" rel="noopener noreferrer"&gt;Your S3 Pipeline Needs a Replay Plan, Not More Optimism&lt;/a&gt;.&lt;/p&gt;

</description>
      <category>databricks</category>
    </item>
    <item>
      <title>Your Kafka Consumer Lag Is Not Your SLA: Offset Math, Retention, and the Missing-Commit Trap</title>
      <dc:creator>Jennifer Evangelista</dc:creator>
      <pubDate>Mon, 19 Oct 2026 20:58:58 +0000</pubDate>
      <link>https://dev.to/jenniferevangelistade/your-kafka-consumer-lag-is-not-your-sla-offset-math-retention-and-the-missing-commit-trap-4pm0</link>
      <guid>https://dev.to/jenniferevangelistade/your-kafka-consumer-lag-is-not-your-sla-offset-math-retention-and-the-missing-commit-trap-4pm0</guid>
      <description>&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F431i4u1q7aq9occc26vh.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F431i4u1q7aq9occc26vh.png" alt="A woman detective unmasks a smiling Kafka dashboard labeled lag = 0, revealing a ghost labeled committed offset = null." width="800" height="1000"&gt;&lt;/a&gt;&lt;br&gt;
&lt;em&gt;Original AI-generated illustration: a zero-lag dashboard can hide a missing committed offset.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;A dashboard declares that your Kafka consumer lag is zero. Support closes the incident. The business report is still missing records.&lt;/p&gt;

&lt;p&gt;The chart is happy. The business would like its data back.&lt;/p&gt;

&lt;p&gt;The apparent contradiction is not necessarily a Kafka defect. It may be a defect in the &lt;em&gt;meaning&lt;/em&gt; assigned to the metric. Kafka offsets tell us how a particular reader relates to a particular partition’s log. They do not, on their own, certify that every business event was processed, that a downstream database commit succeeded, or that a latency target was met.&lt;/p&gt;

&lt;p&gt;The distinction matters because the common formula, log_end_offset - committed_offset, is only meaningful after checking that both offsets exist, refer to the same topic-partition and are compatible with the log's retention boundary. Before summing anything across partitions, I want to know which measurements are trustworthy.&lt;/p&gt;
&lt;h3&gt;
  
  
  Three offsets, three different questions
&lt;/h3&gt;

&lt;p&gt;For each topic-partition, collect these values:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Beginning offset (B).&lt;/strong&gt; The earliest offset still available to read from the log. Retention and log compaction can affect the relationship between offsets and actual accessible records.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;End offset (E).&lt;/strong&gt; The boundary representing the next offset at the end of the readable log, according to the API and isolation level used by your measurement.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Committed consumer-group offset, C. The offset the group has recorded as its next position. A committed offset is not a receipt from your destination system.&lt;/p&gt;&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;When B &amp;lt;= C &amp;lt;= E, the familiar backlog estimate is:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;partition_offset_lag&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="no"&gt;E&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="no"&gt;C&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Example: if a group has committed offset 140 and the observed end is 150, the offset lag is 10. If C equals E, lag is zero. That is a &lt;em&gt;specific statement about the observed positions&lt;/em&gt;, not proof of end-to-end correctness.&lt;/p&gt;

&lt;p&gt;The assumption that breaks many homegrown monitors is that C is always present and always falls in that range. It isn’t.&lt;/p&gt;

&lt;h3&gt;
  
  
  Four statuses instead of one deceptive number
&lt;/h3&gt;

&lt;p&gt;Consider an orders topic observed at 08:00. Four partitions show different states:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Partition:&lt;/strong&gt; 0; &lt;strong&gt;B:&lt;/strong&gt; 100; &lt;strong&gt;E:&lt;/strong&gt; 150; &lt;strong&gt;C:&lt;/strong&gt; 140; &lt;strong&gt;Correct interpretation:&lt;/strong&gt; Offset lag = 10; position is within the retained range.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Partition:&lt;/strong&gt; 1; &lt;strong&gt;B:&lt;/strong&gt; 200; &lt;strong&gt;E:&lt;/strong&gt; 260; &lt;strong&gt;C:&lt;/strong&gt; null; &lt;strong&gt;Correct interpretation:&lt;/strong&gt; No committed offset is available. Lag is &lt;strong&gt;unknown&lt;/strong&gt; , not zero.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Partition:&lt;/strong&gt; 2; &lt;strong&gt;B:&lt;/strong&gt; 500; &lt;strong&gt;E:&lt;/strong&gt; 620; &lt;strong&gt;C:&lt;/strong&gt; 480; &lt;strong&gt;Correct interpretation:&lt;/strong&gt; The committed position precedes retention by 20 offsets. Investigate potential unrecoverable gaps.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Partition:&lt;/strong&gt; 3; &lt;strong&gt;B:&lt;/strong&gt; 700; &lt;strong&gt;E:&lt;/strong&gt; 740; &lt;strong&gt;C:&lt;/strong&gt; 750; &lt;strong&gt;Correct interpretation:&lt;/strong&gt; The committed position is beyond the observed end. Validate snapshot consistency and log history.&lt;/p&gt;

&lt;p&gt;These become four operational states:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;OK:&lt;/strong&gt; B &amp;lt;= C &amp;lt;= E. Compute E - C.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;NO_COMMITTED_OFFSET:&lt;/strong&gt; C is absent. Do not substitute zero. This may be a new group, a deleted commit, a partition with no recorded group offset, or a mismatch between group and topic-partition inventory.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;BEHIND_RETENTION:&lt;/strong&gt; C &amp;lt; B. The span B - C is an unavailable &lt;em&gt;offset range&lt;/em&gt;. It is not automatically the number of lost business events, especially on compacted topics, but it is a concrete reason to stop presenting the group as healthy.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;AHEAD_OF_LOG_END:&lt;/strong&gt; C &amp;gt; E. Never clamp a negative result to zero. A racing measurement, broker/log changes or an incorrect partition association could have produced inconsistent boundaries; investigate before claiming anything about throughput.&lt;/p&gt;

&lt;p&gt;When even one partition is unknown or inconsistent, my group-level total_lag is null. A partial sum of healthy partitions can still be shown, but it must not be labeled the total.&lt;/p&gt;

&lt;h3&gt;
  
  
  A reproducible audit in plain Python
&lt;/h3&gt;

&lt;p&gt;You can test these rules with synthetic snapshots without standing up Kafka. Save this as lag_demo.py and run python lag_demo.py with Python 3.10 or later:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;dataclasses&lt;/span&gt; &lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;dataclass&lt;/span&gt;

&lt;span class="nd"&gt;@dataclass&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;frozen&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="kc"&gt;True&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;Snapshot&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;partition&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;int&lt;/span&gt;
    &lt;span class="n"&gt;beginning&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;int&lt;/span&gt;
    &lt;span class="n"&gt;end&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;int&lt;/span&gt;
    &lt;span class="n"&gt;committed&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;int&lt;/span&gt; &lt;span class="o"&gt;|&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;classify&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;row&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;Snapshot&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="kt"&gt;tuple&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="kt"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kt"&gt;int&lt;/span&gt; &lt;span class="o"&gt;|&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;b&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;e&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;row&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;beginning&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;row&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;end&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;row&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;committed&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;b&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt; &lt;span class="ow"&gt;or&lt;/span&gt; &lt;span class="n"&gt;e&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="n"&gt;b&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;raise&lt;/span&gt; &lt;span class="nc"&gt;ValueError&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"invalid retained log boundary"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt; &lt;span class="k"&gt;is&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"NO_COMMITTED_OFFSET"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;raise&lt;/span&gt; &lt;span class="nc"&gt;ValueError&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"negative committed offset"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="n"&gt;b&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"BEHIND_RETENTION"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="n"&gt;e&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"AHEAD_OF_LOG_END"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"OK"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;e&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="n"&gt;snapshots&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
    &lt;span class="nc"&gt;Snapshot&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;100&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;150&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;140&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="nc"&gt;Snapshot&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;200&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;260&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="nc"&gt;Snapshot&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;500&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;620&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;480&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="nc"&gt;Snapshot&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;700&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;740&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;750&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;]&lt;/span&gt;
&lt;span class="n"&gt;results&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nf"&gt;classify&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;row&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;row&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;snapshots&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;row&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;status&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;lag&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="nb"&gt;zip&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;snapshots&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;results&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"partition={row.partition}, status={status}, lag={lag}"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="k"&gt;assert&lt;/span&gt; &lt;span class="n"&gt;results&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
    &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"OK"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;10&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"NO_COMMITTED_OFFSET"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"BEHIND_RETENTION"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"AHEAD_OF_LOG_END"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;]&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Expected output:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;partition=0, status=OK, lag=10
partition=1, status=NO_COMMITTED_OFFSET, lag=None
partition=2, status=BEHIND_RETENTION, lag=None
partition=3, status=AHEAD_OF_LOG_END, lag=None
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This example classifies provided snapshots; it does &lt;strong&gt;not&lt;/strong&gt; connect to a broker. A production integration would fetch committed group offsets through Kafka’s Admin API (listConsumerGroupOffsets), and the beginning/end boundaries through listOffsets, grouping by exact topic and partition. It would also attach group_id, collection timestamp, cluster identity and client/isolation configuration to each snapshot. Because these calls are not an atomic multi-partition snapshot, transient inconsistencies should trigger a refresh and diagnosis rather than automatic data-loss claims.&lt;/p&gt;

&lt;p&gt;The collector and the calculation should be separate components. The code above is the complete local calculation demo for this article; the public lab does not yet contain a Kafka collector or broker integration. For the same unknown-versus-zero principle in a runnable Spark example, see &lt;a href="https://github.com/jenniferevangelista-de/data-engineering-lab/blob/main/databricks/community_lab/queries/optional_operation_metrics.sql" rel="noopener noreferrer"&gt;optional operation metrics in my Community lab&lt;/a&gt;. That example reads synthetic metric maps, not Kafka offsets.&lt;/p&gt;

&lt;h3&gt;
  
  
  Why low lag can coexist with a broken pipeline
&lt;/h3&gt;

&lt;p&gt;&lt;strong&gt;Commits can run ahead of downstream durability.&lt;/strong&gt; A consumer might commit offsets and then fail while writing to its warehouse. After restart, the consumer group position can look healthy while the target lacks events. Whether this is possible in a particular architecture depends on commit ordering, destination guarantees and transaction design. Kafka’s at-least-once and exactly-once discussions are about specific processing boundaries, not a magic property of the number in a dashboard.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;A group can be fast but stale.&lt;/strong&gt; A source may not have received events for an hour. Its end offset does not move, so its consumer can show zero lag. Your business freshness SLO is still breached. Measure source freshness, event-time age and sink visibility separately.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;One aggregate hides a hot partition.&lt;/strong&gt; A group with 200 partitions can have modest total lag but one partition carrying the orders that matter. Keep per-partition maxima and age percentiles, not just sum and average.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Offsets are not a wall clock.&lt;/strong&gt; E - C = 10,000 can mean seconds or hours depending on the arrival rate. Converting count to time by dividing by a short-term rate is an estimate that fails when traffic is bursty or paused.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Retention changes recovery choices.&lt;/strong&gt; If C falls below B, the consumer may need a policy decision about offset reset or reprocessing from a separate durable source. An automatic reset-to-latest can make the lag chart beautiful while discarding the very history the incident team needs. Define this behavior explicitly and log the decision.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Compacted topics complicate counts.&lt;/strong&gt; Offsets remain positions in the log, but not every offset within a numeric range necessarily represents a currently readable user record. Do not market offset differences as exact record counts for compacted workloads.&lt;/p&gt;

&lt;h3&gt;
  
  
  An on-call contract I would actually sign
&lt;/h3&gt;

&lt;p&gt;A useful consumer dashboard has at least three panels. &lt;strong&gt;Position health&lt;/strong&gt; shows per-partition B/C/E, group assignment, committed-offset availability and retention gaps. &lt;strong&gt;Pipeline health&lt;/strong&gt; tracks successful writes, retries, rejected records, deduplication/transaction results and error budgets. &lt;strong&gt;Business freshness&lt;/strong&gt; measures when the most recent valid business event became queryable in the serving layer, relative to the time it was expected.&lt;/p&gt;

&lt;p&gt;Alerts should distinguish “backlog above threshold” from “cannot calculate lag,” “committed position outside retention,” and “data not visible downstream.” The first is often a capacity or rate problem; the others can be correctness incidents. Treating all four as the same red/green threshold destroys useful diagnosis.&lt;/p&gt;

&lt;p&gt;When a pipeline is repaired, I would require a recovery test: record a known input key, trace its topic-partition-offset, verify the target effect, force a controlled retry, and confirm the intended duplicate-handling behavior. A lag graph cannot run this test for you.&lt;/p&gt;

&lt;h3&gt;
  
  
  Limitations and where to be careful
&lt;/h3&gt;

&lt;p&gt;The Python example is a &lt;strong&gt;local calculation test with fabricated values&lt;/strong&gt;. It does not validate authorization, Kafka brokers, consumer rebalances, committed-offset races, aborted transactions, compaction behavior or sink idempotency. Kafka’s Admin and consumer APIs differ in how they expose group state and offset boundaries; production collectors need a tested client/version policy. Do not infer that every missing committed offset means data loss, or that every retention gap equals lost events.&lt;/p&gt;

&lt;p&gt;If you use Structured Streaming with Kafka, its checkpointed progress and sink semantics must be analyzed as a separate implementation, not conflated with an arbitrary Kafka consumer group’s commits. That is another reason I keep source recovery, pipeline state and business outcomes as separate evidence.&lt;/p&gt;

&lt;h3&gt;
  
  
  The takeaway
&lt;/h3&gt;

&lt;p&gt;Offset lag is useful once you have proved that the inputs make sense. It tells you how far a recorded position is from an observed log boundary. It doesn’t answer whether the pipeline fulfilled its contract. Keep unknowns unknown, surface retention boundaries, and make the business SLO visible on its own terms.&lt;/p&gt;

&lt;h3&gt;
  
  
  Official sources
&lt;/h3&gt;

&lt;p&gt;Apache Kafka, &lt;a href="https://kafka.apache.org/41/javadoc/org/apache/kafka/clients/admin/Admin.html" rel="noopener noreferrer"&gt;Admin API&lt;/a&gt; and &lt;a href="https://kafka.apache.org/41/javadoc/org/apache/kafka/clients/consumer/KafkaConsumer.html" rel="noopener noreferrer"&gt;consumer API&lt;/a&gt;; &lt;a href="https://kafka.apache.org/documentation/" rel="noopener noreferrer"&gt;consumer groups and delivery semantics&lt;/a&gt;.&lt;/p&gt;

&lt;h3&gt;
  
  
  Continue in the lab
&lt;/h3&gt;

&lt;p&gt;&lt;a href="https://github.com/jenniferevangelista-de/data-engineering-lab" rel="noopener noreferrer"&gt;Data Engineering Lab&lt;/a&gt; contains small reproducible examples and explicit validation limits. &lt;a href="https://github.com/jenniferevangelista-de/data-engineering-lab/tree/main/pyspark/day_001_cdc" rel="noopener noreferrer"&gt;CDC preparation&lt;/a&gt; covers replay, ordering and conflicting versions with synthetic data; it does not validate a live Kafka sink.&lt;/p&gt;

&lt;p&gt;For recovery design, read &lt;a href="https://medium.com/@jenniferevangelista.code/your-s3-pipeline-needs-a-replay-plan-not-more-optimism-a62342b93e89" rel="noopener noreferrer"&gt;Your S3 Pipeline Needs a Replay Plan, Not More Optimism&lt;/a&gt;. For deterministic target updates, read &lt;a href="https://medium.com/@jenniferevangelista.code/your-delta-merge-is-not-a-mind-reader-16e81b6c788b" rel="noopener noreferrer"&gt;Your Delta MERGE Is Not a Mind Reader&lt;/a&gt;.&lt;/p&gt;

</description>
      <category>kafka</category>
    </item>
  </channel>
</rss>
