<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Trung Thanh Tran</title>
    <description>The latest articles on DEV Community by Trung Thanh Tran (@tthanh2k6).</description>
    <link>https://dev.to/tthanh2k6</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4216305%2Fa06f7630-f4e1-43d9-8d68-95519ecfa31f.jpg</url>
      <title>DEV Community: Trung Thanh Tran</title>
      <link>https://dev.to/tthanh2k6</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/tthanh2k6"/>
    <language>en</language>
    <item>
      <title>Four DOM Traps That Make Browser Agents Click or Fill the Wrong Element</title>
      <dc:creator>Trung Thanh Tran</dc:creator>
      <pubDate>Fri, 20 Nov 2026 03:25:43 +0000</pubDate>
      <link>https://dev.to/tthanh2k6/four-dom-traps-that-make-browser-agents-click-or-fill-the-wrong-element-7ie</link>
      <guid>https://dev.to/tthanh2k6/four-dom-traps-that-make-browser-agents-click-or-fill-the-wrong-element-7ie</guid>
      <description>&lt;p&gt;Last week I ran an AI coding agent overnight across Gmail, Wikipedia, Reddit, and Dover ATS forms through &lt;a href="https://github.com/Tthanh2k6/turbo-browser-mcp" rel="noopener noreferrer"&gt;turbo-browser-mcp&lt;/a&gt;, my Playwright MCP server. All 750 local fixture tests were green before the run started. By morning, real production DOMs had broken element targeting in four ways that synthetic fixtures missed.&lt;/p&gt;

&lt;h2&gt;
  
  
  1. Recycled DOM nodes in virtualized lists
&lt;/h2&gt;

&lt;p&gt;Most browser MCP servers assign a reference attribute like &lt;code&gt;data-tbm-ref="e4"&lt;/code&gt; during a snapshot so the model can pass &lt;code&gt;ref: "f0:e4"&lt;/code&gt; on the next turn. In a normal SPA re-render, a replaced subtree drops the attribute and a stale ref fails cleanly.&lt;/p&gt;

&lt;p&gt;Virtualized lists (&lt;code&gt;react-window&lt;/code&gt;, Gmail thread tables, infinite feeds) do something different: when the viewport scrolls, they keep the exact same DOM node and swap its text and event handlers for a new row. The &lt;code&gt;data-tbm-ref="e4"&lt;/code&gt; attribute survives on the recycled node, so a stale ref clicks Row 14 when the model intended to click Row 0.&lt;/p&gt;

&lt;p&gt;We fixed this in two layers:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;For buttons, links, and clickable rows, each issued ref records &lt;code&gt;(frameLoaderId, first 128 chars of normalized textContent)&lt;/code&gt;. If either changes before the action runs, the tool refuses the ref and asks for a fresh &lt;code&gt;inspect_compact&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;For value controls (&lt;code&gt;&amp;lt;input&amp;gt;&lt;/code&gt;, &lt;code&gt;&amp;lt;textarea&amp;gt;&lt;/code&gt;, &lt;code&gt;&amp;lt;select&amp;gt;&lt;/code&gt;, &lt;code&gt;contenteditable&lt;/code&gt;), exempting the value is required so typing twice into the same field works, while exempting the identity is a bug. Each value control is stamped with &lt;code&gt;#id:&lt;/code&gt; plus &lt;code&gt;tag&lt;/code&gt;, &lt;code&gt;type&lt;/code&gt;, &lt;code&gt;name&lt;/code&gt;, &lt;code&gt;id&lt;/code&gt;, &lt;code&gt;aria-label&lt;/code&gt;, &lt;code&gt;placeholder&lt;/code&gt;, &lt;code&gt;role&lt;/code&gt;, &lt;code&gt;title&lt;/code&gt;, and resolved &lt;code&gt;&amp;lt;label&amp;gt;&lt;/code&gt; text (with live character counters like &lt;code&gt;(12/160)&lt;/code&gt; normalized to &lt;code&gt;(#/#)&lt;/code&gt;).&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  2. Cloned form inputs when sections reorder or delete
&lt;/h2&gt;

&lt;p&gt;In a review thread on my previous post, Renhua Wei pointed out a gap in attribute stamps: repeated fields like three &lt;code&gt;"Phone"&lt;/code&gt; inputs or multi-line address rows share the exact same &lt;code&gt;tag&lt;/code&gt;, &lt;code&gt;type&lt;/code&gt;, &lt;code&gt;name&lt;/code&gt;, and &lt;code&gt;aria-label&lt;/code&gt;. If two cloned rows swap positions, an attribute-only stamp still matches.&lt;/p&gt;

&lt;p&gt;Appending the element's 0-based index among identical clones solves a straight swap, until the user deletes row 1 of 3. Then row 2 shifts from index &lt;code&gt;1&lt;/code&gt; to &lt;code&gt;0&lt;/code&gt; and row 3 shifts from &lt;code&gt;2&lt;/code&gt; to &lt;code&gt;1&lt;/code&gt;, breaking every surviving ref in the form.&lt;/p&gt;

&lt;p&gt;The fix that survived both cases (&lt;code&gt;ba47c55&lt;/code&gt; and &lt;code&gt;3446e79&lt;/code&gt;) scopes the clone group to the &lt;strong&gt;nearest ancestor holding at least two matching clones&lt;/strong&gt; and records &lt;code&gt;index/total&lt;/code&gt; (&lt;code&gt;0/3&lt;/code&gt;, &lt;code&gt;1/3&lt;/code&gt;, &lt;code&gt;2/3&lt;/code&gt;):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="c1"&gt;// When group size stays 3 (a reorder or swap), an index change is a stale ref error.&lt;/span&gt;
&lt;span class="c1"&gt;// When group size changes (3 -&amp;gt; 2 after deleting a row), shift by (curTotal - storedTotal)&lt;/span&gt;
&lt;span class="c1"&gt;// and attach a one-time note so surviving refs still fill the right inputs.&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That scoping has an important consequence depending on form layout:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;If two &lt;code&gt;&amp;lt;fieldset&amp;gt;&lt;/code&gt; sections each hold three identical &lt;code&gt;Phone&lt;/code&gt; inputs, the nearest clone-holding ancestor is the &lt;code&gt;&amp;lt;fieldset&amp;gt;&lt;/code&gt;. Swapping the two fieldsets wholesale with &lt;code&gt;insertBefore&lt;/code&gt; keeps every input at &lt;code&gt;0/3&lt;/code&gt;, &lt;code&gt;1/3&lt;/code&gt;, &lt;code&gt;2/3&lt;/code&gt; inside its own fieldset, so all six refs stay valid.&lt;/li&gt;
&lt;li&gt;If you flip the layout to three unlabelled &lt;code&gt;&amp;lt;fieldset&amp;gt;&lt;/code&gt; elements with one &lt;code&gt;Phone&lt;/code&gt; input each, the nearest clone-holding ancestor falls back to the &lt;code&gt;&amp;lt;form&amp;gt;&lt;/code&gt;. Swapping the first two fieldsets flips their indices inside the form (&lt;code&gt;0/3 &amp;lt;-&amp;gt; 1/3&lt;/code&gt;) and throws a stale-ref error (&lt;code&gt;test/night-19-11-clone-shift.test.mjs&lt;/code&gt;).&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  3. Hydration mutating &lt;code&gt;title&lt;/code&gt; attributes after the snapshot
&lt;/h2&gt;

&lt;p&gt;On Tuesday night our automated real-site run hit Wikipedia (&lt;code&gt;en.wikipedia.org&lt;/code&gt;). Right after &lt;code&gt;inspect_compact&lt;/code&gt; returned the search box ref, every &lt;code&gt;act_fast&lt;/code&gt; call to type into that search box failed with &lt;code&gt;now points to a different field&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Reproducing it in a standalone script (&lt;code&gt;docs/repro_wiki_search.mjs&lt;/code&gt;) showed why: at initial &lt;code&gt;domcontentloaded&lt;/code&gt;, Wikipedia's search input has &lt;code&gt;title="Search Wikipedia [f]"&lt;/code&gt;. A few hundred milliseconds later, client-side JavaScript rewrites the access-key hint to match the platform modifier: &lt;code&gt;title="Search Wikipedia [alt-f]"&lt;/code&gt; (or &lt;code&gt;[alt-shift-f]&lt;/code&gt;).&lt;/p&gt;

&lt;p&gt;Because &lt;code&gt;title&lt;/code&gt; was part of our identity stamp, the late access-key rewrite invalidated the ref between the snapshot turn and the typing turn. Stripping trailing &lt;code&gt;[...]&lt;/code&gt; access-key suffixes when normalizing &lt;code&gt;title&lt;/code&gt; (&lt;code&gt;c140e2d&lt;/code&gt;) fixed Wikipedia and MediaWiki sites without weakening actual title checks.&lt;/p&gt;

&lt;h2&gt;
  
  
  4. Chrome freezing background tabs and repeating CDP prompts
&lt;/h2&gt;

&lt;p&gt;When an agent drives your real logged-in Chrome over CDP while you work in another tab, two browser behaviors get in the way:&lt;/p&gt;

&lt;p&gt;First, opening a new tab brings it to the foreground and steals keyboard focus. I opened a PR upstream to Google's official &lt;a href="https://github.com/ChromeDevTools/chrome-devtools-mcp/pull/3539" rel="noopener noreferrer"&gt;&lt;code&gt;ChromeDevTools/chrome-devtools-mcp#3539&lt;/code&gt;&lt;/a&gt; adding a &lt;code&gt;--backgroundNewPages&lt;/code&gt; server flag so &lt;code&gt;new_page&lt;/code&gt; keeps tabs in the background even when the model omits &lt;code&gt;background: true&lt;/code&gt; (merged into &lt;code&gt;main&lt;/code&gt; yesterday).&lt;/p&gt;

&lt;p&gt;Second, once a tab stays in the background, Chrome throttles its lifecycle and stops running animation-frame rendering. When our agent opened Gmail in a background tab, the compose editor never mounted until someone clicked the tab. Pairing background tab creation with CDP &lt;code&gt;Emulation.setFocusEmulationEnabled({ enabled: true })&lt;/code&gt; and &lt;code&gt;Page.setWebLifecycleState({ state: "active" })&lt;/code&gt; (&lt;code&gt;d1e755f&lt;/code&gt;) keeps heavy SPAs rendering in the background while your active tab stays untouched.&lt;/p&gt;

&lt;p&gt;All four fixes ship in &lt;code&gt;turbo-browser-mcp&lt;/code&gt; (778 tests across Node 20 and 22):&lt;br&gt;
&lt;a href="https://github.com/Tthanh2k6/turbo-browser-mcp" rel="noopener noreferrer"&gt;https://github.com/Tthanh2k6/turbo-browser-mcp&lt;/a&gt;&lt;/p&gt;

</description>
      <category>mcp</category>
      <category>playwright</category>
      <category>ai</category>
      <category>webdev</category>
    </item>
    <item>
      <title>One "Allow remote debugging" click for every agent on my Chrome</title>
      <dc:creator>Trung Thanh Tran</dc:creator>
      <pubDate>Tue, 17 Nov 2026 07:36:09 +0000</pubDate>
      <link>https://dev.to/tthanh2k6/one-allow-remote-debugging-click-for-every-agent-on-my-chrome-38l5</link>
      <guid>https://dev.to/tthanh2k6/one-allow-remote-debugging-click-for-every-agent-on-my-chrome-38l5</guid>
      <description>&lt;p&gt;Since Chrome 144, turning on remote debugging in &lt;code&gt;chrome://inspect&lt;/code&gt; comes with a prompt: every new DevTools Protocol connection to your normal Chrome profile asks "Allow remote debugging?". That is a reasonable default for a browser that holds all your logins. It is painful when an agent reconnects on every step, or when two MCP servers want the same browser.&lt;/p&gt;

&lt;p&gt;There is an open issue on chrome-devtools-mcp asking to remember the approval (&lt;a href="https://github.com/ChromeDevTools/chrome-devtools-mcp/issues/825" rel="noopener noreferrer"&gt;#825&lt;/a&gt;, 21 upvotes when I last looked). The maintainers' position is that there is no safe way to persist it, and their suggestion is to keep connections open longer. So I pulled the piece of &lt;a href="https://github.com/Tthanh2k6/turbo-browser-mcp" rel="noopener noreferrer"&gt;turbo-browser-mcp&lt;/a&gt; that already did that into its own package: &lt;a href="https://github.com/Tthanh2k6/cdp-share" rel="noopener noreferrer"&gt;cdp-share&lt;/a&gt;.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;npx -y cdp-share
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;It connects to Chrome once (you click Allow), then serves the same &lt;code&gt;/json/version&lt;/code&gt; and WebSocket endpoints Chrome does on &lt;code&gt;http://127.0.0.1:9333&lt;/code&gt;. chrome-devtools-mcp gets &lt;code&gt;--browserUrl http://127.0.0.1:9333&lt;/code&gt;, Playwright gets &lt;code&gt;connectOverCDP('http://127.0.0.1:9333')&lt;/code&gt;, Puppeteer gets &lt;code&gt;browserURL&lt;/code&gt;. On my Chrome 157 I clicked Allow once and then connected Playwright three times, Puppeteer twice and chrome-devtools-mcp twice without another prompt.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why a plain relay broke
&lt;/h2&gt;

&lt;p&gt;The first version inside turbo was a plain WebSocket relay with one upstream socket. It worked with one client. The night I had two Claude Code sessions open, both running turbo against the same Chrome, the second one hung on its first call for over 100 seconds.&lt;/p&gt;

&lt;p&gt;CDP with flattened sessions puts every attached page on one socket, tagged by &lt;code&gt;sessionId&lt;/code&gt;. With two clients on one relay, both see the same session ids, and each client's Playwright assumes it owns them. The second client never got &lt;code&gt;Runtime.executionContextCreated&lt;/code&gt; for the main world, so every &lt;code&gt;evaluate&lt;/code&gt; waited forever, and creating a page failed with &lt;code&gt;Page.setFontFamilies: Font families can only be set once&lt;/code&gt; because the first client had already set up that session.&lt;/p&gt;

&lt;p&gt;cdp-share gives each client its own CDP sessions. When client B attaches to a tab, the daemon attaches a fresh upstream session for B, so B's &lt;code&gt;Runtime.enable&lt;/code&gt; produces B's own context events, and its init scripts and emulation settings do not collide with A's. &lt;code&gt;Target.setAutoAttach&lt;/code&gt; with &lt;code&gt;waitForDebuggerOnStart&lt;/code&gt; is emulated per client: a new tab is held until each auto-attaching client has resumed it, with a timeout so one stuck client cannot freeze the tab.&lt;/p&gt;

&lt;p&gt;A few other things a shared browser needs:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;Browser.close&lt;/code&gt; from a client is swallowed. Most Playwright and Puppeteer scripts end with &lt;code&gt;browser.close()&lt;/code&gt;, and without this the first script to finish closes your Chrome.&lt;/li&gt;
&lt;li&gt;Browser contexts a client created are disposed when it disconnects. Tabs in your default profile stay.&lt;/li&gt;
&lt;li&gt;Commands that were still waiting for an answer when a client left are dropped. I only found that one with a soak test: three clients connecting, opening pages and leaving for 30 overlapping rounds left 576 entries in the pending table. After the fix the same run ends with 0 clients, 0 leftover contexts and sessions, and about 8 MB of memory growth.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Not making it worse
&lt;/h2&gt;

&lt;p&gt;A browser with remote debugging on is fully controllable by anything that can reach the port. cdp-share does not fix that, but it should not widen it:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;it listens on 127.0.0.1 only,&lt;/li&gt;
&lt;li&gt;it rejects requests whose Host header is not localhost (DNS rebinding) and WebSocket upgrades that carry a web page Origin,&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;--token auto&lt;/code&gt; requires a token on every connection,&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;--log audit.jsonl&lt;/code&gt; records which client sent which command to which tab, without command parameters, so typed text and passwords stay out of the log,&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;--exclude-url&lt;/code&gt; hides tabs you do not want agents to see, for example your bank.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Run it with &lt;code&gt;--token auto&lt;/code&gt;. Without a token, any program on your machine that reaches the port gets the connection you already approved, and with it the cookies of every site you are signed in to, with no Allow prompt in the way. The token sits in a plain file under &lt;code&gt;~/.cdp-share/&lt;/code&gt; that your user can read, so it keeps out other users and web pages, not programs running as you.&lt;/p&gt;

&lt;h2&gt;
  
  
  What is not done
&lt;/h2&gt;

&lt;p&gt;The URL filter checks a tab's URL only, and a client already attached to a tab stays attached if the tab later navigates somewhere hidden. I have tested on Windows 11 only; CI runs the suite on Linux and macOS with headless Chromium, but not against a real Chrome profile there. During development the CLI crashed twice on Windows with exit code 0xC0000409. I changed every &lt;code&gt;process.exit()&lt;/code&gt; so the daemon closes its sockets first and lets the event loop drain. The crash has not come back in 35 runs since, but I never reproduced it on demand, so I can't say it is gone.&lt;/p&gt;

&lt;p&gt;If you run more than one agent against your own Chrome, I would like to hear what breaks: &lt;a href="https://github.com/Tthanh2k6/cdp-share/issues" rel="noopener noreferrer"&gt;https://github.com/Tthanh2k6/cdp-share/issues&lt;/a&gt;&lt;/p&gt;

</description>
      <category>chrome</category>
      <category>mcp</category>
      <category>playwright</category>
      <category>devtools</category>
    </item>
    <item>
      <title>What broke when an AI agent drove real websites overnight through my MCP browser server</title>
      <dc:creator>Trung Thanh Tran</dc:creator>
      <pubDate>Tue, 17 Nov 2026 01:22:56 +0000</pubDate>
      <link>https://dev.to/tthanh2k6/what-broke-when-an-ai-agent-drove-real-websites-overnight-through-my-mcp-browser-server-4nje</link>
      <guid>https://dev.to/tthanh2k6/what-broke-when-an-ai-agent-drove-real-websites-overnight-through-my-mcp-browser-server-4nje</guid>
      <description>&lt;p&gt;On the night of Nov 16 I left Claude Code running with turbo-browser-mcp attached to my own Chrome. Every 30 minutes it read Hacker News, dev.to and Reddit, opened job boards, filled forms, set up a Reddit profile and booked a call on a Notion Calendar page. Whenever a tool call did something wrong, I wrote down the exact arguments and output, fixed it, and ran the same step again on the same page.&lt;/p&gt;

&lt;p&gt;That became six rounds of fixes and &lt;a href="https://github.com/Tthanh2k6/turbo-browser-mcp/releases/tag/v0.7.0" rel="noopener noreferrer"&gt;v0.7.0&lt;/a&gt;. These are the bugs I would not have found with unit tests.&lt;/p&gt;

&lt;h2&gt;
  
  
  Two MCP servers on one Chrome hung forever
&lt;/h2&gt;

&lt;p&gt;I had a second Claude session open in another project, also running turbo. Its &lt;code&gt;tabs_batch&lt;/code&gt; call hung for over 100 seconds with nothing on stderr. My first guess was a tab with an &lt;code&gt;alert()&lt;/code&gt; left open, so I added a 1.5 s cap on the per-tab setup script. The guard is still useful, but the guess was wrong.&lt;/p&gt;

&lt;p&gt;The real cause was the CDP keep-alive bridge. Chrome 144+ asks you to click "Allow remote debugging" for every new connection, so turbo runs a small daemon that holds one approved connection and lets clients reuse it. With two clients, the bridge mixed up their sessions. The second one never received &lt;code&gt;Runtime.executionContextCreated&lt;/code&gt; for the main world, so every &lt;code&gt;evaluate&lt;/code&gt; waited forever, and opening a page failed with &lt;code&gt;Page.setFontFamilies: Font families can only be set once&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;The bridge now gives each client its own CDP sessions on top of one upstream socket. When a client disconnects, the bridge closes that client's isolated contexts and leaves the rest alone.&lt;/p&gt;

&lt;h2&gt;
  
  
  One frozen tab could block every call
&lt;/h2&gt;

&lt;p&gt;The wrong guess still pointed at a real weakness. Attaching to an existing Chrome ran a setup script in every frame of every open tab, with no timeout, so one tab whose main world does not answer (paused in DevTools, for example) was enough to freeze &lt;code&gt;tabs_batch list&lt;/code&gt;. Now each tab gets 1.5 s. A tab that does not answer is listed with &lt;code&gt;unresponsive: true&lt;/code&gt;, and a call aimed at it fails in under 2.5 s with "page main world is blocked" instead of hanging. The test pauses a page with &lt;code&gt;debugger;&lt;/code&gt; from a second CDP session to reproduce it.&lt;/p&gt;

&lt;h2&gt;
  
  
  The date field that cleared itself
&lt;/h2&gt;

&lt;p&gt;On the selenium.dev web form, &lt;code&gt;smart_fill_form&lt;/code&gt; reported the date as filled. The submitted form had &lt;code&gt;my-date=&lt;/code&gt; empty. The date picker widget wiped the value on blur. Fill now reads the value back after blur, retypes once if it changed, and reports &lt;code&gt;clearedAfterBlur&lt;/code&gt; or &lt;code&gt;changedAfterBlur&lt;/code&gt; so the agent knows something odd happened. On the re-run the form submitted &lt;code&gt;my-date=11/17/2026&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Same form, different bug: the select's first option is a placeholder ("Open this select menu"), and that counted as a filled field. Placeholder options now count as empty.&lt;/p&gt;

&lt;h2&gt;
  
  
  Submit guard: wrong in both directions
&lt;/h2&gt;

&lt;p&gt;turbo blocks final actions like Submit, Send or Pay unless the agent passes &lt;code&gt;allowSubmit: true&lt;/code&gt;. Real sites showed two problems.&lt;/p&gt;

&lt;p&gt;It fired on things that are not submissions: clicking a story title link on HN, pressing Enter in a site search box. Links and search forms no longer count.&lt;/p&gt;

&lt;p&gt;It missed things that are submissions. I had the agent book a real call on a Notion Calendar page, in Vietnamese. &lt;code&gt;smart_fill_form&lt;/code&gt; returned &lt;code&gt;submitButtons: []&lt;/code&gt; although the form ended in "Lên lịch họp" ("Schedule meeting"), so the guard had no idea that button was the final step. Booking verbs ("Book now", "Schedule meeting", "Đặt lịch") are in the guard now.&lt;/p&gt;

&lt;p&gt;The same booking found a hole in verification. After the click, &lt;code&gt;expect.textMatches: "xác nhận"&lt;/code&gt; ("confirm") passed in 6 ms, because the page already said "details are provided on confirmation" before anything happened. There is now &lt;code&gt;expect.textNew&lt;/code&gt;, which only counts text that was not on the page when the batch started (or a new document title).&lt;/p&gt;

&lt;h2&gt;
  
  
  Logged out without knowing it
&lt;/h2&gt;

&lt;p&gt;Around 05:40 Chrome stopped accepting remote debugging (I had not touched it). The bridge process stayed alive, but its upstream was gone, and the server quietly fell back to a headless browser. The agent kept working, now logged out of everything, and nothing in the tool results said so.&lt;/p&gt;

&lt;p&gt;Two changes. The bridge exits when its upstream closes. In auto mode the first tool result after a fallback carries one &lt;code&gt;connectionNote&lt;/code&gt;: "Chrome remote debugging is no longer available (bridge upstream closed); using a headless browser. Logged-in sites will not be available."&lt;/p&gt;

&lt;p&gt;The headless fallback then hit the next gap. &lt;code&gt;old.reddit.com&lt;/code&gt; redirected to &lt;code&gt;/login?reason=lor2&lt;/code&gt; with a 403 and an empty title, and the agent had nothing to explain why the page was empty. Redirects onto login paths and 401/403 login forms now report &lt;code&gt;blocked: "authwall"&lt;/code&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Smaller ones
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Clicking a disabled button waited for the full action timeout. It now fails in about a second.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;collect&lt;/code&gt; on HN took the vote link instead of the story link when the text selector matched the title anchor.&lt;/li&gt;
&lt;li&gt;Typing into a field with &lt;code&gt;maxlength&lt;/code&gt; or an input mask silently cut the text. The result now carries a &lt;code&gt;mismatch&lt;/code&gt; note, without echoing password or secret values.&lt;/li&gt;
&lt;li&gt;Tabs opened by another turbo process in its own isolated context were listed as &lt;code&gt;default&lt;/code&gt;. They are now &lt;code&gt;other&lt;/code&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Things that came out of reading other tools
&lt;/h2&gt;

&lt;p&gt;While I was at it I read how other browser agents handle untrusted pages and credentials, and added three opt-in features: a host allowlist (&lt;code&gt;--allow-hosts&lt;/code&gt;), &lt;code&gt;$secret:NAME&lt;/code&gt; placeholders read from a dotenv file at typing time, and &lt;code&gt;--content-boundaries&lt;/code&gt;, which wraps page text in &lt;code&gt;&amp;lt;&amp;lt;page:nonce&amp;gt;&amp;gt;&lt;/code&gt; markers. All three are off by default and cost nothing when off.&lt;/p&gt;

&lt;h2&gt;
  
  
  What is still not great
&lt;/h2&gt;

&lt;p&gt;One of the 38 real-site checks failed once and passed on the next run, and I did not keep the output. On a loaded Windows laptop a few timing tests in the suite of 677 also flake; CI on Linux is green. If you hit a hang or a wrong result, the most useful issue is the exact tool arguments plus the output, the same format I used for these notes.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;npx -y turbo-browser-mcp
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Repo: &lt;a href="https://github.com/Tthanh2k6/turbo-browser-mcp" rel="noopener noreferrer"&gt;https://github.com/Tthanh2k6/turbo-browser-mcp&lt;/a&gt;&lt;/p&gt;

</description>
      <category>mcp</category>
      <category>playwright</category>
      <category>ai</category>
      <category>testing</category>
    </item>
    <item>
      <title>Building a Faster Playwright MCP Server: Compact DOM Deltas, 1-Turn Form Filling, and Submit Guards</title>
      <dc:creator>Trung Thanh Tran</dc:creator>
      <pubDate>Sun, 15 Nov 2026 04:17:17 +0000</pubDate>
      <link>https://dev.to/tthanh2k6/building-a-faster-playwright-mcp-server-compact-dom-deltas-1-turn-form-filling-and-submit-guards-2o2p</link>
      <guid>https://dev.to/tthanh2k6/building-a-faster-playwright-mcp-server-compact-dom-deltas-1-turn-form-filling-and-submit-guards-2o2p</guid>
      <description>&lt;p&gt;While using a browser automation agent over Model Context Protocol (MCP) to prefill job applications and publishing forms, four bottlenecks showed up in the execution logs before any site rate limit did:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Full accessibility snapshots kept piling up in the conversation context until single turns carried 90,000+ tokens.&lt;/li&gt;
&lt;li&gt;Filling a form step-by-step (&lt;code&gt;click&lt;/code&gt;, &lt;code&gt;fill&lt;/code&gt;, &lt;code&gt;click&lt;/code&gt;, &lt;code&gt;fill&lt;/code&gt;) spent 10 to 15 MCP turns per page, and typing keystrokes into &lt;code&gt;&amp;lt;select&amp;gt;&lt;/code&gt; elements caused multi-second stalls.&lt;/li&gt;
&lt;li&gt;Passing file uploads as inline Base64 strings through LLM tool arguments forced the model to emit ~20,000 output tokens and stalled tool calls for over three minutes.&lt;/li&gt;
&lt;li&gt;One wrong click on a button labeled &lt;code&gt;Submit&lt;/code&gt; or &lt;code&gt;Publish&lt;/code&gt; sent half-checked data live before a human could inspect the page.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;I built &lt;code&gt;turbo-browser-mcp&lt;/code&gt; (&lt;code&gt;v0.4.0&lt;/code&gt;, MIT, &lt;code&gt;playwright-core&lt;/code&gt; with no other runtime dependencies) to group those patterns into 10 consolidated MCP tools:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://github.com/Tthanh2k6/turbo-browser-mcp" rel="noopener noreferrer"&gt;https://github.com/Tthanh2k6/turbo-browser-mcp&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Here are the four design choices in the codebase, along with where the approach still has limits.&lt;/p&gt;

&lt;h2&gt;
  
  
  1. Bounded &lt;code&gt;data-tbm-ref&lt;/code&gt; snapshots and &lt;code&gt;onlyChanged&lt;/code&gt; deltas
&lt;/h2&gt;

&lt;p&gt;&lt;code&gt;inspect_compact&lt;/code&gt; walks the main document, nested accessible frames, and open Shadow DOM roots (&lt;code&gt;el.shadowRoot&lt;/code&gt;), stamps deterministic &lt;code&gt;f&amp;lt;frameIdx&amp;gt;:e&amp;lt;N&amp;gt;&lt;/code&gt; IDs onto &lt;code&gt;data-tbm-ref&lt;/code&gt;, and caps output at &lt;code&gt;maxChars&lt;/code&gt; (&lt;code&gt;4000&lt;/code&gt; by default).&lt;/p&gt;

&lt;p&gt;Instead of dumping a full accessibility tree after every action, the server hashes each line's signature per tab (&lt;code&gt;frameIdx&lt;/code&gt;, tag, label, value/checked state) and supports &lt;code&gt;onlyChanged: true&lt;/code&gt; to return only added or modified controls. Calling &lt;code&gt;act_fast&lt;/code&gt;, &lt;code&gt;batch_actions&lt;/code&gt;, or &lt;code&gt;smart_fill_form&lt;/code&gt; with &lt;code&gt;returnSnapshot: true&lt;/code&gt; folds the updated view into the same tool response so the agent does not spend a separate read turn.&lt;/p&gt;

&lt;h2&gt;
  
  
  2. One-turn &lt;code&gt;smart_fill_form&lt;/code&gt; for React inputs, rich text, and disk uploads
&lt;/h2&gt;

&lt;p&gt;Standard &lt;code&gt;.value = "..."&lt;/code&gt; writes often vanish on React forms on blur or submit because React tracks internal input state via &lt;code&gt;_valueTracker&lt;/code&gt;. In &lt;code&gt;src/dom-engine.mjs&lt;/code&gt;, &lt;code&gt;smart_fill_form&lt;/code&gt; updates inputs through the native &lt;code&gt;HTMLInputElement.prototype&lt;/code&gt; or &lt;code&gt;HTMLTextAreaElement.prototype&lt;/code&gt; value setter after calling &lt;code&gt;_valueTracker.setValue('')&lt;/code&gt;, then dispatches &lt;code&gt;input&lt;/code&gt;, &lt;code&gt;change&lt;/code&gt;, and &lt;code&gt;blur&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;In the same single MCP call, fields can target:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;React/plain &lt;code&gt;&amp;lt;input&amp;gt;&lt;/code&gt; and &lt;code&gt;&amp;lt;textarea&amp;gt;&lt;/code&gt; elements, single/multi &lt;code&gt;&amp;lt;select&amp;gt;&lt;/code&gt; (&lt;code&gt;values&lt;/code&gt;), &lt;code&gt;&amp;lt;input type="range"&amp;gt;&lt;/code&gt; sliders, radios, checkboxes, and ARIA &lt;code&gt;[role="combobox"]&lt;/code&gt; widgets&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;contenteditable&lt;/code&gt; editors (ProseMirror, Quill, compose boxes) via &lt;code&gt;execCommand('insertText')&lt;/code&gt; with &lt;code&gt;InputEvent&lt;/code&gt; fallback&lt;/li&gt;
&lt;li&gt;Nested iframes (&lt;code&gt;frameIdx&lt;/code&gt;) and open Shadow DOM fields&lt;/li&gt;
&lt;li&gt;Local files via &lt;code&gt;file&lt;/code&gt; or &lt;code&gt;files&lt;/code&gt;, which Playwright attaches straight from disk through &lt;code&gt;setInputFiles&lt;/code&gt; or &lt;code&gt;filechooser&lt;/code&gt; (never through inline Base64 strings)&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  3. A default &lt;code&gt;allowSubmit=false&lt;/code&gt; guard that checks coordinates too
&lt;/h2&gt;

&lt;p&gt;By default, &lt;code&gt;resolveAllowSubmit&lt;/code&gt; evaluates to &lt;code&gt;false&lt;/code&gt; unless &lt;code&gt;allowSubmit: true&lt;/code&gt; is passed per call or &lt;code&gt;TURBO_BROWSER_ALLOW_SUBMIT=true&lt;/code&gt; / &lt;code&gt;--allow-submit&lt;/code&gt; is enabled on launch.&lt;/p&gt;

&lt;p&gt;In &lt;code&gt;src/dom-engine.mjs&lt;/code&gt;, &lt;code&gt;SUBMIT_LABEL_RE&lt;/code&gt; blocks clicks, &lt;code&gt;Enter&lt;/code&gt; key presses, and &lt;code&gt;type&lt;/code&gt; &lt;code&gt;submit: true&lt;/code&gt; actions matching &lt;code&gt;submit&lt;/code&gt;, &lt;code&gt;publish&lt;/code&gt;, &lt;code&gt;post&lt;/code&gt;, &lt;code&gt;send&lt;/code&gt;, &lt;code&gt;apply&lt;/code&gt;, &lt;code&gt;confirm and pay&lt;/code&gt;, &lt;code&gt;pay&lt;/code&gt;, &lt;code&gt;checkout&lt;/code&gt;, &lt;code&gt;delete&lt;/code&gt;, and Vietnamese equivalents (&lt;code&gt;gửi&lt;/code&gt;, &lt;code&gt;đăng bài&lt;/code&gt;, &lt;code&gt;nộp đơn&lt;/code&gt;, &lt;code&gt;thanh toán&lt;/code&gt;), plus &lt;code&gt;&amp;lt;input type="submit"&amp;gt;&lt;/code&gt; elements outside an explicit safe list (&lt;code&gt;next&lt;/code&gt;, &lt;code&gt;continue&lt;/code&gt;, &lt;code&gt;search&lt;/code&gt;, &lt;code&gt;filter&lt;/code&gt;, &lt;code&gt;save draft&lt;/code&gt;). Raw viewport &lt;code&gt;{ x, y }&lt;/code&gt; mouse clicks resolve the element at that screen coordinate first so a coordinate click cannot bypass the label guard.&lt;/p&gt;

&lt;h2&gt;
  
  
  4. Local benchmark numbers (&lt;code&gt;npm run benchmark&lt;/code&gt;)
&lt;/h2&gt;

&lt;p&gt;Measured locally on Windows (&lt;code&gt;Node v24.15.0&lt;/code&gt;, Chromium headless) against the self-contained HTTP fixture in &lt;code&gt;benchmarks/benchmark.mjs&lt;/code&gt;:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Scenario&lt;/th&gt;
&lt;th&gt;Baseline&lt;/th&gt;
&lt;th&gt;&lt;code&gt;turbo-browser-mcp&lt;/code&gt;&lt;/th&gt;
&lt;th&gt;Measured change (&lt;code&gt;o200k_base&lt;/code&gt;)&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Dense 140-button SaaS DOM snapshot&lt;/td&gt;
&lt;td&gt;18,654 tok raw HTML / 3,568 tok full ARIA&lt;/td&gt;
&lt;td&gt;1,135 tok compact (28 ms) / 56 tok delta (18 ms)&lt;/td&gt;
&lt;td&gt;93.9% vs HTML, 68.2% vs ARIA, 98.4% on delta&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10-field form fill + snapshot&lt;/td&gt;
&lt;td&gt;11 MCP turns (574 ms, 12,197 dir / 72,614 cum tok)&lt;/td&gt;
&lt;td&gt;1 MCP turn (82 ms, 811 tok, &lt;code&gt;filled=10&lt;/code&gt;)&lt;/td&gt;
&lt;td&gt;11 -&amp;gt; 1 turns, 7x faster, 93.4% dir / 98.9% cum saved&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;48 KiB PDF file upload (&lt;code&gt;49,152&lt;/code&gt; bytes)&lt;/td&gt;
&lt;td&gt;44,728 tok inline Base64 tool arg&lt;/td&gt;
&lt;td&gt;85 tok disk path (68 ms)&lt;/td&gt;
&lt;td&gt;99.8% tokens saved, no Base64 encode&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Open 6 tabs (&lt;code&gt;60 ms&lt;/code&gt; simulated TTFB)&lt;/td&gt;
&lt;td&gt;6 turns (998 ms sequential)&lt;/td&gt;
&lt;td&gt;1 turn (420 ms parallel &lt;code&gt;open_many&lt;/code&gt;)&lt;/td&gt;
&lt;td&gt;6 -&amp;gt; 1 turns, 2.38x faster&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Heavy media load (&lt;code&gt;turbo&lt;/code&gt; vs clean)&lt;/td&gt;
&lt;td&gt;252 ms (&lt;code&gt;304 KiB&lt;/code&gt;, 9 asset hits)&lt;/td&gt;
&lt;td&gt;165 ms (&lt;code&gt;0 KiB&lt;/code&gt;, 0 asset hits)&lt;/td&gt;
&lt;td&gt;1.53x faster, 100% media bandwidth cut&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Article Markdown + Diagnostics errors&lt;/td&gt;
&lt;td&gt;3,595 tok HTML / 3,460 tok all logs&lt;/td&gt;
&lt;td&gt;156 tok Markdown / 601 tok errors-only&lt;/td&gt;
&lt;td&gt;95.7% and 82.6% tokens saved&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The 405 tests across 85 suites (&lt;code&gt;25&lt;/code&gt; test files under &lt;code&gt;test/*.test.mjs&lt;/code&gt;: core integration, DOM/form and session/stdio stress suites, multi-round adversarial simulations, and 10 independent audit suites) run via &lt;code&gt;npm test&lt;/code&gt; and pass on GitHub Actions CI for Node 20 and Node 22.&lt;/p&gt;

&lt;h2&gt;
  
  
  Trade-offs and open gaps
&lt;/h2&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Pre-existing closed Shadow DOM and sandboxed frames&lt;/strong&gt;: An init script hooks &lt;code&gt;Element.prototype.attachShadow&lt;/code&gt; so closed shadow roots created after page initialization stay inspectable and guarded, while closed roots that already existed on an attached CDP tab or live inside strict sandboxed frames fall back to Playwright frame handles, coordinate clicks, and keyboard input.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Real network latency still dominates navigation&lt;/strong&gt;: batching collapses DOM round-trips, while remote page load times still depend on the target server (blocking images, fonts, and media via &lt;code&gt;network_session&lt;/code&gt; preset &lt;code&gt;"turbo"&lt;/code&gt; trims asset overhead on heavy sites).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Long-lived MCP child processes&lt;/strong&gt;: if your client already has an older MCP stdio process running when you pull updates, you need to reload the MCP client so added schemas (like &lt;code&gt;open_many&lt;/code&gt;, &lt;code&gt;onlyChanged&lt;/code&gt;, &lt;code&gt;capture_page&lt;/code&gt;, or &lt;code&gt;network_session&lt;/code&gt;) register.&lt;/li&gt;
&lt;/ol&gt;

&lt;h2&gt;
  
  
  Looking for community feedback
&lt;/h2&gt;

&lt;p&gt;If you use Playwright MCP or browser agents in daily workflows, what breaks most often in practice for you? I am especially curious about custom virtualized dropdowns, tricky iframe embed patterns, diagnostics you miss from Chrome DevTools, or places where &lt;code&gt;SUBMIT_LABEL_RE&lt;/code&gt; would misclassify a safe navigation button.&lt;/p&gt;

&lt;p&gt;Issues, PRs, and edge-case reports are welcome on GitHub:&lt;br&gt;
&lt;a href="https://github.com/Tthanh2k6/turbo-browser-mcp" rel="noopener noreferrer"&gt;https://github.com/Tthanh2k6/turbo-browser-mcp&lt;/a&gt;&lt;/p&gt;

</description>
      <category>mcp</category>
      <category>playwright</category>
      <category>webdev</category>
      <category>opensource</category>
    </item>
    <item>
      <title>Extension Tables, Expo Configs, and Static MCP Scanners in supabase-rls-check 0.6.3</title>
      <dc:creator>Trung Thanh Tran</dc:creator>
      <pubDate>Sat, 14 Nov 2026 02:58:37 +0000</pubDate>
      <link>https://dev.to/tthanh2k6/extension-tables-expo-configs-and-static-mcp-scanners-in-supabase-rls-check-063-4pni</link>
      <guid>https://dev.to/tthanh2k6/extension-tables-expo-configs-and-static-mcp-scanners-in-supabase-rls-check-063-4pni</guid>
      <description>&lt;p&gt;After shipping &lt;code&gt;supabase-rls-check&lt;/code&gt; 0.6.0 on November 11, 2026, I ran into three edge cases across PostgreSQL system catalogs, React Native build files, and Model Context Protocol (MCP) directory scanners. Versions 0.6.1 through 0.6.3 address those three cases without adding network calls to the analyzer.&lt;/p&gt;

&lt;h2&gt;
  
  
  1. Extension-owned tables in &lt;code&gt;public&lt;/code&gt; where &lt;code&gt;ENABLE ROW LEVEL SECURITY&lt;/code&gt; fails
&lt;/h2&gt;

&lt;p&gt;Rule &lt;code&gt;rls_disabled&lt;/code&gt; flags tables in the &lt;code&gt;public&lt;/code&gt; schema that have Row Level Security turned off, because PostgREST exposes the &lt;code&gt;public&lt;/code&gt; schema to the &lt;code&gt;anon&lt;/code&gt; and &lt;code&gt;authenticated&lt;/code&gt; roles by default. For ordinary application tables, the remediation template in &lt;code&gt;fix.sql&lt;/code&gt; suggests enabling RLS:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;ALTER&lt;/span&gt; &lt;span class="k"&gt;TABLE&lt;/span&gt; &lt;span class="n"&gt;public&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;my_table&lt;/span&gt; &lt;span class="no"&gt;ENABLE&lt;/span&gt; &lt;span class="k"&gt;ROW&lt;/span&gt; &lt;span class="no"&gt;LEVEL&lt;/span&gt; &lt;span class="no"&gt;SECURITY&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That remediation fails when a developer enables an extension such as PostGIS inside the &lt;code&gt;public&lt;/code&gt; schema. PostGIS creates &lt;code&gt;public.spatial_ref_sys&lt;/code&gt; owned by the bootstrap superuser (&lt;code&gt;supabase_admin&lt;/code&gt;). When a project owner runs &lt;code&gt;ALTER TABLE public.spatial_ref_sys ENABLE ROW LEVEL SECURITY&lt;/code&gt; in the Supabase SQL Editor, PostgreSQL rejects the statement with &lt;code&gt;ERROR: must be owner of table spatial_ref_sys&lt;/code&gt; (discussed in Supabase GitHub Discussion #52767).&lt;/p&gt;

&lt;p&gt;PostgreSQL records which objects belong to an extension in &lt;code&gt;pg_catalog.pg_depend&lt;/code&gt; with &lt;code&gt;deptype = 'e'&lt;/code&gt;. In version 0.6.1, I updated the &lt;code&gt;rls_disabled&lt;/code&gt; branch of &lt;code&gt;check.sql&lt;/code&gt; to join &lt;code&gt;pg_depend&lt;/code&gt; and &lt;code&gt;pg_extension&lt;/code&gt; through a &lt;code&gt;LEFT JOIN LATERAL&lt;/code&gt; subquery and populate the &lt;code&gt;basis&lt;/code&gt; column when a table belongs to an extension:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="s1"&gt;'rls_disabled'&lt;/span&gt;&lt;span class="o"&gt;::&lt;/span&gt;&lt;span class="kt"&gt;text&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;rule&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;n&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;nspname&lt;/span&gt;&lt;span class="o"&gt;::&lt;/span&gt;&lt;span class="kt"&gt;text&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;schema_name&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
       &lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;relname&lt;/span&gt;&lt;span class="o"&gt;::&lt;/span&gt;&lt;span class="kt"&gt;text&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;object_name&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;''&lt;/span&gt;&lt;span class="o"&gt;::&lt;/span&gt;&lt;span class="kt"&gt;text&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;policy_name&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
       &lt;span class="nb"&gt;coalesce&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'owned by extension '&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="n"&gt;ext&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;extname&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;''&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="o"&gt;::&lt;/span&gt;&lt;span class="kt"&gt;text&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;basis&lt;/span&gt;
&lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="n"&gt;pg_catalog&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;pg_class&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt;
&lt;span class="k"&gt;JOIN&lt;/span&gt; &lt;span class="n"&gt;pg_catalog&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;pg_namespace&lt;/span&gt; &lt;span class="n"&gt;n&lt;/span&gt; &lt;span class="k"&gt;ON&lt;/span&gt; &lt;span class="n"&gt;n&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;oid&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;relnamespace&lt;/span&gt;
&lt;span class="k"&gt;LEFT&lt;/span&gt; &lt;span class="k"&gt;JOIN&lt;/span&gt; &lt;span class="k"&gt;LATERAL&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;
  &lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="n"&gt;e&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;extname&lt;/span&gt;
  &lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="n"&gt;pg_catalog&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;pg_depend&lt;/span&gt; &lt;span class="n"&gt;d&lt;/span&gt;
  &lt;span class="k"&gt;JOIN&lt;/span&gt; &lt;span class="n"&gt;pg_catalog&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;pg_extension&lt;/span&gt; &lt;span class="n"&gt;e&lt;/span&gt; &lt;span class="k"&gt;ON&lt;/span&gt; &lt;span class="n"&gt;e&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;oid&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;d&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;refobjid&lt;/span&gt;
  &lt;span class="k"&gt;WHERE&lt;/span&gt; &lt;span class="n"&gt;d&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;classid&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s1"&gt;'pg_catalog.pg_class'&lt;/span&gt;&lt;span class="o"&gt;::&lt;/span&gt;&lt;span class="n"&gt;pg_catalog&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;regclass&lt;/span&gt;
    &lt;span class="k"&gt;AND&lt;/span&gt; &lt;span class="n"&gt;d&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;objid&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;oid&lt;/span&gt; &lt;span class="k"&gt;AND&lt;/span&gt; &lt;span class="n"&gt;d&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;objsubid&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;
    &lt;span class="k"&gt;AND&lt;/span&gt; &lt;span class="n"&gt;d&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;refclassid&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s1"&gt;'pg_catalog.pg_extension'&lt;/span&gt;&lt;span class="o"&gt;::&lt;/span&gt;&lt;span class="n"&gt;pg_catalog&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;regclass&lt;/span&gt;
    &lt;span class="k"&gt;AND&lt;/span&gt; &lt;span class="n"&gt;d&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;deptype&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s1"&gt;'e'&lt;/span&gt;
  &lt;span class="k"&gt;ORDER&lt;/span&gt; &lt;span class="k"&gt;BY&lt;/span&gt; &lt;span class="n"&gt;e&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;extname&lt;/span&gt;
  &lt;span class="k"&gt;LIMIT&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="n"&gt;ext&lt;/span&gt; &lt;span class="k"&gt;ON&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;
&lt;span class="k"&gt;WHERE&lt;/span&gt; &lt;span class="n"&gt;n&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;nspname&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s1"&gt;'public'&lt;/span&gt; &lt;span class="k"&gt;AND&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;relkind&lt;/span&gt; &lt;span class="k"&gt;IN&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'r'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'p'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;AND&lt;/span&gt; &lt;span class="k"&gt;NOT&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;relrowsecurity&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;When &lt;code&gt;basis&lt;/code&gt; starts with &lt;code&gt;owned by extension&lt;/code&gt;, &lt;code&gt;fix.sql&lt;/code&gt; and the HTML and Markdown reports skip the &lt;code&gt;ENABLE ROW LEVEL SECURITY&lt;/code&gt; statement. Instead, they recommend moving the extension into a dedicated &lt;code&gt;extensions&lt;/code&gt; schema (&lt;code&gt;CREATE SCHEMA IF NOT EXISTS extensions; ALTER EXTENSION postgis SET SCHEMA extensions;&lt;/code&gt;) or revoking API role grants if the extension does not support &lt;code&gt;SET SCHEMA&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Because &lt;code&gt;basis&lt;/code&gt; is part of the 5-tuple identity key used by &lt;code&gt;supabase-rls-check diff&lt;/code&gt;, a baseline exported before 0.6.1 records &lt;code&gt;spatial_ref_sys&lt;/code&gt; with an empty &lt;code&gt;basis&lt;/code&gt;. Re-exporting the baseline once updates the key so subsequent CI runs stay quiet.&lt;/p&gt;

&lt;h2&gt;
  
  
  2. Scanning Expo config files and skipping password placeholders
&lt;/h2&gt;

&lt;p&gt;Mobile teams using Supabase with React Native and Expo often store configuration values inside &lt;code&gt;app.json&lt;/code&gt;, &lt;code&gt;app.config.js&lt;/code&gt;, &lt;code&gt;app.config.ts&lt;/code&gt;, or &lt;code&gt;eas.json&lt;/code&gt; build profiles. Every key placed under &lt;code&gt;expo.extra&lt;/code&gt; in &lt;code&gt;app.json&lt;/code&gt; or &lt;code&gt;env&lt;/code&gt; in &lt;code&gt;eas.json&lt;/code&gt; can end up bundled into the client binary.&lt;/p&gt;

&lt;p&gt;In version 0.6.1, I expanded &lt;code&gt;supabase-rls-check scan-env&lt;/code&gt; to check &lt;code&gt;app.json&lt;/code&gt;, &lt;code&gt;app.config.*&lt;/code&gt;, and &lt;code&gt;eas.json&lt;/code&gt; for hardcoded &lt;code&gt;service_role&lt;/code&gt; JWTs and &lt;code&gt;sb_secret_&lt;/code&gt; keys under &lt;code&gt;service_role_in_client_code&lt;/code&gt;. I also added the Critical rule &lt;code&gt;db_url_in_client_code&lt;/code&gt;, which checks client source files and mobile config files for &lt;code&gt;postgresql://&lt;/code&gt; or &lt;code&gt;postgres://&lt;/code&gt; connection strings containing a password. The scanner skips build output directories such as &lt;code&gt;web-build&lt;/code&gt;, &lt;code&gt;android/app/build&lt;/code&gt;, and &lt;code&gt;ios/build&lt;/code&gt;, and it never flags &lt;code&gt;anon&lt;/code&gt; or &lt;code&gt;sb_publishable_&lt;/code&gt; keys.&lt;/p&gt;

&lt;p&gt;Adding connection string checks exposed a false positive in coding agent configuration files (&lt;code&gt;.cursor/mcp.json&lt;/code&gt;, &lt;code&gt;.mcp.json&lt;/code&gt;, and &lt;code&gt;.claude/settings.json&lt;/code&gt;). Developers who commit shared MCP configurations often pass environment variable placeholders in the URL:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nl"&gt;"mcpServers"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nl"&gt;"postgres"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
      &lt;span class="nl"&gt;"command"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"npx"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
      &lt;span class="nl"&gt;"args"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"-y"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"@modelcontextprotocol/server-postgres"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"postgresql://postgres:${DB_PASSWORD}@db.example.supabase.co:5432/postgres"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Version 0.6.1 checks the extracted password segment before reporting a finding. If the password is an unexpanded placeholder such as &lt;code&gt;${DB_PASSWORD}&lt;/code&gt; or &lt;code&gt;$DB_PASSWORD&lt;/code&gt;, the scanner ignores it, while real inline passwords stay masked and reported as Critical findings.&lt;/p&gt;

&lt;h2&gt;
  
  
  3. Making MCP tools visible to static directory scanners
&lt;/h2&gt;

&lt;p&gt;&lt;code&gt;supabase-rls-check&lt;/code&gt; ships an optional stdio MCP server (&lt;code&gt;pip install "supabase-rls-check[mcp]"&lt;/code&gt;) with four local tools: &lt;code&gt;analyze_sql_export&lt;/code&gt;, &lt;code&gt;explain_rule&lt;/code&gt;, &lt;code&gt;scan_public_env&lt;/code&gt;, and &lt;code&gt;diff_results&lt;/code&gt;. None of these tools accept database credentials or make network requests.&lt;/p&gt;

&lt;p&gt;When I submitted the server to MCP directories and ran static trust checkers (such as MCP Trust Checker on mcprush), two packaging issues surfaced:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Directories using the newer MCP specification check every tool for four boolean safety hints (&lt;code&gt;readOnlyHint&lt;/code&gt;, &lt;code&gt;destructiveHint&lt;/code&gt;, &lt;code&gt;idempotentHint&lt;/code&gt;, and &lt;code&gt;openWorldHint&lt;/code&gt;) alongside a human-readable &lt;code&gt;title&lt;/code&gt;. Tools missing those annotations receive warnings or lower trust scores because clients cannot tell whether a tool mutates external state.&lt;/li&gt;
&lt;li&gt;Static AST scanners look for direct &lt;code&gt;server.add_tool(...)&lt;/code&gt; calls or &lt;code&gt;@server.tool()&lt;/code&gt; decorators. Earlier versions registered tools in a loop using the curried call &lt;code&gt;server.tool()(fn)&lt;/code&gt;, which caused static analyzers to report zero discovered tools even though the server worked at runtime.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;In versions 0.6.2 and 0.6.3, I bumped the optional &lt;code&gt;mcp&lt;/code&gt; extra to &lt;code&gt;mcp&amp;gt;=1.10&lt;/code&gt;, attached &lt;code&gt;ToolAnnotations&lt;/code&gt; to all four tools, and switched registration to &lt;code&gt;server.add_tool&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;_tool_annotations&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;title&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;str&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nc"&gt;Any&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="s2"&gt;"""All four tools only read local input: read-only, non-destructive, idempotent, closed-world."""&lt;/span&gt;
    &lt;span class="n"&gt;hints&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kt"&gt;dict&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="n"&gt;title&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;title&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;readOnlyHint&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="kc"&gt;True&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;destructiveHint&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="kc"&gt;False&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;idempotentHint&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="kc"&gt;True&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;openWorldHint&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="kc"&gt;False&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;try&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;mcp&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;types&lt;/span&gt; &lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="nc"&gt;ToolAnnotations&lt;/span&gt;
    &lt;span class="k"&gt;except&lt;/span&gt; &lt;span class="nc"&gt;ImportError&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;hints&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nc"&gt;ToolAnnotations&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;**&lt;/span&gt;&lt;span class="n"&gt;hints&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;


&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;create_mcp_server&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;fastmcp_cls&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;Any&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nc"&gt;Any&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="c1"&gt;# ...&lt;/span&gt;
    &lt;span class="n"&gt;server&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;fastmcp_cls&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'supabase-rls-check'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;fn&lt;/span&gt; &lt;span class="nf"&gt;in&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;analyze_sql_export&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;explain_rule&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;scan_public_env&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;diff_results&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;title&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="no"&gt;MCP_TOOL_TITLES&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;fn&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;__name__&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
        &lt;span class="n"&gt;server&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;add_tool&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;fn&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;title&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;title&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;annotations&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="nf"&gt;_tool_annotations&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;title&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;server&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;A unit test in &lt;code&gt;tests/test_mcp_server.py&lt;/code&gt; now inspects the registered tools and asserts all four boolean hints on each one.&lt;/p&gt;

&lt;h2&gt;
  
  
  Boundaries of the checks
&lt;/h2&gt;

&lt;p&gt;Extension detection in &lt;code&gt;check.sql&lt;/code&gt; only covers tables directly registered in &lt;code&gt;pg_depend&lt;/code&gt; with &lt;code&gt;deptype = 'e'&lt;/code&gt;, and some PostgreSQL extensions do not support &lt;code&gt;ALTER EXTENSION ... SET SCHEMA&lt;/code&gt;. Client and config file secret scanning relies on regular expressions over file text, so it will miss keys assembled dynamically at runtime or split across multiple variables. Static catalog and config checks help catch common mistakes early, and they should be paired with integration tests that sign in as two separate tenants and verify row isolation against a real database.&lt;/p&gt;

&lt;h2&gt;
  
  
  Trying version 0.6.3
&lt;/h2&gt;

&lt;p&gt;You can install or upgrade &lt;code&gt;supabase-rls-check&lt;/code&gt; from PyPI:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;pipx install supabase-rls-check
supabase-rls-check --version
supabase-rls-check sql --print-sql
supabase-rls-check scan-env .
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The source code, PostgreSQL 15 test fixtures (32 catalog findings and 133 unit tests), and changelog are on GitHub at &lt;a href="https://github.com/Tthanh2k6/supabase-rls-check" rel="noopener noreferrer"&gt;github.com/Tthanh2k6/supabase-rls-check&lt;/a&gt;.&lt;/p&gt;

</description>
      <category>supabase</category>
      <category>postgres</category>
      <category>security</category>
      <category>mcp</category>
    </item>
    <item>
      <title>Reader Comments Shaped Three Releases of supabase-rls-check</title>
      <dc:creator>Trung Thanh Tran</dc:creator>
      <pubDate>Wed, 11 Nov 2026 05:23:46 +0000</pubDate>
      <link>https://dev.to/tthanh2k6/reader-comments-shaped-three-releases-of-supabase-rls-check-hi3</link>
      <guid>https://dev.to/tthanh2k6/reader-comments-shaped-three-releases-of-supabase-rls-check-hi3</guid>
      <description>&lt;p&gt;On November 10, 2026, I published "Five Supabase RLS Mistakes a Read-Only Query Can Catch" on dev.to. That post introduced &lt;code&gt;supabase-rls-check&lt;/code&gt;, my open-source Python CLI and read-only PostgreSQL 15+ catalog query (&lt;code&gt;check.sql&lt;/code&gt;) that inspects &lt;code&gt;pg_catalog&lt;/code&gt; and &lt;code&gt;storage.buckets&lt;/code&gt; for Row Level Security mistakes.&lt;/p&gt;

&lt;p&gt;Readers began posting comments with edge cases from their own schemas. Within about a day, I shipped versions 0.4.0 and 0.5.0 on November 10, then followed up with version 0.6.0 on November 11. Those releases brought &lt;code&gt;check.sql&lt;/code&gt; from 13 rules to 17. They also added run-to-run diffing for CI and local secret scanning for coding agent config files.&lt;/p&gt;

&lt;h2&gt;
  
  
  Policies that allow any signed-in user
&lt;/h2&gt;

&lt;p&gt;In the discussion on the November 10 post, sunnatulla_dev pointed out a policy pattern that passes basic RLS enablement checks. A developer enables RLS on a table in &lt;code&gt;public&lt;/code&gt; and writes a policy that verifies the caller has a session, using &lt;code&gt;auth.role() = 'authenticated'&lt;/code&gt; or &lt;code&gt;auth.uid() IS NOT NULL&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;In a Supabase project with open sign-ups, any person who creates an account satisfies that check and can access all rows covered by the policy because the expression omits a row ownership check against a column like &lt;code&gt;user_id&lt;/code&gt;. In version 0.4.0, I added the &lt;code&gt;policy_any_authenticated&lt;/code&gt; rule at Medium severity for &lt;code&gt;USING&lt;/code&gt; and &lt;code&gt;WITH CHECK&lt;/code&gt; clauses on tables in &lt;code&gt;public&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;PostgreSQL stores policy expressions inside &lt;code&gt;pg_catalog.pg_policies&lt;/code&gt;. Because developers wrap &lt;code&gt;auth.uid()&lt;/code&gt; in &lt;code&gt;(SELECT ...)&lt;/code&gt; for planner caching and Postgres adds &lt;code&gt;::text&lt;/code&gt; casts or aliases when deparsing SQL, &lt;code&gt;check.sql&lt;/code&gt; strips those wrappers before comparing the normalized string. This simplified query from &lt;code&gt;check.sql&lt;/code&gt; shows the core check:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="c1"&gt;-- Simplified from check.sql (the full query also strips SELECT wrappers, aliases, and ::text casts)&lt;/span&gt;
&lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="s1"&gt;'policy_any_authenticated'&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;rule&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;schemaname&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;tablename&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;policyname&lt;/span&gt;
&lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="n"&gt;pg_catalog&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;pg_policies&lt;/span&gt;
&lt;span class="k"&gt;WHERE&lt;/span&gt; &lt;span class="n"&gt;schemaname&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s1"&gt;'public'&lt;/span&gt;
  &lt;span class="k"&gt;AND&lt;/span&gt; &lt;span class="nb"&gt;lower&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nf"&gt;regexp_replace&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;coalesce&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;qual&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;''&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'[()[:space:]]'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;''&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'g'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
      &lt;span class="k"&gt;IN&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'auth.role=''authenticated'''&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'auth.uidisnotnull'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Writable USING (true) policies paired with table grants
&lt;/h2&gt;

&lt;p&gt;Earlier versions already had a &lt;code&gt;policy_true&lt;/code&gt; rule that flagged any policy in &lt;code&gt;public&lt;/code&gt; where &lt;code&gt;USING&lt;/code&gt; or &lt;code&gt;WITH CHECK&lt;/code&gt; is just &lt;code&gt;true&lt;/code&gt;. A LinkedIn reader pointed at the combination that does the most damage: &lt;code&gt;USING (true)&lt;/code&gt; on an &lt;code&gt;UPDATE&lt;/code&gt; or &lt;code&gt;DELETE&lt;/code&gt; policy while the client roles also hold write grants on the table. Anyone with the anon key can then change or delete every row.&lt;/p&gt;

&lt;p&gt;In version 0.5.0, I shipped &lt;code&gt;policy_true_writable&lt;/code&gt; as a Critical rule. The full query in &lt;code&gt;check.sql&lt;/code&gt; checks &lt;code&gt;pg_catalog.pg_policies&lt;/code&gt; for permissive &lt;code&gt;UPDATE&lt;/code&gt; or &lt;code&gt;DELETE&lt;/code&gt; policies (and &lt;code&gt;ALL&lt;/code&gt; policies) in &lt;code&gt;public&lt;/code&gt; where the normalized &lt;code&gt;USING&lt;/code&gt; clause is &lt;code&gt;true&lt;/code&gt; for &lt;code&gt;anon&lt;/code&gt; or &lt;code&gt;authenticated&lt;/code&gt; (or &lt;code&gt;public&lt;/code&gt;). It then calls &lt;code&gt;pg_catalog.has_table_privilege&lt;/code&gt; to confirm that &lt;code&gt;anon&lt;/code&gt; or &lt;code&gt;authenticated&lt;/code&gt; holds &lt;code&gt;UPDATE&lt;/code&gt; or &lt;code&gt;DELETE&lt;/code&gt; grants on that table. This simplified version of the &lt;code&gt;check.sql&lt;/code&gt; rule illustrates the join:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="c1"&gt;-- Simplified from check.sql&lt;/span&gt;
&lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="s1"&gt;'policy_true_writable'&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;rule&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;p&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;schemaname&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;p&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;tablename&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;p&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;policyname&lt;/span&gt;
&lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="n"&gt;pg_catalog&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;pg_policies&lt;/span&gt; &lt;span class="n"&gt;p&lt;/span&gt;
&lt;span class="k"&gt;JOIN&lt;/span&gt; &lt;span class="n"&gt;pg_catalog&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;pg_class&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt; &lt;span class="k"&gt;ON&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;relname&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;p&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;tablename&lt;/span&gt;
&lt;span class="k"&gt;WHERE&lt;/span&gt; &lt;span class="n"&gt;p&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;schemaname&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s1"&gt;'public'&lt;/span&gt; &lt;span class="k"&gt;AND&lt;/span&gt; &lt;span class="n"&gt;p&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;permissive&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s1"&gt;'PERMISSIVE'&lt;/span&gt;
  &lt;span class="k"&gt;AND&lt;/span&gt; &lt;span class="n"&gt;p&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;cmd&lt;/span&gt; &lt;span class="k"&gt;IN&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'UPDATE'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'DELETE'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'ALL'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
  &lt;span class="k"&gt;AND&lt;/span&gt; &lt;span class="nf"&gt;regexp_replace&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;coalesce&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;p&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;qual&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;''&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'[()[:space:]]'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;''&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'g'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s1"&gt;'true'&lt;/span&gt;
  &lt;span class="k"&gt;AND&lt;/span&gt; &lt;span class="n"&gt;pg_catalog&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;has_table_privilege&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'anon'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;oid&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'UPDATE'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Diffing runs in CI and putting the severity basis into the finding key
&lt;/h2&gt;

&lt;p&gt;Reader launchgatecheck contributed two ideas across versions 0.5.0 and 0.6.0. First, launchgatecheck suggested running &lt;code&gt;supabase-rls-check&lt;/code&gt; in CI and diffing the output between runs so teams with existing findings can block new problems in pull requests while fixing older warnings over time. In version 0.5.0, I added the &lt;code&gt;diff&lt;/code&gt; CLI command and a &lt;code&gt;diff_results(old_path, new_path)&lt;/code&gt; MCP tool to compare two local CSV or JSON exports and report new or resolved findings with their severities. Passing &lt;code&gt;--fail-on-new&lt;/code&gt; fails the CI job if a new finding reaches your chosen severity threshold.&lt;/p&gt;

&lt;p&gt;After 0.5.0 shipped, launchgatecheck asked what &lt;code&gt;diff&lt;/code&gt; does when a finding's severity basis changes between runs without the finding itself appearing or disappearing. In version 0.3.0, following an earlier dev.to reader comment, I had updated &lt;code&gt;sensitive_anon_policy&lt;/code&gt; to check for a &lt;code&gt;pii&lt;/code&gt; table comment tag (&lt;code&gt;COMMENT ON TABLE public.x IS 'pii'&lt;/code&gt;) alongside its table-name regular expression. Keying findings by &lt;code&gt;(rule, schema_name, object_name, policy_name)&lt;/code&gt; meant a finding whose underlying reason changed or escalated between runs kept the same identity key.&lt;/p&gt;

&lt;p&gt;In version 0.6.0, I added an optional &lt;code&gt;basis&lt;/code&gt; column to &lt;code&gt;check.sql&lt;/code&gt; and expanded the &lt;code&gt;diff&lt;/code&gt; identity key to &lt;code&gt;(rule, schema_name, object_name, policy_name, basis)&lt;/code&gt; while keeping backward compatibility with older JSON files. If a finding's trigger reason changes, &lt;code&gt;diff&lt;/code&gt; reports the old finding as resolved and flags the new basis as a fresh finding. In &lt;code&gt;check.sql&lt;/code&gt;, &lt;code&gt;sensitive_anon_policy&lt;/code&gt; builds &lt;code&gt;basis&lt;/code&gt; with a &lt;code&gt;CASE&lt;/code&gt; expression:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="c1"&gt;-- Simplified basis column from sensitive_anon_policy in check.sql&lt;/span&gt;
&lt;span class="k"&gt;CASE&lt;/span&gt;
  &lt;span class="k"&gt;WHEN&lt;/span&gt; &lt;span class="n"&gt;pg_catalog&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;obj_description&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;oid&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'pg_class'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;~*&lt;/span&gt; &lt;span class="s1"&gt;'\mpii\M'&lt;/span&gt;
    &lt;span class="k"&gt;THEN&lt;/span&gt; &lt;span class="s1"&gt;'table comment tag ''pii'''&lt;/span&gt;
  &lt;span class="k"&gt;ELSE&lt;/span&gt; &lt;span class="s1"&gt;'table name matches pattern '''&lt;/span&gt;
    &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="nb"&gt;substring&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;lower&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;p&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;tablename&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="s1"&gt;'(user|profile|order|payment|invoice|customer|patient|secret|token|document)'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="s1"&gt;''''&lt;/span&gt;
&lt;span class="k"&gt;END&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;basis&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  SECURITY DEFINER functions that trust a user_id argument
&lt;/h2&gt;

&lt;p&gt;Another reader, avivgerzhue, brought up &lt;code&gt;SECURITY DEFINER&lt;/code&gt; RPC functions that trust an identifier argument from the caller. Because a &lt;code&gt;SECURITY DEFINER&lt;/code&gt; function runs with the privileges of its owner, queries inside the function bypass RLS on underlying tables. If a function executable by &lt;code&gt;anon&lt;/code&gt; or &lt;code&gt;authenticated&lt;/code&gt; takes a &lt;code&gt;user_id&lt;/code&gt; or &lt;code&gt;tenant_id&lt;/code&gt; argument without checking &lt;code&gt;auth.uid()&lt;/code&gt; or &lt;code&gt;auth.jwt()&lt;/code&gt;, a client can call &lt;code&gt;/rest/v1/rpc&lt;/code&gt; with another user's UUID.&lt;/p&gt;

&lt;p&gt;In version 0.6.0, I added &lt;code&gt;definer_trusts_id_arg&lt;/code&gt; at Medium severity and marked it for human review. In &lt;code&gt;check.sql&lt;/code&gt;, the query inspects &lt;code&gt;pg_catalog.pg_proc&lt;/code&gt; for non-trigger &lt;code&gt;SECURITY DEFINER&lt;/code&gt; functions in &lt;code&gt;public&lt;/code&gt; executable by &lt;code&gt;anon&lt;/code&gt; or &lt;code&gt;authenticated&lt;/code&gt;. It filters input arguments in &lt;code&gt;proargnames&lt;/code&gt; and &lt;code&gt;proargmodes&lt;/code&gt; against &lt;code&gt;(_id$|^uid$|^user$|^org$|^tenant$)&lt;/code&gt; for the &lt;code&gt;basis&lt;/code&gt; column, and it checks that &lt;code&gt;prosrc&lt;/code&gt; omits &lt;code&gt;auth.uid()&lt;/code&gt; and &lt;code&gt;auth.jwt()&lt;/code&gt;. This simplified query from &lt;code&gt;check.sql&lt;/code&gt; shows the check:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="c1"&gt;-- Simplified from check.sql&lt;/span&gt;
&lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="s1"&gt;'definer_trusts_id_arg'&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;rule&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;p&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;proname&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;p&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;proargnames&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;basis&lt;/span&gt;
&lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="n"&gt;pg_catalog&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;pg_proc&lt;/span&gt; &lt;span class="n"&gt;p&lt;/span&gt;
&lt;span class="k"&gt;WHERE&lt;/span&gt; &lt;span class="n"&gt;p&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;prosecdef&lt;/span&gt; &lt;span class="k"&gt;AND&lt;/span&gt; &lt;span class="n"&gt;p&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;prokind&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s1"&gt;'f'&lt;/span&gt;
  &lt;span class="k"&gt;AND&lt;/span&gt; &lt;span class="nf"&gt;array_to_string&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;p&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;proargnames&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;','&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;~*&lt;/span&gt; &lt;span class="s1"&gt;'(_id|uid|user|org|tenant)'&lt;/span&gt;
  &lt;span class="k"&gt;AND&lt;/span&gt; &lt;span class="n"&gt;p&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;prosrc&lt;/span&gt; &lt;span class="o"&gt;!~*&lt;/span&gt; &lt;span class="s1"&gt;'\mauth[[:space:]]*\.[[:space:]]*(uid|jwt)[[:space:]]*\([[:space:]]*\)'&lt;/span&gt;
  &lt;span class="k"&gt;AND&lt;/span&gt; &lt;span class="n"&gt;pg_catalog&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;has_function_privilege&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'authenticated'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;p&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;oid&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'EXECUTE'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Because &lt;code&gt;definer_trusts_id_arg&lt;/code&gt; relies on argument names and a regular expression over &lt;code&gt;prosrc&lt;/code&gt;, it has false positives and false negatives and requires human review. A function that accepts a public record identifier such as &lt;code&gt;product_id&lt;/code&gt; or delegates caller checks to a helper function will produce a false positive. A function that names its caller parameter &lt;code&gt;caller_account&lt;/code&gt; or mentions &lt;code&gt;auth.uid()&lt;/code&gt; inside a comment while still trusting the input argument will produce a false negative.&lt;/p&gt;

&lt;h2&gt;
  
  
  Exposed secret columns and coding agent config files
&lt;/h2&gt;

&lt;p&gt;Version 0.6.0 added two more checks for leaked secrets. In &lt;code&gt;check.sql&lt;/code&gt;, &lt;code&gt;secret_column_exposed&lt;/code&gt; (Medium) checks tables in &lt;code&gt;public&lt;/code&gt; for columns matching &lt;code&gt;api_key&lt;/code&gt;, &lt;code&gt;apikey&lt;/code&gt;, &lt;code&gt;secret&lt;/code&gt;, &lt;code&gt;client_secret&lt;/code&gt;, &lt;code&gt;access_token&lt;/code&gt;, &lt;code&gt;refresh_token&lt;/code&gt;, &lt;code&gt;private_key&lt;/code&gt;, or &lt;code&gt;password&lt;/code&gt; (exact or with an underscore prefix) where &lt;code&gt;anon&lt;/code&gt; or &lt;code&gt;authenticated&lt;/code&gt; holds table or column &lt;code&gt;SELECT&lt;/code&gt; grants, putting the column name into &lt;code&gt;basis&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="c1"&gt;-- Simplified from check.sql&lt;/span&gt;
&lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="s1"&gt;'secret_column_exposed'&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;rule&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;relname&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;object_name&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;a&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;attname&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;basis&lt;/span&gt;
&lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="n"&gt;pg_catalog&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;pg_class&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt;
&lt;span class="k"&gt;JOIN&lt;/span&gt; &lt;span class="n"&gt;pg_catalog&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;pg_attribute&lt;/span&gt; &lt;span class="n"&gt;a&lt;/span&gt; &lt;span class="k"&gt;ON&lt;/span&gt; &lt;span class="n"&gt;a&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;attrelid&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;oid&lt;/span&gt;
&lt;span class="k"&gt;WHERE&lt;/span&gt; &lt;span class="n"&gt;a&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;attnum&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt; &lt;span class="k"&gt;AND&lt;/span&gt; &lt;span class="k"&gt;NOT&lt;/span&gt; &lt;span class="n"&gt;a&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;attisdropped&lt;/span&gt;
  &lt;span class="k"&gt;AND&lt;/span&gt; &lt;span class="n"&gt;a&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;attname&lt;/span&gt; &lt;span class="o"&gt;~*&lt;/span&gt; &lt;span class="s1"&gt;'(^|_)(api_key|apikey|secret|client_secret|access_token|refresh_token|private_key|password)$'&lt;/span&gt;
  &lt;span class="k"&gt;AND&lt;/span&gt; &lt;span class="n"&gt;pg_catalog&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;has_column_privilege&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'anon'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;oid&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;a&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;attnum&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'SELECT'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;In &lt;code&gt;scan-env&lt;/code&gt; and the &lt;code&gt;scan_public_env(path)&lt;/code&gt; MCP tool, I added &lt;code&gt;agent_config_secret&lt;/code&gt; (Critical). This check followed a November 2026 Ars Technica report about Claude Code settings files leaking secrets on GitHub. Running &lt;code&gt;supabase-rls-check scan-env .&lt;/code&gt; scans &lt;code&gt;.claude/settings.local.json&lt;/code&gt; and &lt;code&gt;.claude/settings.json&lt;/code&gt; along with &lt;code&gt;.cursor/mcp.json&lt;/code&gt; and &lt;code&gt;.mcp.json&lt;/code&gt; (plus &lt;code&gt;.vscode/mcp.json&lt;/code&gt;) for &lt;code&gt;sb_secret_&lt;/code&gt; keys and &lt;code&gt;service_role&lt;/code&gt; JWTs, as well as PostgreSQL connection strings with inline passwords, masking all secret values.&lt;/p&gt;

&lt;h2&gt;
  
  
  Limits of the tool
&lt;/h2&gt;

&lt;p&gt;You should keep the boundaries of static catalog analysis in mind when reviewing a report. The queries in &lt;code&gt;check.sql&lt;/code&gt; read catalog metadata from &lt;code&gt;pg_catalog&lt;/code&gt; and &lt;code&gt;storage.buckets&lt;/code&gt; without reading table data, so the tool cannot see row contents or business context. Several rules depend on heuristics such as regular expressions over identifier names and function bodies, which means custom naming can miss a rule and safe code can trigger a warning that needs manual inspection. Running a catalog check and a config scanner does not replace a penetration test or an application security review.&lt;/p&gt;

&lt;h2&gt;
  
  
  Trying it on your project
&lt;/h2&gt;

&lt;p&gt;You can install &lt;code&gt;supabase-rls-check&lt;/code&gt; with &lt;code&gt;pipx&lt;/code&gt; and print the bundled read-only SQL query to inspect or run in the Supabase SQL Editor. Once you export two runs as CSV or JSON, you can compare them with &lt;code&gt;diff&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;pipx install supabase-rls-check
supabase-rls-check sql --print-sql
supabase-rls-check diff --old a.json --new b.json --fail-on-new high
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Do you have another Supabase RLS or Postgres privilege mistake in your review checklist that a read-only catalog query could catch?&lt;/p&gt;

</description>
      <category>supabase</category>
      <category>postgres</category>
      <category>security</category>
      <category>opensource</category>
    </item>
    <item>
      <title>Five Supabase RLS Mistakes a Read-Only Query Can Catch</title>
      <dc:creator>Trung Thanh Tran</dc:creator>
      <pubDate>Tue, 10 Nov 2026 06:17:41 +0000</pubDate>
      <link>https://dev.to/tthanh2k6/five-supabase-rls-mistakes-a-read-only-query-can-catch-100p</link>
      <guid>https://dev.to/tthanh2k6/five-supabase-rls-mistakes-a-read-only-query-can-catch-100p</guid>
      <description>&lt;p&gt;Supabase gives every app a public &lt;code&gt;anon&lt;/code&gt; key, and that key can talk to your database through the REST API. Row Level Security is the only thing standing between that key and your tables. When an app gets built fast, sometimes with an AI coding tool writing the migrations, RLS is often on but written in a way that still lets one user read another user's rows. CVE-2025-48757 was exactly this class of bug, across a large number of generated apps.&lt;/p&gt;

&lt;p&gt;Most of these mistakes are visible in Postgres's own catalog. You can find them with a &lt;code&gt;SELECT&lt;/code&gt;, without touching a single row of user data. Here are five, with the query that spots each one and the fix.&lt;/p&gt;

&lt;h2&gt;
  
  
  1. RLS is simply off
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;select&lt;/span&gt; &lt;span class="n"&gt;n&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;nspname&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="k"&gt;schema&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;relname&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;table_name&lt;/span&gt;
&lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;pg_class&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt;
&lt;span class="k"&gt;join&lt;/span&gt; &lt;span class="n"&gt;pg_namespace&lt;/span&gt; &lt;span class="n"&gt;n&lt;/span&gt; &lt;span class="k"&gt;on&lt;/span&gt; &lt;span class="n"&gt;n&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;oid&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;relnamespace&lt;/span&gt;
&lt;span class="k"&gt;where&lt;/span&gt; &lt;span class="n"&gt;n&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;nspname&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s1"&gt;'public'&lt;/span&gt;
  &lt;span class="k"&gt;and&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;relkind&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'r'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'p'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
  &lt;span class="k"&gt;and&lt;/span&gt; &lt;span class="k"&gt;not&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;relrowsecurity&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Any row this returns is a table the &lt;code&gt;anon&lt;/code&gt; key can read and write in full, as long as the role has the usual grants. Fix:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;alter&lt;/span&gt; &lt;span class="k"&gt;table&lt;/span&gt; &lt;span class="n"&gt;public&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;profiles&lt;/span&gt; &lt;span class="n"&gt;enable&lt;/span&gt; &lt;span class="k"&gt;row&lt;/span&gt; &lt;span class="n"&gt;level&lt;/span&gt; &lt;span class="n"&gt;security&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Then add policies. With RLS on and no policies, the table returns nothing, which is safe but breaks the app, so people sometimes "fix" it with the next mistake.&lt;/p&gt;

&lt;h2&gt;
  
  
  2. A policy that is just &lt;code&gt;true&lt;/code&gt;
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;create&lt;/span&gt; &lt;span class="n"&gt;policy&lt;/span&gt; &lt;span class="s2"&gt;"anyone can read"&lt;/span&gt; &lt;span class="k"&gt;on&lt;/span&gt; &lt;span class="n"&gt;public&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;profiles&lt;/span&gt;
  &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="k"&gt;select&lt;/span&gt; &lt;span class="k"&gt;using&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This turns RLS back into "everyone sees everything". Find them:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;select&lt;/span&gt; &lt;span class="n"&gt;schemaname&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;tablename&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;policyname&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;cmd&lt;/span&gt;
&lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;pg_policies&lt;/span&gt;
&lt;span class="k"&gt;where&lt;/span&gt; &lt;span class="n"&gt;schemaname&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s1"&gt;'public'&lt;/span&gt;
  &lt;span class="k"&gt;and&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;qual&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s1"&gt;'true'&lt;/span&gt; &lt;span class="k"&gt;or&lt;/span&gt; &lt;span class="n"&gt;with_check&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s1"&gt;'true'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Sometimes this is intentional (a public list of blog posts). On &lt;code&gt;profiles&lt;/code&gt;, &lt;code&gt;orders&lt;/code&gt; or anything with personal data it usually isn't. The fix ties the row to the caller:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;create&lt;/span&gt; &lt;span class="n"&gt;policy&lt;/span&gt; &lt;span class="s2"&gt;"read own profile"&lt;/span&gt; &lt;span class="k"&gt;on&lt;/span&gt; &lt;span class="n"&gt;public&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;profiles&lt;/span&gt;
  &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="k"&gt;select&lt;/span&gt; &lt;span class="k"&gt;using&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;select&lt;/span&gt; &lt;span class="n"&gt;auth&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;uid&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;user_id&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  3. Trusting &lt;code&gt;user_metadata&lt;/code&gt; for roles
&lt;/h2&gt;

&lt;p&gt;This one looks reasonable and is not:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;create&lt;/span&gt; &lt;span class="n"&gt;policy&lt;/span&gt; &lt;span class="s2"&gt;"admins read all"&lt;/span&gt; &lt;span class="k"&gt;on&lt;/span&gt; &lt;span class="n"&gt;public&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;orders&lt;/span&gt;
  &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="k"&gt;select&lt;/span&gt; &lt;span class="k"&gt;using&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;auth&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;jwt&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'user_metadata'&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'role'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s1"&gt;'admin'&lt;/span&gt;
  &lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;user_metadata&lt;/code&gt; is editable by the signed-in user. Any user can call &lt;code&gt;supabase.auth.updateUser({ data: { role: 'admin' } })&lt;/code&gt;, refresh their session, and the policy now lets them read every order. Supabase's docs call this out, but it still shows up because the field name sounds authoritative.&lt;/p&gt;

&lt;p&gt;Find policies that read it:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;select&lt;/span&gt; &lt;span class="n"&gt;schemaname&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;tablename&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;policyname&lt;/span&gt;
&lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;pg_policies&lt;/span&gt;
&lt;span class="k"&gt;where&lt;/span&gt; &lt;span class="n"&gt;schemaname&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s1"&gt;'public'&lt;/span&gt;
  &lt;span class="k"&gt;and&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;qual&lt;/span&gt; &lt;span class="k"&gt;ilike&lt;/span&gt; &lt;span class="s1"&gt;'%user_metadata%'&lt;/span&gt; &lt;span class="k"&gt;or&lt;/span&gt; &lt;span class="n"&gt;with_check&lt;/span&gt; &lt;span class="k"&gt;ilike&lt;/span&gt; &lt;span class="s1"&gt;'%user_metadata%'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Fix: keep roles in &lt;code&gt;app_metadata&lt;/code&gt; (only the service role can write it) or, better, in your own table such as &lt;code&gt;memberships(user_id, org_id, role)&lt;/code&gt; that users cannot insert into themselves.&lt;/p&gt;

&lt;h2&gt;
  
  
  4. &lt;code&gt;SECURITY DEFINER&lt;/code&gt; functions the anon key can call
&lt;/h2&gt;

&lt;p&gt;Functions in &lt;code&gt;public&lt;/code&gt; are exposed at &lt;code&gt;/rest/v1/rpc/&amp;lt;name&amp;gt;&lt;/code&gt;. A &lt;code&gt;SECURITY DEFINER&lt;/code&gt; function runs with its owner's privileges, which usually means RLS doesn't apply inside it. If &lt;code&gt;anon&lt;/code&gt; can execute it, anyone on the internet can.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;select&lt;/span&gt; &lt;span class="n"&gt;p&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;proname&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;function_name&lt;/span&gt;
&lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;pg_proc&lt;/span&gt; &lt;span class="n"&gt;p&lt;/span&gt;
&lt;span class="k"&gt;join&lt;/span&gt; &lt;span class="n"&gt;pg_namespace&lt;/span&gt; &lt;span class="n"&gt;n&lt;/span&gt; &lt;span class="k"&gt;on&lt;/span&gt; &lt;span class="n"&gt;n&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;oid&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;p&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;pronamespace&lt;/span&gt;
&lt;span class="k"&gt;where&lt;/span&gt; &lt;span class="n"&gt;n&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;nspname&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s1"&gt;'public'&lt;/span&gt;
  &lt;span class="k"&gt;and&lt;/span&gt; &lt;span class="n"&gt;p&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;prosecdef&lt;/span&gt;
  &lt;span class="k"&gt;and&lt;/span&gt; &lt;span class="nf"&gt;has_function_privilege&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'anon'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;p&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;oid&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'execute'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Postgres grants &lt;code&gt;EXECUTE&lt;/code&gt; to &lt;code&gt;PUBLIC&lt;/code&gt; by default, so this list is often longer than people expect. Fix the grant, and pin the &lt;code&gt;search_path&lt;/code&gt; while you're there so the function can't be tricked into resolving a different object:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;revoke&lt;/span&gt; &lt;span class="n"&gt;execute&lt;/span&gt; &lt;span class="k"&gt;on&lt;/span&gt; &lt;span class="k"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;public&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get_report&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;anon&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;public&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="k"&gt;alter&lt;/span&gt; &lt;span class="k"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;public&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get_report&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;set&lt;/span&gt; &lt;span class="n"&gt;search_path&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s1"&gt;''&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;If it must stay &lt;code&gt;SECURITY DEFINER&lt;/code&gt;, move it to a schema the API doesn't expose and check the caller inside the function.&lt;/p&gt;

&lt;h2&gt;
  
  
  5. Views that skip RLS
&lt;/h2&gt;

&lt;p&gt;A normal view runs with the privileges of whoever created it, so it can return rows the caller's policies would hide. On Postgres 15+ you can make it respect the caller instead:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;alter&lt;/span&gt; &lt;span class="k"&gt;view&lt;/span&gt; &lt;span class="n"&gt;public&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;order_summary&lt;/span&gt; &lt;span class="k"&gt;set&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;security_invoker&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Find views that haven't been switched:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;select&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;relname&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;view_name&lt;/span&gt;
&lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;pg_class&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt;
&lt;span class="k"&gt;join&lt;/span&gt; &lt;span class="n"&gt;pg_namespace&lt;/span&gt; &lt;span class="n"&gt;n&lt;/span&gt; &lt;span class="k"&gt;on&lt;/span&gt; &lt;span class="n"&gt;n&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;oid&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;relnamespace&lt;/span&gt;
&lt;span class="k"&gt;where&lt;/span&gt; &lt;span class="n"&gt;n&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;nspname&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s1"&gt;'public'&lt;/span&gt;
  &lt;span class="k"&gt;and&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;relkind&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s1"&gt;'v'&lt;/span&gt;
  &lt;span class="k"&gt;and&lt;/span&gt; &lt;span class="k"&gt;not&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;coalesce&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;reloptions&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'{}'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
           &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="kt"&gt;array&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;'security_invoker=true'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'security_invoker=on'&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The worst version of this is a view over &lt;code&gt;auth.users&lt;/code&gt; that &lt;code&gt;anon&lt;/code&gt; can select from: that is your user list, emails included.&lt;/p&gt;

&lt;h2&gt;
  
  
  A small bonus: &lt;code&gt;auth.uid()&lt;/code&gt; per row
&lt;/h2&gt;

&lt;p&gt;Not a leak, but common: &lt;code&gt;using (auth.uid() = user_id)&lt;/code&gt; calls &lt;code&gt;auth.uid()&lt;/code&gt; once per row. Writing &lt;code&gt;using ((select auth.uid()) = user_id)&lt;/code&gt; lets Postgres evaluate it once per query. On large tables the difference is noticeable.&lt;/p&gt;

&lt;h2&gt;
  
  
  Running all of this at once
&lt;/h2&gt;

&lt;p&gt;I put these checks, plus a few more (public Storage buckets, extensions in &lt;code&gt;public&lt;/code&gt;), into one read-only query and a small CLI around it: &lt;a href="https://github.com/Tthanh2k6/supabase-rls-check" rel="noopener noreferrer"&gt;supabase-rls-check&lt;/a&gt;.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;pip install supabase-rls-check
supabase-rls-check sql --print-sql &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; check.sql
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;You paste &lt;code&gt;check.sql&lt;/code&gt; into the Supabase SQL Editor, export the result as CSV, and the CLI turns it into an HTML/Markdown report plus a &lt;code&gt;fix.sql&lt;/code&gt; draft. The draft is wrapped in &lt;code&gt;BEGIN; ... ROLLBACK;&lt;/code&gt;, so nothing changes until you read it and edit that last line. It also runs as a GitHub Action that fails a pull request when a migration introduces one of these findings, and as an MCP server for coding agents.&lt;/p&gt;

&lt;p&gt;Two honest caveats. Supabase's built-in Security Advisor already covers several of these, and you should use it. And none of this proves isolation: a catalog query can't tell you that user A can't read user B's rows through some combination of policies. For that you need tests that sign in as two real users and try. I wrote up that pattern, with 19 such tests, in &lt;a href="https://github.com/Tthanh2k6/supabase-multitenant-rls" rel="noopener noreferrer"&gt;supabase-multitenant-rls&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;If you've seen an RLS mistake that isn't on this list, I'd like to hear it in the comments. It might become the next rule.&lt;/p&gt;

</description>
      <category>supabase</category>
      <category>postgres</category>
      <category>security</category>
      <category>webdev</category>
    </item>
    <item>
      <title>Detecting Chain Ownership on US Clinic Websites with Python</title>
      <dc:creator>Trung Thanh Tran</dc:creator>
      <pubDate>Sat, 07 Nov 2026 17:11:46 +0000</pubDate>
      <link>https://dev.to/tthanh2k6/detecting-chain-ownership-on-us-clinic-websites-with-python-513</link>
      <guid>https://dev.to/tthanh2k6/detecting-chain-ownership-on-us-clinic-websites-with-python-513</guid>
      <description>&lt;p&gt;Suppose you build a lead list for a team selling software or supplies to independent US dental, veterinary, audiology, or optometry clinics. Map listings return neighborhood names like Oakridge Family Dentistry or Suncoast Pet Hospital. The list looks full of single-office owners.&lt;/p&gt;

&lt;p&gt;Once reps start calling, many of those offices turn out to belong to corporate groups, including Dental Support Organizations (DSOs) like Heartland Dental and Aspen Dental or veterinary networks like Banfield and VCA. An acquired clinic can keep its local domain while a parent company controls purchasing. Reps end up pitching staff who cannot approve new vendors.&lt;/p&gt;

&lt;p&gt;To filter clinic lists before outreach, I built an open-source Apify Actor, &lt;a href="https://github.com/Tthanh2k6/apify-us-healthcare-clinic-contact-enricher" rel="noopener noreferrer"&gt;&lt;code&gt;apify-us-healthcare-clinic-contact-enricher&lt;/code&gt;&lt;/a&gt;. It fetches public clinic websites in async Python, pulls contact details and Schema.org metadata, and checks sites against 38 US chains. Static HTML checks are all the live actor runs today. Querying the NPPES NPI registry and scanning for booking widgets are separate ideas for a later version.&lt;/p&gt;

&lt;h2&gt;
  
  
  Scraping contacts and chain clues from static HTML
&lt;/h2&gt;

&lt;p&gt;I wrote the actor with &lt;code&gt;httpx&lt;/code&gt;, &lt;code&gt;BeautifulSoup4&lt;/code&gt;, and &lt;code&gt;dnspython&lt;/code&gt; so it runs without a headless browser. For each URL, the crawler fetches the homepage and up to two internal contact, about, locations, or team pages. Skipping browser rendering saves memory, though sites that render contacts with JavaScript return fewer fields.&lt;/p&gt;

&lt;p&gt;The parser collects emails from HTML and &lt;code&gt;mailto:&lt;/code&gt; links while dropping asset filenames, &lt;code&gt;sentry.io&lt;/code&gt;, &lt;code&gt;wixpress.com&lt;/code&gt;, and placeholder domains. Next, &lt;code&gt;dnspython&lt;/code&gt; checks whether each domain has an MX record (or an A record if no MX exists) for &lt;code&gt;mx_dns_valid_emails&lt;/code&gt;. That DNS check covers the domain without verifying individual mailboxes. Alongside emails, the script formats US phone numbers from &lt;code&gt;tel:&lt;/code&gt; links and text as &lt;code&gt;+1-XXX-XXX-XXXX&lt;/code&gt;, collects up to three links each for Facebook, Instagram, LinkedIn, YouTube, and X/Twitter, and tags a rough specialty category from keywords.&lt;/p&gt;

&lt;p&gt;For ownership, the actor computes a &lt;code&gt;chain_confidence_score&lt;/code&gt; from four heuristics. First, it matches the domain and HTML against a hand-written list of 38 US chains across dental DSOs, hearing-aid retailers, veterinary groups, and chiropractic, PT, urgent-care, and optical brands (such as Heartland Dental, Aspen Dental, Miracle-Ear, Banfield, VCA, and MyEyeDr), plus terms you pass in &lt;code&gt;customChainKeywords&lt;/code&gt;. Second, it reads Schema.org JSON-LD blocks (&lt;code&gt;LocalBusiness&lt;/code&gt;, &lt;code&gt;MedicalOrganization&lt;/code&gt;) and raises the score if &lt;code&gt;parentOrganization&lt;/code&gt; appears. Third, it checks page text for franchise wording such as &lt;code&gt;"each office is independently owned and operated"&lt;/code&gt;. Fourth, five or more internal store-locator links raise the score, and 15 or more push it to chain status.&lt;/p&gt;

&lt;p&gt;Scores of &lt;code&gt;0.80&lt;/code&gt; or higher return &lt;code&gt;Chain Excluded&lt;/code&gt;, &lt;code&gt;0.45&lt;/code&gt; to &lt;code&gt;0.80&lt;/code&gt; return &lt;code&gt;Needs Review&lt;/code&gt;, and lower scores return &lt;code&gt;Likely Independent&lt;/code&gt;. That last label means none of the four signals matched. Sites that fail to load or return HTTP 4xx or 5xx responses come back as &lt;code&gt;Needs Review&lt;/code&gt; and log the failure in &lt;code&gt;error&lt;/code&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Idea 1: checking organizational records in the public NPPES NPI API
&lt;/h2&gt;

&lt;p&gt;A fixed list of 38 chains has clear limits. Regional groups outside that list, or acquired clinics that keep a local domain and omit parent names from their HTML, come back as &lt;code&gt;Likely Independent&lt;/code&gt;. Checking federal provider records is an idea I want to test next; the actor does not include it today.&lt;/p&gt;

&lt;p&gt;CMS runs a free public NPPES NPI Registry API at &lt;code&gt;https://npiregistry.cms.hhs.gov/api/&lt;/code&gt; (version &lt;code&gt;2.1&lt;/code&gt;) that needs no API key. &lt;code&gt;NPI-1&lt;/code&gt; records cover individuals, while &lt;code&gt;NPI-2&lt;/code&gt; records cover organizations. An &lt;code&gt;NPI-2&lt;/code&gt; record includes the organization name, &lt;code&gt;LOCATION&lt;/code&gt; and &lt;code&gt;MAILING&lt;/code&gt; addresses, and an &lt;code&gt;authorized_official&lt;/code&gt; block with the official's name and title. This async sketch (its query parameters and field names match a live call I made to the API on 5 Nov 2026) queries &lt;code&gt;NPI-2&lt;/code&gt; records and extracts the authorized official:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;httpx&lt;/span&gt;

&lt;span class="no"&gt;NPPES_API_URL&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"https://npiregistry.cms.hhs.gov/api/"&lt;/span&gt;

&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;lookup_clinic_npi&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;httpx&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;AsyncClient&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;organization_name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;city&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;state&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;str&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="kt"&gt;dict&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;params&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="s2"&gt;"version"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"2.1"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="s2"&gt;"enumeration_type"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"NPI-2"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="s2"&gt;"organization_name"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"{organization_name}*"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="s2"&gt;"city"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;city&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="s2"&gt;"state"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;state&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="s2"&gt;"limit"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="n"&gt;resp&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="no"&gt;NPPES_API_URL&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;params&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;params&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;timeout&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mf"&gt;8.0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;resp&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;raise_for_status&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;results&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;resp&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"results"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="ow"&gt;or&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="ow"&gt;not&lt;/span&gt; &lt;span class="n"&gt;results&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="n"&gt;basic&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;results&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"basic"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="ow"&gt;or&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="s2"&gt;"npi_number"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;results&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"number"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="s2"&gt;"legal_business_name"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;basic&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"organization_name"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="s2"&gt;"authorized_official_first_name"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;basic&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"authorized_official_first_name"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="s2"&gt;"authorized_official_last_name"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;basic&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"authorized_official_last_name"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="s2"&gt;"authorized_official_title"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;basic&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"authorized_official_title_or_position"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Using this in a batch pipeline requires careful matching. Clinics display local trade names on their websites while registering &lt;code&gt;NPI-2&lt;/code&gt; records under legal entity names, so name-and-city queries can miss or misalign offices. Cross-checking phone numbers and street addresses from the website against the &lt;code&gt;LOCATION&lt;/code&gt; entry in &lt;code&gt;results[0]["addresses"]&lt;/code&gt; should cut false matches, while comparing &lt;code&gt;LOCATION&lt;/code&gt; and &lt;code&gt;MAILING&lt;/code&gt; states could flag out-of-state billing hubs.&lt;/p&gt;

&lt;h2&gt;
  
  
  Idea 2: detecting booking widgets in the fetched HTML
&lt;/h2&gt;

&lt;p&gt;Scanning that same HTML for online booking scripts is a second idea outside the current actor. Teams selling scheduling or messaging software want to know whether a clinic embeds NexHealth, LocalMed, Weave, or Zocdoc. A dictionary of compiled regex patterns can spot those vendor domains during the crawl:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;re&lt;/span&gt;

&lt;span class="no"&gt;BOOKING_STACK_SIGNATURES&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;dict&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="kt"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;re&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Pattern&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="kt"&gt;str&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="s2"&gt;"NexHealth"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;re&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nb"&gt;compile&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="s2"&gt;"nexhealth\.com|nh-widget"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;re&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="no"&gt;I&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s2"&gt;"LocalMed"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;re&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nb"&gt;compile&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="s2"&gt;"localmed\.com"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;re&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="no"&gt;I&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s2"&gt;"Weave"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;re&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nb"&gt;compile&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="s2"&gt;"getweave\.com|weavehelp\.com"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;re&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="no"&gt;I&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s2"&gt;"Dentrix Ascend"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;re&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nb"&gt;compile&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="s2"&gt;"dentrixascend\.com"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;re&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="no"&gt;I&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s2"&gt;"Open Dental"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;re&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nb"&gt;compile&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="s2"&gt;"opendental\.com|patientviewer\.com"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;re&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="no"&gt;I&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s2"&gt;"Zocdoc"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;re&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nb"&gt;compile&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="s2"&gt;"zocdoc\.com"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;re&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="no"&gt;I&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="n"&gt;detected_stack&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
    &lt;span class="n"&gt;vendor&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;vendor&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;pat&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="no"&gt;BOOKING_STACK_SIGNATURES&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;items&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;pat&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;search&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;combined_html&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="p"&gt;]&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Static HTML limits this check in the same way it limits contact extraction. Clinics that load booking modals at runtime through Google Tag Manager hide the vendor URL from &lt;code&gt;httpx&lt;/code&gt;, so an empty list means no widget appeared in the initial HTML.&lt;/p&gt;

&lt;h2&gt;
  
  
  What the Actor does today / what I'd add next
&lt;/h2&gt;

&lt;p&gt;You can run the current build on &lt;a href="https://apify.com/thanh81646/apify-us-healthcare-clinic-contact-enricher" rel="noopener noreferrer"&gt;Apify Store&lt;/a&gt; or read the code on &lt;a href="https://github.com/Tthanh2k6/apify-us-healthcare-clinic-contact-enricher" rel="noopener noreferrer"&gt;GitHub&lt;/a&gt;. On the Apify free tier the actor caps each run at 25 URLs. Today the actor crawls static HTML across each homepage and up to two internal pages, returning emails, domain MX/A checks, normalized US phones, social links, specialty tags, and Schema.org JSON-LD metadata. Its chain check combines the 38-chain US list, &lt;code&gt;customChainKeywords&lt;/code&gt;, JSON-LD &lt;code&gt;parentOrganization&lt;/code&gt; fields, franchise wording, and store-locator link counts. You can trigger runs from Python with &lt;code&gt;apify-client&lt;/code&gt; or from n8n and Make.com.&lt;/p&gt;

&lt;p&gt;The live actor skips headless browsers, misses groups outside its 38-chain list, and includes neither NPPES NPI lookups nor booking-widget checks. Next, I want to add an optional NPPES &lt;code&gt;NPI-2&lt;/code&gt; step that joins clinics to CMS records by phone number and street address to pull &lt;code&gt;authorized_official&lt;/code&gt; fields, and then scans the fetched HTML for booking widgets.&lt;/p&gt;

&lt;p&gt;Joining scraped local websites to public registries tends to surface odd edge cases across states. Have you matched healthcare or local-business leads against NPPES records, and did phone numbers or street addresses give you cleaner joins than clinic names?&lt;/p&gt;

</description>
      <category>python</category>
      <category>dataengineering</category>
      <category>webdev</category>
      <category>automation</category>
    </item>
    <item>
      <title>Offline Vietnamese fuel receipt OCR in Python and the apostrophe bug that zeroed three fields</title>
      <dc:creator>Trung Thanh Tran</dc:creator>
      <pubDate>Wed, 04 Nov 2026 10:53:42 +0000</pubDate>
      <link>https://dev.to/tthanh2k6/offline-vietnamese-fuel-receipt-ocr-in-python-and-the-apostrophe-bug-that-zeroed-three-fields-8n5</link>
      <guid>https://dev.to/tthanh2k6/offline-vietnamese-fuel-receipt-ocr-in-python-and-the-apostrophe-bug-that-zeroed-three-fields-8n5</guid>
      <description>&lt;p&gt;Truck drivers in Vietnam photograph paper fuel receipts and send the images over Zalo. At the fleet office, an accountant opens each photo and types the figures by hand into a fuel reconciliation spreadsheet. I built a local Python script that reads receipt images into Excel to skip that data entry step. No transport company uses the script, and I have zero real receipts on hand.&lt;/p&gt;

&lt;h2&gt;
  
  
  Offline stack and spreadsheet checks
&lt;/h2&gt;

&lt;p&gt;I run the pipeline on Windows with Python 3.11, &lt;code&gt;paddlepaddle&lt;/code&gt; 2.6.2, and &lt;code&gt;PaddleOCR&lt;/code&gt; 2.9.1 on CPU. For models, I paired &lt;code&gt;en_PP-OCRv3&lt;/code&gt; detection with &lt;code&gt;latin_PP-OCRv3&lt;/code&gt; recognition and &lt;code&gt;ch_ppocr_mobile_v2.0&lt;/code&gt; angle classification. During inference, I block network sockets in Python so receipt images stay on the machine and out of the cloud.&lt;/p&gt;

&lt;p&gt;From each slip, I extract date, time, station name, and license plate, normalizing plates like &lt;code&gt;51C-123.45&lt;/code&gt; into &lt;code&gt;51C12345&lt;/code&gt;. Alongside those fields, I parse the fuel item (&lt;code&gt;Dầu DO 0,05S&lt;/code&gt;, &lt;code&gt;Xăng RON95&lt;/code&gt;, or &lt;code&gt;E5 RON92&lt;/code&gt;), liters with a comma decimal such as &lt;code&gt;120,50&lt;/code&gt;, unit price, and total amount with dot thousands separators such as &lt;code&gt;2.410.000&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;The output Excel workbook has two sheets. Sheet one holds the parsed rows, and sheet two stores the raw OCR text for cross-checking. I shade a row yellow and mark it &lt;code&gt;CẦN XEM&lt;/code&gt; ("needs review") if a field is missing, if OCR confidence is low, or if &lt;code&gt;liters * unit price&lt;/code&gt; differs from the total amount by more than 1%.&lt;/p&gt;

&lt;h2&gt;
  
  
  Six synthetic receipts and a 0/6 first run
&lt;/h2&gt;

&lt;p&gt;Because I lack real fuel slips, I used PIL to generate six synthetic receipt images. All six use clear printed text, with slight rotation and light noise on two images and no license plate on one image. On my first run, fuel item scored 0/6, unit price scored 0/6, and station name scored 0/6.&lt;/p&gt;

&lt;p&gt;Before touching my regexes, I opened sheet two to inspect the raw OCR text. The Latin recognition model read Vietnamese horn vowels (&lt;code&gt;ơ&lt;/code&gt; and &lt;code&gt;ư&lt;/code&gt;) as a base letter plus an apostrophe: &lt;code&gt;"Đơn giá"&lt;/code&gt; became &lt;code&gt;"Do'n gia"&lt;/code&gt;, &lt;code&gt;"CỬA HÀNG"&lt;/code&gt; became &lt;code&gt;"CU'A HANG"&lt;/code&gt;, and &lt;code&gt;"Đường"&lt;/code&gt; became &lt;code&gt;"Du'ng"&lt;/code&gt;. Other diacritics shifted to Latin umlauts or acute marks. In that raw dump, &lt;code&gt;"Mặt hàng: Dầu DO"&lt;/code&gt; appeared as &lt;code&gt;"Mät hang: Däu DO"&lt;/code&gt; (turning both &lt;code&gt;ặ&lt;/code&gt; and &lt;code&gt;ầ&lt;/code&gt; into &lt;code&gt;ä&lt;/code&gt;) and &lt;code&gt;"Số lít"&lt;/code&gt; appeared as &lt;code&gt;"Só lit"&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Umlauts and acute marks caused zero trouble during matching because Unicode &lt;code&gt;NFD&lt;/code&gt; decomposition strips combining characters, turning &lt;code&gt;"Mät hang"&lt;/code&gt; into &lt;code&gt;"mat hang"&lt;/code&gt; and &lt;code&gt;"Só lit"&lt;/code&gt; into &lt;code&gt;"so lit"&lt;/code&gt;. An ASCII apostrophe is punctuation, so &lt;code&gt;"Do'n gia"&lt;/code&gt; survived &lt;code&gt;NFD&lt;/code&gt; stripping as &lt;code&gt;"do'n gia"&lt;/code&gt; and missed my &lt;code&gt;"don gia"&lt;/code&gt; label regex. Worse, the apostrophe put a word boundary right after &lt;code&gt;"Do"&lt;/code&gt; in &lt;code&gt;"Do'n"&lt;/code&gt;. My case-insensitive &lt;code&gt;\bDO\b&lt;/code&gt; pattern caught &lt;code&gt;"Do'n"&lt;/code&gt; as a second &lt;code&gt;"Dầu DO"&lt;/code&gt; item and reported multiple fuel items.&lt;/p&gt;

&lt;h2&gt;
  
  
  Dropping letter-sandwiched apostrophes in fold()
&lt;/h2&gt;

&lt;p&gt;I run a &lt;code&gt;fold()&lt;/code&gt; helper to strip diacritics before regex matching and keep the original OCR string for spreadsheet display. One regex pass at the top drops apostrophes between two letters before &lt;code&gt;NFD&lt;/code&gt; decomposition:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;fold&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;str&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="kt"&gt;str&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="c1"&gt;# The Latin model reads ơ/ư as o'/u' (Do'n, CU'A): drop apostrophes between letters first.&lt;/span&gt;
    &lt;span class="n"&gt;text&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;re&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;sub&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="s2"&gt;"(?&amp;lt;=[^\W\d_])['’`´](?=[^\W\d_])"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;""&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;text&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;unicodedata&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;normalize&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"NFD"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;replace&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"Đ"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"D"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;replace&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"đ"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"d"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="s2"&gt;""&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;join&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;c&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;text&lt;/span&gt; &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;unicodedata&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;category&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;!=&lt;/span&gt; &lt;span class="s2"&gt;"Mn"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;lower&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Because &lt;code&gt;[^\W\d_]&lt;/code&gt; matches Unicode letters, &lt;code&gt;"Do'n gia"&lt;/code&gt; folds to &lt;code&gt;"don gia"&lt;/code&gt; and &lt;code&gt;"CU'A HANG"&lt;/code&gt; folds to &lt;code&gt;"cua hang"&lt;/code&gt; without touching digits or outer quotes. I added one regression test using that raw OCR string, and all 70 tests in the suite pass.&lt;/p&gt;

&lt;h2&gt;
  
  
  Before and after on the six PIL receipts
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Field&lt;/th&gt;
&lt;th&gt;First run&lt;/th&gt;
&lt;th&gt;After fix&lt;/th&gt;
&lt;th&gt;Notes&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Fuel item&lt;/td&gt;
&lt;td&gt;0/6&lt;/td&gt;
&lt;td&gt;6/6&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;Do'n&lt;/code&gt; stops triggering a second &lt;code&gt;\bDO\b&lt;/code&gt; match&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Unit price&lt;/td&gt;
&lt;td&gt;0/6&lt;/td&gt;
&lt;td&gt;6/6&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;Do'n gia&lt;/code&gt; folds to &lt;code&gt;don gia&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Station name&lt;/td&gt;
&lt;td&gt;0/6&lt;/td&gt;
&lt;td&gt;0/6&lt;/td&gt;
&lt;td&gt;Still outputs &lt;code&gt;CU'A HANG XANG DAU SO 01&lt;/code&gt; without diacritics&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fqostr8v0e0hts3eo950b.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fqostr8v0e0hts3eo950b.png" alt=" " width="799" height="257"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;After the fix, date, time, liters, and total amount each scored 6/6 on the synthetic batch. License plate extraction hit 6/6: the parser read all five printed plates and marked the plateless sixth receipt with &lt;code&gt;CẦN XEM&lt;/code&gt;. Station name stays at 0/6 against accented ground truth because &lt;code&gt;latin_PP-OCRv3&lt;/code&gt; drops Vietnamese diacritics and the sheet displays raw OCR text, though &lt;code&gt;"CU'A HANG XANG DAU SO 01"&lt;/code&gt; lets a person recognize the station.&lt;/p&gt;

&lt;p&gt;Image 05 triggered &lt;code&gt;CẦN XEM&lt;/code&gt; on the arithmetic check. That synthetic slip prints &lt;code&gt;95,8&lt;/code&gt; liters at &lt;code&gt;20.500&lt;/code&gt; VND per liter (&lt;code&gt;1.963.900&lt;/code&gt; VND) next to a total of &lt;code&gt;2.000.000&lt;/code&gt; VND from a round-amount fill-up. Because the gap exceeds 1%, I shade that row yellow for the accountant to verify.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I'd check next
&lt;/h2&gt;

&lt;p&gt;I saved myself a blind regex hunt by dumping raw OCR text onto sheet two, since the parsed cells gave no clue why &lt;code&gt;\bDO\b&lt;/code&gt; matched twice. Keeping &lt;code&gt;fold()&lt;/code&gt; for matching while writing raw OCR strings to Excel prevented my apostrophe fix from altering the visible text.&lt;/p&gt;

&lt;p&gt;Scoring 6/6 on PIL-generated images does not measure real accuracy. Clean bitmap fonts with mild tilt and light noise hide the failure modes you hit on phone photos. My next step is testing this script on real smartphone photos of fuel receipts from Zalo, where slips arrive blurry, angled, or crumpled. Have you run OCR on Vietnamese receipts or documents, and which recognition model kept Vietnamese diacritics intact for you?&lt;/p&gt;

</description>
      <category>python</category>
      <category>ocr</category>
      <category>automation</category>
      <category>showdev</category>
    </item>
    <item>
      <title>Getting Past Google Apps Script's 6-Minute Limit and n8n Out-of-Memory Crashes</title>
      <dc:creator>Trung Thanh Tran</dc:creator>
      <pubDate>Tue, 03 Nov 2026 13:05:20 +0000</pubDate>
      <link>https://dev.to/tthanh2k6/getting-past-google-apps-scripts-6-minute-limit-and-n8n-out-of-memory-crashes-4964</link>
      <guid>https://dev.to/tthanh2k6/getting-past-google-apps-scripts-6-minute-limit-and-n8n-out-of-memory-crashes-4964</guid>
      <description>&lt;p&gt;I build Google Sheets and Apps Script tools alongside small Python services, and I keep running into the same two walls as row counts grow.&lt;/p&gt;

&lt;p&gt;The first is the &lt;code&gt;03:06 AM&lt;/code&gt; Google Sheets failure email. An internal Google Apps Script that synced 400 orders or CRM leads in January now has 8,500 rows to process. Google kills the morning run halfway through with &lt;code&gt;Exceeded maximum execution time&lt;/code&gt;. The script stops with half the spreadsheet updated, after firing duplicate API webhooks and breaking formulas.&lt;/p&gt;

&lt;p&gt;The second wall is a quiet &lt;code&gt;n8n&lt;/code&gt; container restart on a 2 GB or 4 GB VPS. A self-hosted &lt;code&gt;n8n&lt;/code&gt; workflow runs &lt;code&gt;SplitInBatches&lt;/code&gt; over a 15,000-row CSV or Shopify catalog and dies with &lt;code&gt;Exit Code 137&lt;/code&gt; or &lt;code&gt;FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Teams respond by renting more VPS RAM for &lt;code&gt;n8n&lt;/code&gt; or splitting a Google Sheet across five tabs. Both workarounds buy a little breathing room while leaving the real bottlenecks intact: per-cell network round trips and growing execution state.&lt;/p&gt;

&lt;h2&gt;
  
  
  Anatomy of the two failure modes
&lt;/h2&gt;

&lt;p&gt;Google's V8 Apps Script sandbox and &lt;code&gt;n8n&lt;/code&gt;'s Node.js worker fail in different places, though both punish row-by-row loops:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Bottleneck layer&lt;/th&gt;
&lt;th&gt;Google Apps Script (&lt;code&gt;V8 Runtime&lt;/code&gt;)&lt;/th&gt;
&lt;th&gt;Self-hosted or cloud &lt;code&gt;n8n&lt;/code&gt; (&lt;code&gt;Node.js Runtime&lt;/code&gt;)&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Hard ceiling&lt;/td&gt;
&lt;td&gt;360 seconds (6 minutes) hard kill per execution; &lt;code&gt;90 min/day&lt;/code&gt; total trigger quota on consumer Gmail (&lt;code&gt;6 hr/day&lt;/code&gt; on Workspace).&lt;/td&gt;
&lt;td&gt;Node.js V8 heap limit (default ~1.5 to 2 GB unless you raise &lt;code&gt;NODE_OPTIONS=--max-old-space-size&lt;/code&gt;) or Docker cgroup &lt;code&gt;OOMKilled (137)&lt;/code&gt;.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Main antipattern&lt;/td&gt;
&lt;td&gt;Calling &lt;code&gt;sheet.getRange(r, c).setValue(...)&lt;/code&gt; or &lt;code&gt;UrlFetchApp.fetch()&lt;/code&gt; one by one inside a &lt;code&gt;for&lt;/code&gt; loop (~80 to 150 ms of IPC overhead per cell).&lt;/td&gt;
&lt;td&gt;Looping 10,000 items inside a single workflow with &lt;code&gt;SplitInBatches&lt;/code&gt;, since &lt;code&gt;n8n&lt;/code&gt; retains item history for each node across loop iterations in RAM.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Common trap&lt;/td&gt;
&lt;td&gt;Two simultaneous &lt;code&gt;doPost(e)&lt;/code&gt; webhooks read &lt;code&gt;getLastRow()&lt;/code&gt; at the same millisecond and overwrite each other's row.&lt;/td&gt;
&lt;td&gt;Keeping binary files like PDFs or 25 MB CSV exports in &lt;code&gt;N8N_DEFAULT_BINARY_DATA_MODE=default&lt;/code&gt; (RAM) instead of &lt;code&gt;filesystem&lt;/code&gt;.&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h2&gt;
  
  
  Part 1: Engineering around Google Apps Script's 6-minute wall
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Fix 1A: Eliminate per-cell RPC calls (&lt;code&gt;getValues()&lt;/code&gt; / &lt;code&gt;setValues()&lt;/code&gt; + &lt;code&gt;UrlFetchApp.fetchAll()&lt;/code&gt;)
&lt;/h3&gt;

&lt;p&gt;In Apps Script, each &lt;code&gt;getValue()&lt;/code&gt; or &lt;code&gt;setValue()&lt;/code&gt; call crosses an internal network boundary to Google Sheets. At 80 to 150 ms a call, 3,000 &lt;code&gt;setValue()&lt;/code&gt; writes eat most of your 6-minute budget on round trips alone.&lt;/p&gt;

&lt;p&gt;You can batch outbound HTTP traffic the same way. Pass an array of request objects to &lt;code&gt;UrlFetchApp.fetchAll(requests)&lt;/code&gt; and Google runs those requests in parallel on its own infrastructure:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="cm"&gt;/**
 * BAD: O(N) Sheet RPCs + sequential HTTP = ~340 seconds for 400 rows (times out!)
 * GOOD: Single 2D matrix read + parallel fetchAll chunks + single matrix write = ~9.4 seconds!
 */&lt;/span&gt;
&lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;enrichSheetBatchFast&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;sheet&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;startRow&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;batchSize&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;apiEndpoint&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;apiKey&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;values&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;sheet&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getRange&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;startRow&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;batchSize&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getValues&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="c1"&gt;// [domain, status, email, tech]&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;pendingIndices&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;httpRequests&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

  &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="n"&gt;i&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="n"&gt;i&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="n"&gt;values&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;length&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="n"&gt;i&lt;/span&gt;&lt;span class="o"&gt;++&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;domain&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;String&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;values&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;i&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="s2"&gt;""&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;trim&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;status&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;String&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;values&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;i&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="s2"&gt;""&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;trim&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="n"&gt;domain&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="n"&gt;status&lt;/span&gt; &lt;span class="o"&gt;===&lt;/span&gt; &lt;span class="s2"&gt;"DONE"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;continue&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

    &lt;span class="n"&gt;pendingIndices&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;push&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;i&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="n"&gt;httpRequests&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;push&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
      &lt;span class="n"&gt;url&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sb"&gt;`${apiEndpoint}?domain=${encodeURIComponent(domain)}`&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
      &lt;span class="n"&gt;method&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"get"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
      &lt;span class="n"&gt;headers&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="s2"&gt;"Authorization"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sb"&gt;`Bearer ${apiKey}`&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"Accept"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"application/json"&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
      &lt;span class="n"&gt;muteHttpExceptions&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;

  &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;httpRequests&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;length&lt;/span&gt; &lt;span class="o"&gt;===&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

  &lt;span class="c1"&gt;// Execute HTTP calls concurrently in chunks of 25&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="no"&gt;CHUNK&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;25&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="n"&gt;offset&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="n"&gt;offset&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="n"&gt;httpRequests&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;length&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="n"&gt;offset&lt;/span&gt; &lt;span class="o"&gt;+=&lt;/span&gt; &lt;span class="no"&gt;CHUNK&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;reqSlice&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;httpRequests&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;slice&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;offset&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;offset&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="no"&gt;CHUNK&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;idxSlice&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;pendingIndices&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;slice&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;offset&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;offset&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="no"&gt;CHUNK&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;responses&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;UrlFetchApp&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;fetchAll&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;reqSlice&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="n"&gt;j&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="n"&gt;j&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="n"&gt;responses&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;length&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="n"&gt;j&lt;/span&gt;&lt;span class="o"&gt;++&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
      &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;rowIdx&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;idxSlice&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;j&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
      &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;code&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;responses&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;j&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getResponseCode&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
      &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;code&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;=&lt;/span&gt; &lt;span class="mi"&gt;200&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="n"&gt;code&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="mi"&gt;300&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;data&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;JSON&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;parse&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;responses&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;j&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getContentText&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
        &lt;span class="n"&gt;values&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;rowIdx&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"DONE"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
        &lt;span class="n"&gt;values&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;rowIdx&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;data&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;primary_email&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="s2"&gt;""&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
        &lt;span class="n"&gt;values&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;rowIdx&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;data&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;tech_stack&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;join&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;", "&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
      &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;else&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="n"&gt;values&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;rowIdx&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="sb"&gt;`ERR_${code}`&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
      &lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;

  &lt;span class="c1"&gt;// Commit the entire updated block in ONE single Spreadsheet write call&lt;/span&gt;
  &lt;span class="n"&gt;sheet&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getRange&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;startRow&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;values&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;length&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;setValues&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;values&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="nc"&gt;SpreadsheetApp&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;flush&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;httpRequests&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;length&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Fix 1B: Stateful 270-second continuation checkpointing + &lt;code&gt;LockService&lt;/code&gt;
&lt;/h3&gt;

&lt;p&gt;Batched reads and writes hit the same 6-minute limit once a sheet holds 45,000 rows or calls a slow external ERP API that needs 25 minutes end to end. Checkpointing does not lift the daily quota either: a consumer account gets 20,000 &lt;code&gt;UrlFetchApp&lt;/code&gt; calls per day (100,000 on Workspace), so a job that calls an API for every row of a 45,000-row sheet has to spread over more than one day on a consumer account.&lt;/p&gt;

&lt;p&gt;Do not wait until second 359 for Google to kill the script mid-write. Set a soft wall-clock budget of 270 seconds (4.5 minutes) and save &lt;code&gt;cursor_row&lt;/code&gt; in &lt;code&gt;PropertiesService.getScriptProperties()&lt;/code&gt; after each batch. Before exiting, clear old continuation triggers to prevent duplicate trigger storms, then schedule a single trigger to resume 60 seconds later:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="no"&gt;CURSOR_KEY&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"SYNC_CURSOR_ROW"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="no"&gt;TRIGGER_HANDLER&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"runResumablePipeline"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="no"&gt;SOFT_LIMIT_MS&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;270&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mi"&gt;1000&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="c1"&gt;// 4.5 minutes (90s safety buffer before Google's 6m kill)&lt;/span&gt;
&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="no"&gt;BATCH_ROWS&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;200&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;runResumablePipeline&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="c1"&gt;// 1. Prevent overlapping executions (e.g. manual click + scheduled trigger colliding)&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;lock&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;LockService&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getScriptLock&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="n"&gt;lock&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;tryLock&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;5000&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nb"&gt;console&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;log&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"Another pipeline instance holds the script lock. Exiting cleanly."&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;

  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;startedAt&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;Date&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;now&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;props&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;PropertiesService&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getScriptProperties&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

  &lt;span class="k"&gt;try&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nf"&gt;clearContinuationTriggers_&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="no"&gt;TRIGGER_HANDLER&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;ss&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;SpreadsheetApp&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getActiveSpreadsheet&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;sheet&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;ss&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getSheetByName&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"Leads"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;lastRow&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;sheet&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getLastRow&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="n"&gt;cursor&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;parseInt&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;props&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getProperty&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="no"&gt;CURSOR_KEY&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="s2"&gt;"2"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;10&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

    &lt;span class="k"&gt;while&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;cursor&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;=&lt;/span&gt; &lt;span class="n"&gt;lastRow&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
      &lt;span class="c1"&gt;// Check soft wall-clock budget BEFORE starting the next batch&lt;/span&gt;
      &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;Date&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;now&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="n"&gt;startedAt&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;=&lt;/span&gt; &lt;span class="no"&gt;SOFT_LIMIT_MS&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="n"&gt;props&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;setProperty&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="no"&gt;CURSOR_KEY&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nb"&gt;String&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;cursor&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
        &lt;span class="nf"&gt;scheduleNextContinuation_&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="no"&gt;TRIGGER_HANDLER&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;60&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mi"&gt;1000&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
        &lt;span class="nb"&gt;console&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;log&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sb"&gt;`Soft time limit reached. Saved checkpoint at row ${cursor}. Resuming in 60s.`&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
      &lt;span class="p"&gt;}&lt;/span&gt;

      &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;remaining&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;lastRow&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="n"&gt;cursor&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
      &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;currentBatch&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;Math&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;min&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="no"&gt;BATCH_ROWS&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;remaining&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
      &lt;span class="nf"&gt;enrichSheetBatchFast&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="n"&gt;sheet&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;cursor&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;currentBatch&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;props&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getProperty&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"ENRICH_API_URL"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;props&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getProperty&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"ENRICH_API_KEY"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
      &lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

      &lt;span class="n"&gt;cursor&lt;/span&gt; &lt;span class="o"&gt;+=&lt;/span&gt; &lt;span class="n"&gt;currentBatch&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
      &lt;span class="n"&gt;props&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;setProperty&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="no"&gt;CURSOR_KEY&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nb"&gt;String&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;cursor&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="c1"&gt;// Completed all rows cleanly: reset cursor for tomorrow's scheduled run!&lt;/span&gt;
    &lt;span class="n"&gt;props&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;deleteProperty&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="no"&gt;CURSOR_KEY&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="nb"&gt;console&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;log&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sb"&gt;`Pipeline completed all ${lastRow - 1} rows with zero timeouts.`&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;finally&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="n"&gt;lock&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;releaseLock&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;clearContinuationTriggers_&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;handlerName&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;triggers&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;ScriptApp&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getProjectTriggers&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;t&lt;/span&gt; &lt;span class="k"&gt;of&lt;/span&gt; &lt;span class="n"&gt;triggers&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;t&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getHandlerFunction&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;===&lt;/span&gt; &lt;span class="n"&gt;handlerName&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
      &lt;span class="nc"&gt;ScriptApp&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;deleteTrigger&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;t&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;scheduleNextContinuation_&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;handlerName&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;delayMs&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nc"&gt;ScriptApp&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;newTrigger&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;handlerName&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;timeBased&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;after&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;delayMs&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;create&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;blockquote&gt;
&lt;p&gt;You also want a lock on &lt;code&gt;doPost(e)&lt;/code&gt; webhook receivers. Wrap the write in &lt;code&gt;const lock = LockService.getScriptLock(); lock.waitLock(15000); try { ... } finally { lock.releaseLock(); }&lt;/code&gt;. The overwrite happens when the handler computes its target row with &lt;code&gt;getLastRow() + 1&lt;/code&gt; and then calls &lt;code&gt;setValues()&lt;/code&gt;: 5 Stripe or Shopify webhooks arriving in the same second can all read the same last row. The lock also covers any read-then-write logic, such as checking for a duplicate order ID before appending.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h2&gt;
  
  
  Part 2: Stopping &lt;code&gt;n8n&lt;/code&gt; out-of-memory (&lt;code&gt;OOM&lt;/code&gt;) crashes
&lt;/h2&gt;

&lt;p&gt;A 2 GB self-hosted &lt;code&gt;n8n&lt;/code&gt; instance can crash on a 12 MB JSON payload or a 5,000-row &lt;code&gt;SplitInBatches&lt;/code&gt; loop because its execution engine snapshots output items at each node boundary. Those snapshots let the UI canvas render intermediate outputs and resolve expressions like &lt;code&gt;$('Previous Node').item.json&lt;/code&gt;. Pass a 12 MB array through 8 transformation nodes in one execution and V8 ends up holding several serialized copies, eating hundreds of MB of heap.&lt;/p&gt;

&lt;p&gt;Keeping &lt;code&gt;SplitInBatches&lt;/code&gt; inside one monolithic workflow compounds the problem. The parent execution holds onto previous batch iterations in its memory tree until the entire run finishes.&lt;/p&gt;

&lt;h3&gt;
  
  
  The 3-step &lt;code&gt;n8n&lt;/code&gt; memory fix
&lt;/h3&gt;

&lt;h4&gt;
  
  
  1. Move binary payloads off RAM (&lt;code&gt;filesystem&lt;/code&gt; mode)
&lt;/h4&gt;

&lt;p&gt;In your &lt;code&gt;docker-compose.yml&lt;/code&gt; or &lt;code&gt;.env&lt;/code&gt; for self-hosted &lt;code&gt;n8n&lt;/code&gt;, switch binary handling off &lt;code&gt;default&lt;/code&gt; (RAM) if your workflows touch CSV exports, PDFs, audio files, or scraped HTML snapshots:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="na"&gt;environment&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
  &lt;span class="p"&gt;-&lt;/span&gt; &lt;span class="s"&gt;N8N_DEFAULT_BINARY_DATA_MODE=filesystem&lt;/span&gt;
  &lt;span class="p"&gt;-&lt;/span&gt; &lt;span class="s"&gt;EXECUTIONS_DATA_SAVE_ON_SUCCESS=none&lt;/span&gt;
  &lt;span class="p"&gt;-&lt;/span&gt; &lt;span class="s"&gt;EXECUTIONS_DATA_SAVE_ON_ERROR=all&lt;/span&gt;
  &lt;span class="p"&gt;-&lt;/span&gt; &lt;span class="s"&gt;EXECUTIONS_DATA_PRUNE=true&lt;/span&gt;
  &lt;span class="p"&gt;-&lt;/span&gt; &lt;span class="s"&gt;EXECUTIONS_DATA_MAX_AGE=72&lt;/span&gt;
  &lt;span class="p"&gt;-&lt;/span&gt; &lt;span class="s"&gt;NODE_OPTIONS=--max-old-space-size=2560&lt;/span&gt;  &lt;span class="c1"&gt;# 4 GB VPS; on a 2 GB VPS keep it around 1536
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Setting &lt;code&gt;EXECUTIONS_DATA_SAVE_ON_SUCCESS=none&lt;/code&gt; on high-volume worker workflows stops &lt;code&gt;n8n&lt;/code&gt; from serializing 100 MB JSON execution blobs into SQLite or Postgres at completion.&lt;/p&gt;

&lt;h4&gt;
  
  
  2. Isolate loop batches in a child sub-workflow
&lt;/h4&gt;

&lt;p&gt;Move those 12 enrichment and CRM nodes out of your parent &lt;code&gt;SplitInBatches&lt;/code&gt; loop and split the work across two workflows:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;The parent workflow queries lightweight record IDs or 200-row slices (&lt;code&gt;SELECT id, domain FROM leads WHERE synced = false LIMIT 200&lt;/code&gt;) and calls the &lt;code&gt;Execute Sub-workflow&lt;/code&gt; node (&lt;code&gt;Mode: Run once for all items&lt;/code&gt; per 200-row batch, &lt;code&gt;Wait For Sub-Workflow Completion: ON&lt;/code&gt;).&lt;/li&gt;
&lt;li&gt;The child sub-workflow takes those 200 rows and runs the heavy enrichment, from OpenAI Structured Output parsing to CRM upserts. At the end, it returns a small summary object: &lt;code&gt;{"processed": 200, "failed": 0}&lt;/code&gt;.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Once the child sub-workflow finishes and hands back that 32-byte summary, V8 can garbage-collect the child execution's intermediate data before batch &lt;code&gt;#2&lt;/code&gt; starts. The process holds one batch in memory at a time instead of growing with the total row count.&lt;/p&gt;

&lt;p&gt;Update after a reader comment: let the child return the IDs that failed next to the counts, for example &lt;code&gt;{"processed": 200, "failed": 2, "failed_ids": [1043, 1187]}&lt;/code&gt;. The parent marks the batch as synced and flips only those IDs back to &lt;code&gt;synced = false&lt;/code&gt;. The next run then picks up exactly the failed rows instead of re-running the whole 200-row slice and writing the same records to the CRM twice. Thanks to &lt;a class="mentioned-user" href="https://dev.to/fun_fun_19313eced54af3c68"&gt;@fun_fun_19313eced54af3c68&lt;/a&gt; for the suggestion.&lt;/p&gt;

&lt;h4&gt;
  
  
  3. Offload heavy scraping and deduplication to a 35 MB async Python sidecar
&lt;/h4&gt;

&lt;p&gt;&lt;code&gt;n8n&lt;/code&gt; fits stateful orchestration and webhook routing, while 5,000-URL crawls and fuzzy CSV deduplication bog down its Node worker. Once a &lt;code&gt;Code&lt;/code&gt; node in &lt;code&gt;n8n&lt;/code&gt; passes 80 lines of nested loops or &lt;code&gt;cheerio&lt;/code&gt; HTML parsing, I move that compute step into a stateless FastAPI (Python 3.11) container on the same Docker network, or call an Apify Actor over a webhook:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="c1"&gt;# Lightweight FastAPI chunk worker (`~34 MB` RSS) called by n8n HTTP Request node&lt;/span&gt;
&lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;fastapi&lt;/span&gt; &lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="nc"&gt;FastAPI&lt;/span&gt;
&lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;pydantic&lt;/span&gt; &lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="nc"&gt;BaseModel&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nc"&gt;Field&lt;/span&gt;
&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;httpx&lt;/span&gt;
&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;asyncio&lt;/span&gt;

&lt;span class="n"&gt;app&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;FastAPI&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;title&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"n8n Offload Chunk Worker"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;


&lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;BatchRequest&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nc"&gt;BaseModel&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;domains&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;list&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="kt"&gt;str&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Field&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;max_length&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;250&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;concurrency&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;int&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Field&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;default&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;15&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;ge&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;le&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;40&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;


&lt;span class="nd"&gt;@app&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;post&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"/v1/probe-batch"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;probe_domains_batch&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;payload&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;BatchRequest&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="kt"&gt;dict&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;sem&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;asyncio&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Semaphore&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;payload&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;concurrency&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;results&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;list&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="kt"&gt;dict&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;

    &lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;_probe&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;httpx&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;AsyncClient&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;domain&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;str&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;url&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;domain&lt;/span&gt; &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;domain&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;startswith&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"http"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;else&lt;/span&gt; &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"https://{domain}"&lt;/span&gt;
        &lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;with&lt;/span&gt; &lt;span class="n"&gt;sem&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="k"&gt;try&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
                &lt;span class="n"&gt;r&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;url&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;timeout&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mf"&gt;6.0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
                &lt;span class="n"&gt;results&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;append&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"domain"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;domain&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"status"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;status_code&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"ok"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;status_code&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="mi"&gt;400&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
            &lt;span class="k"&gt;except&lt;/span&gt; &lt;span class="nc"&gt;Exception&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;exc&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
                &lt;span class="n"&gt;results&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;append&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"domain"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;domain&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"status"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"ok"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;False&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"err"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kd"&gt;type&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;exc&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;__name__&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;with&lt;/span&gt; &lt;span class="n"&gt;httpx&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;AsyncClient&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;follow_redirects&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="kc"&gt;True&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;asyncio&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;gather&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nf"&gt;_probe&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;d&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;d&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;payload&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;domains&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="c1"&gt;# Return compact structured summary back to n8n — zero raw HTML enters n8n's V8 heap!&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="s2"&gt;"total"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;results&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="s2"&gt;"alive_count"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;sum&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;x&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;results&lt;/span&gt; &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;x&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"ok"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="s2"&gt;"items"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;results&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  What changes, in short
&lt;/h2&gt;

&lt;p&gt;These are expected behaviors from how each runtime manages limits; I did not benchmark the four setups side by side on a client system.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Architecture&lt;/th&gt;
&lt;th&gt;Large sheet / batch&lt;/th&gt;
&lt;th&gt;When it fails&lt;/th&gt;
&lt;th&gt;Recovery&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Naive Apps Script (&lt;code&gt;setValue&lt;/code&gt; loop)&lt;/td&gt;
&lt;td&gt;Hits the 6-minute kill partway through&lt;/td&gt;
&lt;td&gt;Mid-write, half the sheet updated&lt;/td&gt;
&lt;td&gt;Manual cleanup&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Checkpointed Apps Script (batched writes + &lt;code&gt;270s&lt;/code&gt; cursor + &lt;code&gt;LockService&lt;/code&gt;)&lt;/td&gt;
&lt;td&gt;Finishes across chained runs&lt;/td&gt;
&lt;td&gt;Each run stops at the &lt;code&gt;270s&lt;/code&gt; budget&lt;/td&gt;
&lt;td&gt;Resumes from the saved row cursor&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Monolithic &lt;code&gt;n8n&lt;/code&gt; (&lt;code&gt;SplitInBatches&lt;/code&gt; in one workflow, in-memory binaries)&lt;/td&gt;
&lt;td&gt;Memory grows on each loop&lt;/td&gt;
&lt;td&gt;OS can OOM-kill the container (exit 137)&lt;/td&gt;
&lt;td&gt;Partial, unlogged state&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;code&gt;n8n&lt;/code&gt; parent/child sub-workflow + async Python sidecar&lt;/td&gt;
&lt;td&gt;Memory stays bounded by batch size&lt;/td&gt;
&lt;td&gt;One batch fails while the rest of the run continues&lt;/td&gt;
&lt;td&gt;Per-batch retry + alert&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h2&gt;
  
  
  Templates and help
&lt;/h2&gt;

&lt;p&gt;If your Google Sheets scripts hit the 6-minute execution limit, or your &lt;code&gt;n8n&lt;/code&gt; workflows drop webhooks and crash under load, you can reuse a few pieces instead of rebuilding from scratch.&lt;/p&gt;

&lt;p&gt;I published async Python hidden-API and automation starter templates on Gumroad as the free &lt;a href="https://trungthanh16.gumroad.com/l/python-hidden-api-starter-kit-lite" rel="noopener noreferrer"&gt;Python &amp;amp; Playwright Hidden-API Starter Kit Lite (&lt;code&gt;$0+&lt;/code&gt;)&lt;/a&gt;. The full 6-engine set with importable &lt;code&gt;n8n&lt;/code&gt; and Apps Script blueprints is in the &lt;a href="https://trungthanh16.gumroad.com/l/python-ai-automation-n8n-scraper-toolkit-pro" rel="noopener noreferrer"&gt;Pro Toolkit&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;To move B2B lead enrichment, Shopify or WooCommerce price monitoring, US healthcare clinic DSO screening, EU hospitality scraping, or &lt;code&gt;2-Opt TSP&lt;/code&gt; route optimization out of &lt;code&gt;n8n&lt;/code&gt; RAM, you can call my 256 MB pay-per-event Actors on Apify Store (&lt;code&gt;@thanh81646&lt;/code&gt;): &lt;a href="https://apify.com/thanh81646" rel="noopener noreferrer"&gt;https://apify.com/thanh81646&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;For a broken workflow or a custom pipeline, I refactor and build Python, Playwright, &lt;code&gt;n8n&lt;/code&gt;, FastAPI, and Google Apps Script automations on small fixed-price milestones. My portfolio and contact info are at &lt;a href="https://trungthanhtran.vercel.app" rel="noopener noreferrer"&gt;trungthanhtran.vercel.app&lt;/a&gt;.&lt;/p&gt;

</description>
      <category>automation</category>
      <category>javascript</category>
      <category>python</category>
      <category>webdev</category>
    </item>
    <item>
      <title>Building a B2B Lead Enricher and a 2-Opt Route Optimizer in Python (No Headless Browser)</title>
      <dc:creator>Trung Thanh Tran</dc:creator>
      <pubDate>Fri, 30 Oct 2026 03:40:38 +0000</pubDate>
      <link>https://dev.to/tthanh2k6/engineering-sub-second-b2b-lead-enrichment-2-opt-route-optimization-pipelines-in-python-1le8</link>
      <guid>https://dev.to/tthanh2k6/engineering-sub-second-b2b-lead-enrichment-2-opt-route-optimization-pipelines-in-python-1le8</guid>
      <description>&lt;p&gt;When sales operations and field-dispatch teams build outbound or territory pipelines, they almost always run into two surprisingly expensive bottlenecks:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Dirty B2B Lead Lists Full of Corporate Chains &amp;amp; Dead Domains:&lt;/strong&gt; You scrape &lt;code&gt;2,000&lt;/code&gt; healthcare, dental, audiology, or veterinary clinics from a local map directory—only to realize that a good share of them are franchises or private-equity DSOs/MSOs (&lt;em&gt;Aspen Dental, Heartland Dental, Miracle-Ear, Banfield&lt;/em&gt;), and that some of the scraped "emails" are &lt;code&gt;sentry.wixpress.com&lt;/code&gt; telemetry strings or addresses on dead domains.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Zig-Zag Field Territory Routes:&lt;/strong&gt; Once you have &lt;code&gt;15–25&lt;/code&gt; verified local prospects in a metro area, ordering visits chronologically or alphabetically can waste a big part of a rep's shift crisscrossing highways—while enterprise Route Matrix APIs charge per-element fees (&lt;code&gt;N × N&lt;/code&gt; matrix billing).&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Both problems can be handled in &lt;strong&gt;plain Python 3.11&lt;/strong&gt; with no headless browser and no paid third-party API keys (geocoding uses the free OpenStreetMap Nominatim service, which allows about 1 request per second).&lt;/p&gt;

&lt;p&gt;In this deep dive, we will walk through the engineering behind three small Apify Actors I published recently (the source is public on GitHub; they are new and have only a handful of runs so far):&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Stage 1:&lt;/strong&gt; Async HTTP + &lt;code&gt;DNS/MX&lt;/code&gt; Contact Enricher &amp;amp; Corporate DSO/Chain Classifier (&lt;a href="https://github.com/Tthanh2k6/apify-us-healthcare-clinic-contact-enricher" rel="noopener noreferrer"&gt;&lt;code&gt;apify-us-healthcare-clinic-contact-enricher&lt;/code&gt;&lt;/a&gt; &amp;amp; &lt;a href="https://github.com/Tthanh2k6/apify-b2b-lead-and-contact-enricher" rel="noopener noreferrer"&gt;&lt;code&gt;apify-b2b-lead-and-contact-enricher&lt;/code&gt;&lt;/a&gt;).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Stage 2:&lt;/strong&gt; Road-Adjusted Haversine + &lt;strong&gt;Nearest-Neighbor &amp;amp; 2-Opt TSP&lt;/strong&gt; Multi-Stop Route Optimizer (&lt;a href="https://github.com/Tthanh2k6/apify-multi-stop-route-and-distance-optimizer" rel="noopener noreferrer"&gt;&lt;code&gt;apify-multi-stop-route-and-distance-optimizer&lt;/code&gt;&lt;/a&gt;).&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  Stage 1: Zero-Browser B2B Contact Enrichment + DNS/MX Verification
&lt;/h2&gt;

&lt;p&gt;Most naive email scrapers run a single regex (&lt;code&gt;[a-zA-Z0-9._%+-]+@...&lt;/code&gt;) across raw homepage HTML and call it a day. On modern websites, that immediately pollutes your CRM with three classes of garbage:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Asset filename collisions:&lt;/strong&gt; Retina image naming conventions like &lt;code&gt;hero-banner@2x.png&lt;/code&gt; or &lt;code&gt;logo@3x.webp&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Frontend error-tracking DSNs:&lt;/strong&gt; Bundled Sentry keys such as &lt;code&gt;89f2a1@o4504.ingest.sentry.io&lt;/code&gt; or &lt;code&gt;user@sentry-next.wixpress.com&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Unroutable / parked domains:&lt;/strong&gt; Legacy domains in outdated footers with no valid &lt;code&gt;MX&lt;/code&gt; or &lt;code&gt;A&lt;/code&gt; records.&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  Step 1A: Async Domain &lt;code&gt;MX&lt;/code&gt; Verification with In-Memory Caching
&lt;/h3&gt;

&lt;p&gt;Instead of paying per-email SMTP verification credits upfront, you can drop dead domains cheaply (each domain is looked up once and cached) using asynchronous DNS &lt;code&gt;MX&lt;/code&gt; lookups via &lt;code&gt;dnspython&lt;/code&gt;'s &lt;code&gt;dns.asyncresolver&lt;/code&gt;, falling back cleanly to &lt;code&gt;A&lt;/code&gt; record resolution per RFC 5321 Section 5.1:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;asyncio&lt;/span&gt;
&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;re&lt;/span&gt;
&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;dns&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;asyncresolver&lt;/span&gt;

&lt;span class="no"&gt;EMAIL_REGEX&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;re&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nb"&gt;compile&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="s2"&gt;"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,12}"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="no"&gt;JUNK_EXTS&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;".png"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;".jpg"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;".jpeg"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;".gif"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;".svg"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;".webp"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;".css"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;".js"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;".woff"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;".woff2"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="no"&gt;JUNK_DOMAINS&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"sentry.io"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"sentry.wixpress.com"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"sentry-next.wixpress.com"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"example.com"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"wix.com"&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;


&lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;AsyncMXValidator&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;__init__&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;self&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="nb"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;_domain_cache&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;dict&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="kt"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kt"&gt;bool&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;has_valid_mail_route&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;self&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;domain&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;str&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="kt"&gt;bool&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;d&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;domain&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;strip&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;lower&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;rstrip&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"."&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="ow"&gt;not&lt;/span&gt; &lt;span class="n"&gt;d&lt;/span&gt; &lt;span class="ow"&gt;or&lt;/span&gt; &lt;span class="s2"&gt;"."&lt;/span&gt; &lt;span class="ow"&gt;not&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;d&lt;/span&gt; &lt;span class="ow"&gt;or&lt;/span&gt; &lt;span class="n"&gt;d&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="no"&gt;JUNK_DOMAINS&lt;/span&gt; &lt;span class="ow"&gt;or&lt;/span&gt; &lt;span class="s2"&gt;"sentry"&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;d&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="kc"&gt;False&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;d&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="nb"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;_domain_cache&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nb"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;_domain_cache&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;d&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;

        &lt;span class="n"&gt;resolver&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;dns&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;asyncresolver&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Resolver&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="n"&gt;resolver&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;lifetime&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mf"&gt;3.0&lt;/span&gt;
        &lt;span class="n"&gt;deliverable&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;False&lt;/span&gt;
        &lt;span class="k"&gt;try&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="n"&gt;mx_answers&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;resolver&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;resolve&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;d&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"MX"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
            &lt;span class="n"&gt;deliverable&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;mx_answers&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;
        &lt;span class="k"&gt;except&lt;/span&gt; &lt;span class="nc"&gt;Exception&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="c1"&gt;# RFC 5321 fallback: implicit MX via A record if no explicit MX exists&lt;/span&gt;
            &lt;span class="k"&gt;try&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
                &lt;span class="n"&gt;a_answers&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;resolver&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;resolve&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;d&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"A"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
                &lt;span class="n"&gt;deliverable&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;a_answers&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;
            &lt;span class="k"&gt;except&lt;/span&gt; &lt;span class="nc"&gt;Exception&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
                &lt;span class="n"&gt;deliverable&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;False&lt;/span&gt;

        &lt;span class="nb"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;_domain_cache&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;d&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;deliverable&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;deliverable&lt;/span&gt;


&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;extract_verified_emails&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;html&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;validator&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;AsyncMXValidator&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="kt"&gt;list&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="kt"&gt;str&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;candidates&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;set&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="kt"&gt;str&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kt"&gt;set&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;raw&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="no"&gt;EMAIL_REGEX&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;findall&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;html&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;em&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;raw&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;strip&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;lower&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;rstrip&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;".,;:"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="nb"&gt;any&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;em&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;endswith&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ext&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;ext&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="no"&gt;JUNK_EXTS&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="k"&gt;continue&lt;/span&gt;
        &lt;span class="n"&gt;local&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;_&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;domain&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;em&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;partition&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"@"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;local&lt;/span&gt; &lt;span class="ow"&gt;and&lt;/span&gt; &lt;span class="n"&gt;domain&lt;/span&gt; &lt;span class="ow"&gt;and&lt;/span&gt; &lt;span class="nb"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;local&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;=&lt;/span&gt; &lt;span class="mi"&gt;64&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="n"&gt;candidates&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;add&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;em&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="n"&gt;verified&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;list&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="kt"&gt;str&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;em&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="nb"&gt;sorted&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;candidates&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;validator&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;has_valid_mail_route&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;em&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;split&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"@"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="n"&gt;verified&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;append&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;em&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;verified&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Because B2B enrichment crawls &lt;code&gt;/&lt;/code&gt;, &lt;code&gt;/contact&lt;/code&gt;, and &lt;code&gt;/about&lt;/code&gt; on the same domain, caching by &lt;code&gt;domain&lt;/code&gt; means most candidate checks after the first page hit &lt;code&gt;_domain_cache&lt;/code&gt; instead of the network. (An &lt;code&gt;MX&lt;/code&gt; record only proves the domain can receive mail; it does not prove that a specific mailbox exists.)&lt;/p&gt;




&lt;h3&gt;
  
  
  Step 1B: Heuristic Corporate Chain / DSO / Franchise Screening
&lt;/h3&gt;

&lt;p&gt;If you sell medical devices, dental lab services, hearing aids, or practice-management SaaS to &lt;strong&gt;independent clinic owners&lt;/strong&gt;, calling a disguised corporate clinic is a complete waste of SDR dial time—local clinic managers cannot sign software or vendor contracts when procurement is locked at corporate HQ.&lt;/p&gt;

&lt;p&gt;Even when a private-equity Dental Support Organization (DSO) or veterinary consolidator keeps a local-sounding brand name (&lt;code&gt;"Oak Creek Family Dental"&lt;/code&gt;), they often leave footprints like these in the HTML and &lt;code&gt;Schema.org&lt;/code&gt; metadata:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Corporate Privacy Policy &amp;amp; Script Domains:&lt;/strong&gt; Footer links pointing to &lt;code&gt;heartland.com&lt;/code&gt;, &lt;code&gt;aspendental.com&lt;/code&gt;, &lt;code&gt;banfield.com&lt;/code&gt;, &lt;code&gt;vcahospitals.com&lt;/code&gt;, &lt;code&gt;miracle-ear.com&lt;/code&gt;, &lt;code&gt;beltone.com&lt;/code&gt;, or &lt;code&gt;thejoint.com&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;Schema.org&lt;/code&gt; &lt;code&gt;parentOrganization&lt;/code&gt; / &lt;code&gt;brand&lt;/code&gt; Declarations:&lt;/strong&gt; Nested inside &lt;code&gt;&amp;lt;script type="application/ld+json"&amp;gt;&lt;/code&gt; for &lt;code&gt;MedicalOrganization&lt;/code&gt;, &lt;code&gt;Dentist&lt;/code&gt;, or &lt;code&gt;VeterinaryCare&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Franchise / MSO Legal Disclaimers:&lt;/strong&gt; Regex-detectable regulatory boilerplate in the footer (&lt;code&gt;"each office is independently owned and operated"&lt;/code&gt;, &lt;code&gt;"backed by ... dental partners"&lt;/code&gt;).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Store-Locator URL Cardinality:&lt;/strong&gt; Counting unique internal &lt;code&gt;&amp;lt;a href="/locations/..."&amp;gt;&lt;/code&gt; subpaths (the Actor treats &lt;code&gt;&amp;gt;= 15&lt;/code&gt; distinct location slugs as a regional/national multi-site operator, and &lt;code&gt;5–14&lt;/code&gt; as a regional group worth reviewing).&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Each signal maps to a fixed score and the Actor keeps the highest one (a simple rule-based &lt;code&gt;0.0–1.0&lt;/code&gt; score, not a trained model), then labels each enriched clinic:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;Verified Independent&lt;/code&gt;&lt;/strong&gt; (&lt;code&gt;score &amp;lt; 0.45&lt;/code&gt;): Route straight into outbound CRM sequences.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;Needs Review&lt;/code&gt;&lt;/strong&gt; (&lt;code&gt;0.45 &amp;lt;= score &amp;lt; 0.80&lt;/code&gt;): Ambiguous or multi-office group (e.g., &lt;code&gt;5–14&lt;/code&gt; location pages, or a declared &lt;code&gt;parentOrganization&lt;/code&gt; with no other chain signal), worth a manual look.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;Chain Excluded&lt;/code&gt;&lt;/strong&gt; (&lt;code&gt;score &amp;gt;= 0.80&lt;/code&gt;): Automatically suppressed when &lt;code&gt;excludeChainsFromOutput=True&lt;/code&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The default input of &lt;a href="https://github.com/Tthanh2k6/apify-us-healthcare-clinic-contact-enricher" rel="noopener noreferrer"&gt;&lt;code&gt;apify-us-healthcare-clinic-contact-enricher&lt;/code&gt;&lt;/a&gt; is a small mixed batch (&lt;code&gt;pacificaudiology.com&lt;/code&gt;, &lt;code&gt;brooklyndentalstudio.com&lt;/code&gt;, &lt;code&gt;banfield.com&lt;/code&gt;). If the page loads, &lt;code&gt;banfield.com&lt;/code&gt; matches a built-in brand signature and is labelled &lt;code&gt;Chain Excluded&lt;/code&gt; with the signal &lt;code&gt;Banfield Pet Hospital (Mars Veterinary Health)&lt;/code&gt;. The other records get US phones normalized to &lt;code&gt;+1-XXX-XXX-XXXX&lt;/code&gt; and the emails whose domains pass the &lt;code&gt;MX&lt;/code&gt; check. Actual results depend on what each site serves on the day you run it.&lt;/p&gt;




&lt;h2&gt;
  
  
  Stage 2: Multi-Stop Route Ordering with Nearest-Neighbor + &lt;code&gt;2-Opt&lt;/code&gt;
&lt;/h2&gt;

&lt;p&gt;Now suppose your enrichment pipeline outputs &lt;code&gt;18&lt;/code&gt; verified independent clinics or property leads across Kansas City or Austin for a field representative who only has a &lt;strong&gt;&lt;code&gt;120-minute&lt;/code&gt; afternoon window&lt;/strong&gt; (&lt;code&gt;maxMinutes=120&lt;/code&gt;, with &lt;code&gt;12 minutes&lt;/code&gt; on-site dwell time per stop).&lt;/p&gt;

&lt;h3&gt;
  
  
  Why Road-Adjusted Haversine Is Often Good Enough for Clustering &amp;amp; Ordering
&lt;/h3&gt;

&lt;p&gt;Calling a commercial Distance Matrix API on &lt;code&gt;25&lt;/code&gt; stops requires &lt;code&gt;25 × 25 = 625&lt;/code&gt; matrix elements &lt;em&gt;per run&lt;/em&gt;. Across hundreds of daily territory batches, matrix billing adds up fast.&lt;/p&gt;

&lt;p&gt;For clustering and ordering stops, you often don't need exact road distances. A common shortcut, studied in circuity-factor research (for example Ballou, Rahardja &amp;amp; Sakai, 2002), is to estimate road distance as great-circle &lt;strong&gt;Haversine distance multiplied by a circuity factor&lt;/strong&gt;. The &lt;code&gt;1.28&lt;/code&gt; default below is my own starting value, not a measured figure for any particular city, so calibrate it against a few real routes in your area:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;math&lt;/span&gt;


&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;road_adjusted_haversine_miles&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="n"&gt;lat1&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;float&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;lng1&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;float&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;lat2&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;float&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;lng2&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;float&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;circuity&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;float&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mf"&gt;1.28&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="kt"&gt;float&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="s2"&gt;"""Computes street-grid adjusted distance in statute miles."""&lt;/span&gt;
    &lt;span class="n"&gt;r_miles&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mf"&gt;3958.7613&lt;/span&gt;
    &lt;span class="n"&gt;p1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;p2&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;math&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;radians&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;lat1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;math&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;radians&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;lat2&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;dp&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;math&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;radians&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;lat2&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="n"&gt;lat1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;dl&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;math&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;radians&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;lng2&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="n"&gt;lng1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="n"&gt;a&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;math&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;sin&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;dp&lt;/span&gt; &lt;span class="o"&gt;/&lt;/span&gt; &lt;span class="mf"&gt;2.0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;**&lt;/span&gt; &lt;span class="mi"&gt;2&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="n"&gt;math&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;cos&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;p1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="n"&gt;math&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;cos&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;p2&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="n"&gt;math&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;sin&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;dl&lt;/span&gt; &lt;span class="o"&gt;/&lt;/span&gt; &lt;span class="mf"&gt;2.0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;**&lt;/span&gt; &lt;span class="mi"&gt;2&lt;/span&gt;
    &lt;span class="n"&gt;c&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mf"&gt;2.0&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="n"&gt;math&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;atan2&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;math&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;sqrt&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;a&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;math&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;sqrt&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;max&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mf"&gt;0.0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mf"&gt;1.0&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="n"&gt;a&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;r_miles&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="n"&gt;circuity&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Why Nearest-Neighbor Alone Isn't Enough (And How &lt;code&gt;2-Opt&lt;/code&gt; Fixes It)
&lt;/h3&gt;

&lt;p&gt;A greedy &lt;strong&gt;Nearest-Neighbor&lt;/strong&gt; heuristic ("always drive to the closest unvisited stop") is fast (&lt;code&gt;O(N²)&lt;/code&gt;), but it is notoriously myopic: early greedy hops leave isolated outlier stops behind, forcing long criss-crossing return legs at the end of the shift (on random instances, Johnson &amp;amp; McGeoch's TSP experiments put nearest-neighbor tours roughly 25% above optimal).&lt;/p&gt;

&lt;p&gt;Adding &lt;strong&gt;2-Opt Local Search&lt;/strong&gt; removes most of those crossings. The simple version below recomputes the whole tour length for every candidate swap, so it slows down quickly as &lt;code&gt;N&lt;/code&gt; grows (roughly &lt;code&gt;O(N³)&lt;/code&gt; per pass): fine for &lt;code&gt;15–30&lt;/code&gt; stops, but over a second for &lt;code&gt;60&lt;/code&gt; stops on my laptop (numbers below).&lt;/p&gt;

&lt;p&gt;The geometric intuition behind &lt;code&gt;2-Opt&lt;/code&gt; is simple: take any pair of non-adjacent edges in your tour, reverse the sub-segment &lt;code&gt;tour[i : k + 1]&lt;/code&gt; between them, and check if the total route mileage strictly decreases. Repeat until no edge swap improves the tour:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="nc"&gt;Point&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kt"&gt;tuple&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="kt"&gt;float&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kt"&gt;float&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;


&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;tour_miles&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="n"&gt;pts&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;list&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nc"&gt;Point&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;order&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;list&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="kt"&gt;int&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;origin&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;Point&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;dest&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;Point&lt;/span&gt; &lt;span class="o"&gt;|&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;circuity&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;float&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mf"&gt;1.28&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="kt"&gt;float&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="ow"&gt;not&lt;/span&gt; &lt;span class="n"&gt;order&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="mf"&gt;0.0&lt;/span&gt;
    &lt;span class="n"&gt;dist&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;road_adjusted_haversine_miles&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;origin&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;pts&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;order&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;circuity&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;idx&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="nb"&gt;range&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;order&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;dist&lt;/span&gt; &lt;span class="o"&gt;+=&lt;/span&gt; &lt;span class="nf"&gt;road_adjusted_haversine_miles&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;pts&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;order&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;idx&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;pts&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;order&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;idx&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;circuity&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;dest&lt;/span&gt; &lt;span class="k"&gt;is&lt;/span&gt; &lt;span class="ow"&gt;not&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;dist&lt;/span&gt; &lt;span class="o"&gt;+=&lt;/span&gt; &lt;span class="nf"&gt;road_adjusted_haversine_miles&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;pts&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;order&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;dest&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;circuity&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;dist&lt;/span&gt;


&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;optimize_stops_2opt&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="n"&gt;pts&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;list&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nc"&gt;Point&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;origin&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;Point&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;dest&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;Point&lt;/span&gt; &lt;span class="o"&gt;|&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;circuity&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;float&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mf"&gt;1.28&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="kt"&gt;list&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="kt"&gt;int&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;n&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;pts&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;n&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;=&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="kt"&gt;list&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;range&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;n&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="c1"&gt;# 1. Greedy Nearest-Neighbor seed tour from fixed origin&lt;/span&gt;
    &lt;span class="n"&gt;unvisited&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kt"&gt;set&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;range&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;n&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;tour&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;list&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="kt"&gt;int&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
    &lt;span class="n"&gt;curr&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;origin&lt;/span&gt;
    &lt;span class="k"&gt;while&lt;/span&gt; &lt;span class="n"&gt;unvisited&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;nxt&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;min&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;unvisited&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;key&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="k"&gt;lambda&lt;/span&gt; &lt;span class="n"&gt;i&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nf"&gt;road_adjusted_haversine_miles&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;curr&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;pts&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;i&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;circuity&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="n"&gt;tour&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;append&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;nxt&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="n"&gt;unvisited&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;remove&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;nxt&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="n"&gt;curr&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;pts&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;nxt&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;

    &lt;span class="c1"&gt;# 2. Iterative 2-Opt edge-swap refinement (origin &amp;amp; optional dest remain anchored)&lt;/span&gt;
    &lt;span class="n"&gt;best&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;tour_miles&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;pts&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;tour&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;origin&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;dest&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;circuity&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;improved&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;True&lt;/span&gt;
    &lt;span class="k"&gt;while&lt;/span&gt; &lt;span class="n"&gt;improved&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;improved&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;False&lt;/span&gt;
        &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;i&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="nb"&gt;range&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;n&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;k&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="nb"&gt;range&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;i&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;n&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
                &lt;span class="n"&gt;candidate&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;tour&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="n"&gt;i&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="kt"&gt;list&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;reversed&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;tour&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;i&lt;/span&gt; &lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;k&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="n"&gt;tour&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;k&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt; &lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
                &lt;span class="n"&gt;cand_dist&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;tour_miles&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;pts&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;candidate&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;origin&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;dest&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;circuity&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
                &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;cand_dist&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="mf"&gt;1e-6&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="n"&gt;best&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
                    &lt;span class="n"&gt;tour&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;best&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;candidate&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;cand_dist&lt;/span&gt;
                    &lt;span class="n"&gt;improved&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;True&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;tour&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Benchmark: Random Order vs. Nearest-Neighbor vs. &lt;code&gt;NN + 2-Opt&lt;/code&gt;
&lt;/h3&gt;

&lt;p&gt;I ran the code above on my laptop (Python 3.11): for each size, &lt;code&gt;20&lt;/code&gt; sets of random points inside a ~30 × 30 mile box around Kansas City, closed round trip from the center, &lt;code&gt;circuity = 1.28&lt;/code&gt;. Values are medians. Distances are Haversine × 1.28 estimates, not real road miles, and a random input order is a deliberately bad baseline:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Stop Count (&lt;code&gt;N&lt;/code&gt;)&lt;/th&gt;
&lt;th&gt;Random Input Order&lt;/th&gt;
&lt;th&gt;Nearest-Neighbor Only&lt;/th&gt;
&lt;th&gt;&lt;strong&gt;Nearest-Neighbor + 2-Opt&lt;/strong&gt;&lt;/th&gt;
&lt;th&gt;Solver Time (median / max)&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;&lt;code&gt;6&lt;/code&gt; stops&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;140.4 mi&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;103.4 mi&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;
&lt;strong&gt;&lt;code&gt;97.6 mi&lt;/code&gt;&lt;/strong&gt; (&lt;code&gt;-5.6%&lt;/code&gt; vs NN)&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;0.33 ms&lt;/code&gt; / &lt;code&gt;0.67 ms&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;&lt;code&gt;15&lt;/code&gt; stops&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;303.6 mi&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;148.8 mi&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;
&lt;strong&gt;&lt;code&gt;138.0 mi&lt;/code&gt;&lt;/strong&gt; (&lt;code&gt;-7.2%&lt;/code&gt; vs NN)&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;6.0 ms&lt;/code&gt; / &lt;code&gt;9.8 ms&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;&lt;code&gt;30&lt;/code&gt; stops&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;600.9 mi&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;197.3 mi&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;
&lt;strong&gt;&lt;code&gt;185.0 mi&lt;/code&gt;&lt;/strong&gt; (&lt;code&gt;-6.3%&lt;/code&gt; vs NN)&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;55 ms&lt;/code&gt; / &lt;code&gt;98 ms&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;&lt;code&gt;60&lt;/code&gt; stops&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;1,192.3 mi&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;298.9 mi&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;
&lt;strong&gt;&lt;code&gt;256.0 mi&lt;/code&gt;&lt;/strong&gt; (&lt;code&gt;-14.3%&lt;/code&gt; vs NN)&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;1.28 s&lt;/code&gt; / &lt;code&gt;1.77 s&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Most of the gain over a random order comes from nearest-neighbor; 2-opt then trims roughly another 5–15%. For larger &lt;code&gt;N&lt;/code&gt;, precompute the distance matrix and score each swap by the change in its edge lengths instead of re-summing the whole tour. Also note that in the Actor itself, geocoding through Nominatim (about 1 request per second) takes far longer than the solver.&lt;/p&gt;

&lt;p&gt;Once &lt;code&gt;optimize_stops_2opt&lt;/code&gt; returns the improved visit order (2-opt finds a good local optimum, not a guaranteed optimal tour), enforcing a shift budget (&lt;code&gt;maxMinutes = 120&lt;/code&gt;) is a single linear pass accumulating &lt;code&gt;leg_drive_minutes + service_minutes&lt;/code&gt;—marking stops that fit inside the window as &lt;code&gt;Scheduled&lt;/code&gt; and pushing overflow stops to &lt;code&gt;Deferred&lt;/code&gt; for tomorrow's run, then URL-encoding the scheduled addresses directly into a 1-click mobile &lt;strong&gt;Google Maps Directions link&lt;/strong&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;https://www.google.com/maps/dir/?api=1&amp;amp;origin=...&amp;amp;destination=...&amp;amp;travelmode=driving&amp;amp;waypoints=Stop1|Stop2|Stop3
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  Chaining Both Engines Together in &lt;code&gt;n8n&lt;/code&gt; or &lt;code&gt;Make.com&lt;/code&gt;
&lt;/h2&gt;

&lt;p&gt;Because both engines follow a clean JSON-in / JSON-out contract, chaining them in &lt;strong&gt;n8n&lt;/strong&gt;, &lt;strong&gt;Make.com&lt;/strong&gt;, or &lt;strong&gt;Google Sheets&lt;/strong&gt; takes four nodes:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Ingest:&lt;/strong&gt; Pull raw clinic or business URLs + street addresses from &lt;strong&gt;Google Sheets&lt;/strong&gt;, &lt;strong&gt;HubSpot&lt;/strong&gt;, or a Google Maps scraper.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Enrich &amp;amp; Filter Chains:&lt;/strong&gt; Call &lt;a href="https://github.com/Tthanh2k6/apify-us-healthcare-clinic-contact-enricher" rel="noopener noreferrer"&gt;&lt;code&gt;apify-us-healthcare-clinic-contact-enricher&lt;/code&gt;&lt;/a&gt; (or &lt;a href="https://github.com/Tthanh2k6/apify-b2b-lead-and-contact-enricher" rel="noopener noreferrer"&gt;&lt;code&gt;apify-b2b-lead-and-contact-enricher&lt;/code&gt;&lt;/a&gt; for general B2B tech-stack &amp;amp; contact enrichment) with &lt;code&gt;excludeChainsFromOutput: true&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Optimize Territory Route:&lt;/strong&gt; Pass the surviving &lt;code&gt;Verified Independent&lt;/code&gt; addresses into &lt;a href="https://github.com/Tthanh2k6/apify-multi-stop-route-and-distance-optimizer" rel="noopener noreferrer"&gt;&lt;code&gt;apify-multi-stop-route-and-distance-optimizer&lt;/code&gt;&lt;/a&gt; (for a simpler Haversine-only planner that runs inside Sheets, see the Apps Script example &lt;a href="https://github.com/Tthanh2k6/ai-automation-and-rag-toolkit/blob/main/sql_and_apps_script/google_sheets_haversine_route_planner.js" rel="noopener noreferrer"&gt;&lt;code&gt;google_sheets_haversine_route_planner.js&lt;/code&gt;&lt;/a&gt;; it is written for one specific sheet layout and needs adapting).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Dispatch:&lt;/strong&gt; Push verified &lt;code&gt;MX&lt;/code&gt; emails to your outreach tool and text the 1-click &lt;code&gt;google_maps_nav_url&lt;/code&gt; itinerary straight to your field rep's phone.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;All repositories linked above are public. The three Actors are also published on &lt;a href="https://apify.com/thanh81646" rel="noopener noreferrer"&gt;Apify Store (&lt;code&gt;@thanh81646&lt;/code&gt;)&lt;/a&gt;, so you can call them through Apify's standard synchronous &lt;code&gt;/run-sync-get-dataset-items&lt;/code&gt; endpoint. They are new and have only a few runs so far, so treat them as worked examples rather than battle-tested tools. Feel free to clone, benchmark, or wire them into your own workflows, and issues are welcome!&lt;/p&gt;




&lt;p&gt;&lt;em&gt;I'm &lt;strong&gt;Trung Thanh Tran&lt;/strong&gt;, a Python developer building small data-enrichment and route-planning tools as Apify Actors and scripts.&lt;/em&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;em&gt;Want 6 complete Python 3.11 CLI engines + 3 importable n8n/Make workflow blueprints in one instant download? Grab the &lt;strong&gt;&lt;a href="https://trungthanh16.gumroad.com/l/python-ai-automation-n8n-scraper-toolkit-pro" rel="noopener noreferrer"&gt;Python, AI &amp;amp; n8n Scraper Toolkit Pro on Gumroad&lt;/a&gt;&lt;/strong&gt;.&lt;/em&gt;&lt;/li&gt;
&lt;li&gt;&lt;em&gt;Looking for a custom B2B lead enrichment engine, Google Sheets / Apps Script route planner, or Apify Actor? Hire me on &lt;strong&gt;&lt;a href="https://www.upwork.com/freelancers/trungthanhtrandev" rel="noopener noreferrer"&gt;Upwork&lt;/a&gt;&lt;/strong&gt; or &lt;strong&gt;&lt;a href="https://www.fiverr.com/trungthnhtrn244" rel="noopener noreferrer"&gt;Fiverr&lt;/a&gt;&lt;/strong&gt;.&lt;/em&gt;&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>python</category>
      <category>algorithms</category>
      <category>automation</category>
      <category>opensource</category>
    </item>
    <item>
      <title>Handling Broken LLM JSON with Strict Schemas and finish_reason Guards (FastAPI, Node.js &amp; n8n)</title>
      <dc:creator>Trung Thanh Tran</dc:creator>
      <pubDate>Fri, 30 Oct 2026 03:35:13 +0000</pubDate>
      <link>https://dev.to/tthanh2k6/eliminating-broken-llm-json-in-production-workflows-nodejs-fastapi-n8n-strict-schemas-3fli</link>
      <guid>https://dev.to/tthanh2k6/eliminating-broken-llm-json-in-production-workflows-nodejs-fastapi-n8n-strict-schemas-3fli</guid>
      <description>&lt;p&gt;If you have ever wired an LLM into a CRM, billing webhook, or database ingestion workflow, you have probably met the 2:00 AM &lt;code&gt;SyntaxError: Unexpected end of JSON input&lt;/code&gt; alert.&lt;/p&gt;

&lt;p&gt;At &lt;code&gt;10&lt;/code&gt; test runs on your laptop, telling a model &lt;em&gt;"Respond ONLY with valid JSON, no markdown fences"&lt;/em&gt; feels like it works. At &lt;code&gt;10,000&lt;/code&gt; automated runs per week inside an &lt;strong&gt;n8n&lt;/strong&gt;, &lt;strong&gt;Make.com&lt;/strong&gt;, &lt;strong&gt;Node.js&lt;/strong&gt;, or &lt;strong&gt;FastAPI&lt;/strong&gt; pipeline, prompt-only JSON fails in four predictable ways:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Silent Key Drift:&lt;/strong&gt; The model outputs &lt;code&gt;urgencyScore&lt;/code&gt; on Tuesday and &lt;code&gt;urgency_score&lt;/code&gt; on Wednesday, leaving downstream CRM fields &lt;code&gt;null&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Markdown Preamble Bleed:&lt;/strong&gt; Under edge-case inputs, the model wraps the payload in

&lt;code&gt;```json ... ```&lt;/code&gt;

or adds &lt;em&gt;"Sure, here is the extracted payload:"&lt;/em&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Mid-String Token Truncation (&lt;code&gt;finish_reason: "length"&lt;/code&gt;):&lt;/strong&gt; A verbose customer email causes the completion to hit &lt;code&gt;max_tokens&lt;/code&gt; halfway through a string (&lt;code&gt;{"summary": "Customer requested an enterprise...&lt;/code&gt;), crashing &lt;code&gt;JSON.parse()&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Safety Refusal Collisions:&lt;/strong&gt; When an input triggers a content policy refusal, the assistant message body is plain prose while &lt;code&gt;message.content&lt;/code&gt; is empty or non-JSON.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Many teams respond by stacking regex strippers and retry-repair prompts (&lt;code&gt;jsonrepair&lt;/code&gt;) on top of broken outputs—adding latency and token cost without fixing schema drift.&lt;/p&gt;

&lt;p&gt;The real fix is architectural: &lt;strong&gt;Constrained Token Decoding (&lt;code&gt;strict: true&lt;/code&gt; JSON Schemas) paired with &lt;code&gt;finish_reason&lt;/code&gt; state guards and SHA-256 webhook idempotency&lt;/strong&gt;. Here is how to engineer that stack cleanly across &lt;strong&gt;FastAPI (Python)&lt;/strong&gt;, &lt;strong&gt;Node.js / TypeScript&lt;/strong&gt;, and &lt;strong&gt;n8n&lt;/strong&gt;.&lt;/p&gt;




&lt;h2&gt;
  
  
  Prompt Mode vs. &lt;code&gt;json_object&lt;/code&gt; vs. Strict &lt;code&gt;json_schema&lt;/code&gt;
&lt;/h2&gt;

&lt;p&gt;Before writing code, it helps to understand what happens inside the inference engine across the three JSON modes:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Mechanism&lt;/th&gt;
&lt;th&gt;How It Works at Decode Time&lt;/th&gt;
&lt;th&gt;Guarantees Valid JSON Syntax?&lt;/th&gt;
&lt;th&gt;Guarantees Exact Keys, Types &amp;amp; Enums?&lt;/th&gt;
&lt;th&gt;Failure Mode to Watch&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;1. System Prompt Only&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Purely probabilistic next-token prediction&lt;/td&gt;
&lt;td&gt;No&lt;/td&gt;
&lt;td&gt;No&lt;/td&gt;
&lt;td&gt;Markdown fences, trailing commas, key hallucinations&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;2. Legacy JSON Mode (&lt;code&gt;json_object&lt;/code&gt;)&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Output is constrained to valid JSON syntax, with no schema&lt;/td&gt;
&lt;td&gt;Yes (unless truncated)&lt;/td&gt;
&lt;td&gt;
&lt;strong&gt;No&lt;/strong&gt; (keys/types can still drift)&lt;/td&gt;
&lt;td&gt;Missing required fields or wrong nested types&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;3. Structured Outputs (&lt;code&gt;json_schema&lt;/code&gt;, &lt;code&gt;strict: true&lt;/code&gt;)&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Compiles your JSON Schema into a &lt;strong&gt;context-free grammar (CFG)&lt;/strong&gt; masking logits at every token step&lt;/td&gt;
&lt;td&gt;
&lt;strong&gt;Yes&lt;/strong&gt; (unless truncated)&lt;/td&gt;
&lt;td&gt;
&lt;strong&gt;Yes&lt;/strong&gt; (unless truncated or refused)&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;finish_reason == "length"&lt;/code&gt; or explicit &lt;code&gt;refusal&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;With &lt;code&gt;strict: true&lt;/code&gt;, the model literally &lt;em&gt;cannot&lt;/em&gt; sample a token that violates your property names, enum values, or required types—because invalid tokens have their logit probabilities masked to &lt;code&gt;-∞&lt;/code&gt; during sampling. (When OpenAI launched Structured Outputs in August 2024, it reported a 100% score on its own complex-schema evals in strict mode. Truncation and refusals are the failure modes that remain, and the guards below handle them.)&lt;/p&gt;

&lt;h3&gt;
  
  
  The Three Strict-Grammar Rules Most Developers Miss
&lt;/h3&gt;

&lt;p&gt;If your &lt;code&gt;strict: true&lt;/code&gt; request returns a &lt;code&gt;400 Invalid Schema&lt;/code&gt; error from OpenAI (or compatible vLLM / structured-decoding endpoints), check these three constraints:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Every &lt;code&gt;object&lt;/code&gt; level&lt;/strong&gt; (root and nested) &lt;strong&gt;must&lt;/strong&gt; set &lt;code&gt;"additionalProperties": false&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Every property&lt;/strong&gt; defined in &lt;code&gt;"properties"&lt;/code&gt; &lt;strong&gt;must&lt;/strong&gt; be listed in &lt;code&gt;"required"&lt;/code&gt; (for optional values, use a union type like &lt;code&gt;["string", "null"]&lt;/code&gt; rather than omitting the key from &lt;code&gt;required&lt;/code&gt;).&lt;/li&gt;
&lt;li&gt;Keep recursive nesting shallow and prefer explicit &lt;code&gt;enum&lt;/code&gt; arrays over open-ended strings for routing keys.&lt;/li&gt;
&lt;/ol&gt;




&lt;h2&gt;
  
  
  Reference Implementation #1: FastAPI + Pydantic Strict Service (Python 3.11)
&lt;/h2&gt;

&lt;p&gt;In Python, &lt;code&gt;client.beta.chat.completions.parse()&lt;/code&gt; compiles Pydantic &lt;code&gt;BaseModel&lt;/code&gt; classes into strict JSON schemas automatically—&lt;strong&gt;if&lt;/strong&gt; you guard &lt;code&gt;finish_reason&lt;/code&gt; and &lt;code&gt;message.refusal&lt;/code&gt; before touching &lt;code&gt;.parsed&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Here is a FastAPI endpoint pattern built on the same &lt;code&gt;.parse()&lt;/code&gt; call I use in &lt;a href="https://github.com/Tthanh2k6/ai-automation-and-rag-toolkit/blob/main/src/fastapi_openai_structured_rag_service.py" rel="noopener noreferrer"&gt;&lt;code&gt;fastapi_openai_structured_rag_service.py&lt;/code&gt;&lt;/a&gt;. That file only has the basic call plus retry; the &lt;code&gt;finish_reason&lt;/code&gt;, refusal and idempotency guards below are additions for this post:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;hashlib&lt;/span&gt;
&lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;enum&lt;/span&gt; &lt;span class="nf"&gt;import&lt;/span&gt; &lt;span class="nc"&gt;Enum&lt;/span&gt;
&lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;typing&lt;/span&gt; &lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="nc"&gt;Optional&lt;/span&gt;
&lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;fastapi&lt;/span&gt; &lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="nc"&gt;FastAPI&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nc"&gt;HTTPException&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nc"&gt;Header&lt;/span&gt;
&lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;openai&lt;/span&gt; &lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="nc"&gt;AsyncOpenAI&lt;/span&gt;
&lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;pydantic&lt;/span&gt; &lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="nc"&gt;BaseModel&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nc"&gt;ConfigDict&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nc"&gt;Field&lt;/span&gt;

&lt;span class="n"&gt;app&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;FastAPI&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;title&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"Deterministic Lead Triage Pipeline"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;client&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;AsyncOpenAI&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;


&lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;RouteTeam&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kt"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nc"&gt;Enum&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="no"&gt;REVOPS_ENTERPRISE&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"revops_enterprise"&lt;/span&gt;
    &lt;span class="no"&gt;SELF_SERVE_SUPPORT&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"self_serve_support"&lt;/span&gt;
    &lt;span class="no"&gt;BILLING_OPS&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"billing_ops"&lt;/span&gt;
    &lt;span class="no"&gt;SPAM_QUARANTINE&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"spam_quarantine"&lt;/span&gt;


&lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;NextAction&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kt"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nc"&gt;Enum&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="no"&gt;CREATE_HUBSPOT_DEAL&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"create_hubspot_deal"&lt;/span&gt;
    &lt;span class="no"&gt;OPEN_ZENDESK_TICKET&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"open_zendesk_ticket"&lt;/span&gt;
    &lt;span class="no"&gt;ARCHIVE_NO_ACTION&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"archive_no_action"&lt;/span&gt;


&lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;LeadTriageDecision&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nc"&gt;BaseModel&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;model_config&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;ConfigDict&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;extra&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"forbid"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="n"&gt;intent_summary&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;str&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Field&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;description&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"Concise 1-sentence summary under 160 chars."&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;urgency_score&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;int&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Field&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;ge&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;le&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;100&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;owner_team&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;RouteTeam&lt;/span&gt;
    &lt;span class="n"&gt;next_action&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;NextAction&lt;/span&gt;
    &lt;span class="n"&gt;extracted_company_domain&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;Optional&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="kt"&gt;str&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Field&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="n"&gt;default&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="kc"&gt;None&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;description&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"Normalized company domain or null if consumer email."&lt;/span&gt;
    &lt;span class="p"&gt;)&lt;/span&gt;


&lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;InboundWebhookPayload&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nc"&gt;BaseModel&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;source_event_id&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;str&lt;/span&gt;
    &lt;span class="n"&gt;sender_email&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;str&lt;/span&gt;
    &lt;span class="n"&gt;raw_message&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;str&lt;/span&gt;


&lt;span class="c1"&gt;# Replace with Redis SET NX EX in multi-worker deployments&lt;/span&gt;
&lt;span class="no"&gt;PROCESSED_KEYS&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;set&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="kt"&gt;str&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kt"&gt;set&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;


&lt;span class="nd"&gt;@app&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;post&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"/v1/triage"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;triage_inbound_lead&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="n"&gt;payload&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;InboundWebhookPayload&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;x_idempotency_key&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;Optional&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="kt"&gt;str&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Header&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;default&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="kc"&gt;None&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="kt"&gt;dict&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="c1"&gt;# 1. Deterministic Idempotency Guard (prevents duplicate HubSpot deals on webhook retries)&lt;/span&gt;
    &lt;span class="n"&gt;idem_raw&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;x_idempotency_key&lt;/span&gt; &lt;span class="ow"&gt;or&lt;/span&gt; &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"{payload.source_event_id}:{payload.sender_email}"&lt;/span&gt;
    &lt;span class="n"&gt;idem_hash&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;hashlib&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;sha256&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;idem_raw&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;encode&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"utf-8"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;hexdigest&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="mi"&gt;24&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;idem_hash&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="no"&gt;PROCESSED_KEYS&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"status"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"duplicate_ignored"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"idempotency_key"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;idem_hash&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="c1"&gt;# 2. Constrained Decoding Call with Strict Pydantic Schema&lt;/span&gt;
    &lt;span class="n"&gt;completion&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;beta&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;chat&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;completions&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;parse&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="n"&gt;model&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"gpt-4o-mini"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;temperature&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mf"&gt;0.0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;max_completion_tokens&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;450&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;messages&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;
            &lt;span class="p"&gt;{&lt;/span&gt;
                &lt;span class="s2"&gt;"role"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"system"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                &lt;span class="s2"&gt;"content"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;
                    &lt;span class="s2"&gt;"Classify the inbound message into the strict triage schema. "&lt;/span&gt;
                    &lt;span class="s2"&gt;"Never invent domains not present in sender_email or message body."&lt;/span&gt;
                &lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="p"&gt;{&lt;/span&gt;
                &lt;span class="s2"&gt;"role"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"user"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                &lt;span class="s2"&gt;"content"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"From: {payload.sender_email}\n\nBody:\n{payload.raw_message[:6000]}"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;response_format&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="nc"&gt;LeadTriageDecision&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="n"&gt;choice&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;completion&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;choices&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;

    &lt;span class="c1"&gt;# 3. Critical Production Guards: Truncation &amp;amp; Policy Refusals&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;choice&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;finish_reason&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="s2"&gt;"length"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;raise&lt;/span&gt; &lt;span class="nc"&gt;HTTPException&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
            &lt;span class="n"&gt;status_code&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;502&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="n"&gt;detail&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"LLM output hit max_completion_tokens; refusing to emit truncated JSON."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;choice&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;message&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;refusal&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;raise&lt;/span&gt; &lt;span class="nc"&gt;HTTPException&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
            &lt;span class="n"&gt;status_code&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;422&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="n"&gt;detail&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"Model safety refusal: {choice.message.refusal}"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="n"&gt;parsed&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;LeadTriageDecision&lt;/span&gt; &lt;span class="o"&gt;|&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;choice&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;message&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;parsed&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;parsed&lt;/span&gt; &lt;span class="k"&gt;is&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;raise&lt;/span&gt; &lt;span class="nc"&gt;HTTPException&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;status_code&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;500&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;detail&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"Parsed schema payload missing."&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="no"&gt;PROCESSED_KEYS&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;add&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;idem_hash&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="s2"&gt;"status"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"processed"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="s2"&gt;"idempotency_key"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;idem_hash&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="s2"&gt;"finish_reason"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;choice&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;finish_reason&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="s2"&gt;"decision"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;parsed&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;model_dump&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Notice what happens when &lt;code&gt;choice.finish_reason == "length"&lt;/code&gt; is checked &lt;em&gt;before&lt;/em&gt; downstream side effects: instead of writing corrupted state or silently dropping fields, your webhook returns a clean &lt;code&gt;502&lt;/code&gt; or routes the oversized payload to a chunking/summarization dead-letter queue.&lt;/p&gt;




&lt;h2&gt;
  
  
  Reference Implementation #2: Node.js / TypeScript with &lt;code&gt;zodResponseFormat&lt;/code&gt;
&lt;/h2&gt;

&lt;p&gt;If your backend or serverless worker runs on &lt;strong&gt;Node.js / TypeScript&lt;/strong&gt;, pair &lt;code&gt;zod&lt;/code&gt; with &lt;code&gt;zodResponseFormat&lt;/code&gt; from the official &lt;code&gt;openai/helpers/zod&lt;/code&gt; module so your compile-time TypeScript types and runtime LLM decoding grammar come from a single source of truth:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="nc"&gt;OpenAI&lt;/span&gt; &lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="s2"&gt;"openai"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="n"&gt;zodResponseFormat&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="s2"&gt;"openai/helpers/zod"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="n"&gt;z&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="s2"&gt;"zod"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;openai&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nc"&gt;OpenAI&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="n"&gt;apiKey&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;process&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;env&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="no"&gt;OPENAI_API_KEY&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nc"&gt;InvoiceExtractionSchema&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;z&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="kt"&gt;object&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="n"&gt;vendor_name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;z&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="kt"&gt;string&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;invoice_number&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;z&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="kt"&gt;string&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;currency&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;z&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="kd"&gt;enum&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"USD"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"EUR"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"GBP"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"CAD"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"AUD"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;line_items&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;z&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;array&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="n"&gt;z&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="kt"&gt;object&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
      &lt;span class="n"&gt;description&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;z&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="kt"&gt;string&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
      &lt;span class="n"&gt;quantity&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;z&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="kt"&gt;number&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
      &lt;span class="n"&gt;unit_price&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;z&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="kt"&gt;number&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
      &lt;span class="n"&gt;line_total&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;z&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="kt"&gt;number&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
  &lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;subtotal&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;z&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="kt"&gt;number&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;tax_amount&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;z&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="kt"&gt;number&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;grand_total&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;z&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="kt"&gt;number&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="k"&gt;export&lt;/span&gt; &lt;span class="kd"&gt;type&lt;/span&gt; &lt;span class="nc"&gt;InvoiceExtraction&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;z&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;infer&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="k"&gt;typeof&lt;/span&gt; &lt;span class="nc"&gt;InvoiceExtractionSchema&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="k"&gt;export&lt;/span&gt; &lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;extractVerifiedInvoice&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ocrText&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;string&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;completion&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;openai&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;beta&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;chat&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;completions&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;parse&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="n"&gt;model&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"gpt-4o-mini"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;temperature&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;max_completion_tokens&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;1200&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;messages&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
      &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="n"&gt;role&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"system"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;content&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"Extract structured invoice fields accurately from raw OCR text."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
      &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
      &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="n"&gt;role&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"user"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;content&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;ocrText&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;slice&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;12000&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;response_format&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nf"&gt;zodResponseFormat&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nc"&gt;InvoiceExtractionSchema&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"invoice_extraction_v1"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;choice&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;completion&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;choices&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;choice&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;finish_reason&lt;/span&gt; &lt;span class="o"&gt;===&lt;/span&gt; &lt;span class="s2"&gt;"length"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;throw&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nb"&gt;Error&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"Token budget exceeded (finish_reason=length); increase max_completion_tokens."&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
  &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;choice&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;message&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;refusal&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="n"&gt;choice&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;message&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;parsed&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;throw&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nb"&gt;Error&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sb"&gt;`Extraction refused or empty: ${choice.message.refusal ?? "unknown"}`&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;

  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;data&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;choice&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;message&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;parsed&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

  &lt;span class="c1"&gt;// Deterministic Post-LLM Arithmetic Invariant Check!&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;computedSum&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;data&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;line_items&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;reduce&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;acc&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;item&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="n"&gt;acc&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="n"&gt;item&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;line_total&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;mathDelta&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;Math&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;abs&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;computedSum&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="n"&gt;data&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;tax_amount&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="n"&gt;data&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;grand_total&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

  &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;data&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;needs_human_audit&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;mathDelta&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="mf"&gt;0.05&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;computed_delta&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;Number&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;mathDelta&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;toFixed&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Pro-Tip: Separate Structure from Arithmetic Verification
&lt;/h3&gt;

&lt;p&gt;Strict JSON schemas guarantee &lt;strong&gt;structural validity&lt;/strong&gt; (&lt;code&gt;grand_total&lt;/code&gt; is a &lt;code&gt;number&lt;/code&gt;), not &lt;strong&gt;mathematical truth&lt;/strong&gt; (&lt;code&gt;2 + 2 == 4&lt;/code&gt; on smudged OCR text). Adding a 3-line deterministic invariant (&lt;code&gt;mathDelta &amp;gt; 0.05 -&amp;gt; needs_human_audit: true&lt;/code&gt;) flags invoices whose numbers don't add up for human review, instead of spot-checking every one by hand.&lt;/p&gt;




&lt;h2&gt;
  
  
  Wiring Strict Schemas Directly Inside &lt;code&gt;n8n&lt;/code&gt; (Without Fragile AI-Agent Loops)
&lt;/h2&gt;

&lt;p&gt;In &lt;strong&gt;n8n&lt;/strong&gt;, a common pitfall is dropping the high-level &lt;em&gt;AI Agent&lt;/em&gt; + &lt;em&gt;Structured Output Parser&lt;/em&gt; sub-node onto high-volume webhooks. As far as I can tell from the docs, the Structured Output Parser sub-node works by adding format instructions to the prompt (and the Auto-fixing Output Parser retries with another LLM call) rather than sending a native API-level &lt;code&gt;response_format.type = "json_schema"&lt;/code&gt;. Check how your n8n version behaves before relying on it.&lt;/p&gt;

&lt;p&gt;For mission-critical &lt;strong&gt;n8n&lt;/strong&gt; or &lt;strong&gt;Make.com&lt;/strong&gt; workflows, call the Chat Completions endpoint directly via an &lt;strong&gt;HTTP Request&lt;/strong&gt; node with explicit &lt;code&gt;json_schema&lt;/code&gt; enforcement:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Node 1 (&lt;code&gt;Webhook&lt;/code&gt;):&lt;/strong&gt; Receive inbound lead/document payload.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Node 2 (&lt;code&gt;Crypto&lt;/code&gt; -&amp;gt; SHA-256):&lt;/strong&gt; Hash &lt;code&gt;{{$json.event_id}}&lt;/code&gt; and check against Redis / Data Store / Supabase to drop duplicate retries immediately.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Node 3 (&lt;code&gt;HTTP Request&lt;/code&gt; -&amp;gt; &lt;code&gt;POST https://api.openai.com/v1/chat/completions&lt;/code&gt;):&lt;/strong&gt; Pass the exact JSON body below:
&lt;/li&gt;
&lt;/ol&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nl"&gt;"model"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"gpt-4o-mini"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"temperature"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"max_completion_tokens"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;500&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"messages"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
    &lt;span class="p"&gt;{&lt;/span&gt;
      &lt;span class="nl"&gt;"role"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"system"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
      &lt;span class="nl"&gt;"content"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"Extract B2B routing fields strictly according to the schema."&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;{&lt;/span&gt;
      &lt;span class="nl"&gt;"role"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"user"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
      &lt;span class="nl"&gt;"content"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="n"&gt;JSON&lt;/span&gt;&lt;span class="err"&gt;.&lt;/span&gt;&lt;span class="n"&gt;stringify&lt;/span&gt;&lt;span class="err"&gt;(&lt;/span&gt;&lt;span class="n"&gt;JSON&lt;/span&gt;&lt;span class="err"&gt;.&lt;/span&gt;&lt;span class="n"&gt;stringify&lt;/span&gt;&lt;span class="err"&gt;($&lt;/span&gt;&lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="err"&gt;.&lt;/span&gt;&lt;span class="n"&gt;body&lt;/span&gt;&lt;span class="err"&gt;).&lt;/span&gt;&lt;span class="n"&gt;slice&lt;/span&gt;&lt;span class="err"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;6000&lt;/span&gt;&lt;span class="err"&gt;))&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
  &lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"response_format"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nl"&gt;"type"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"json_schema"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="nl"&gt;"json_schema"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
      &lt;span class="nl"&gt;"name"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"n8n_crm_router_v1"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
      &lt;span class="nl"&gt;"strict"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
      &lt;span class="nl"&gt;"schema"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="nl"&gt;"type"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"object"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="nl"&gt;"additionalProperties"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="nl"&gt;"properties"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
          &lt;span class="nl"&gt;"intent"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="nl"&gt;"type"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"string"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="nl"&gt;"enum"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"demo_request"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"support_issue"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"partnership"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"noise"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
          &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
          &lt;span class="nl"&gt;"priority"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nl"&gt;"type"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"integer"&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
          &lt;span class="nl"&gt;"crm_note"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nl"&gt;"type"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"string"&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;
        &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="nl"&gt;"required"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"intent"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"priority"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"crm_note"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
      &lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;(The outer &lt;code&gt;JSON.stringify&lt;/code&gt; makes n8n insert a properly quoted and escaped JSON string. Wrapping a single &lt;code&gt;JSON.stringify&lt;/code&gt; in quotes breaks the request body as soon as the payload contains a &lt;code&gt;"&lt;/code&gt; character.)&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Node 4 (&lt;code&gt;IF&lt;/code&gt; Guard Node):&lt;/strong&gt; Assert &lt;code&gt;{{ $json.choices[0].finish_reason === "stop" }}&lt;/code&gt; before &lt;code&gt;JSON.parse($json.choices[0].message.content)&lt;/code&gt; routes into &lt;strong&gt;HubSpot&lt;/strong&gt;, &lt;strong&gt;Pipedrive&lt;/strong&gt;, &lt;strong&gt;Slack&lt;/strong&gt;, or &lt;strong&gt;Google Sheets&lt;/strong&gt;. Any non-&lt;code&gt;stop&lt;/code&gt; completion branches cleanly into your error-alert channel with full input context preserved.&lt;/li&gt;
&lt;/ol&gt;




&lt;h2&gt;
  
  
  Production Checklist Before You Ship
&lt;/h2&gt;

&lt;p&gt;Before promoting any LLM automation from staging to production, verify all five invariants:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;[ ] &lt;code&gt;"strict": true&lt;/code&gt; and &lt;code&gt;"additionalProperties": false&lt;/code&gt; are set on &lt;strong&gt;every&lt;/strong&gt; nested object in your schema.&lt;/li&gt;
&lt;li&gt;[ ] &lt;code&gt;temperature: 0&lt;/code&gt; is set for deterministic classification and extraction tasks.&lt;/li&gt;
&lt;li&gt;[ ] &lt;code&gt;finish_reason === "length"&lt;/code&gt; and &lt;code&gt;message.refusal&lt;/code&gt; are explicitly handled before &lt;code&gt;JSON.parse()&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;[ ] Input text length is sliced/bounded (&lt;code&gt;slice(0, N)&lt;/code&gt;) so a single &lt;code&gt;5 MB&lt;/code&gt; email thread cannot blow out context limits.&lt;/li&gt;
&lt;li&gt;[ ] Downstream CRM/database writes are guarded by an idempotency hash (&lt;code&gt;event_id&lt;/code&gt; + &lt;code&gt;schema_version&lt;/code&gt;).&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;You can explore related example code (FastAPI with structured outputs, plus a hybrid RAG search example) in my public repository: &lt;strong&gt;&lt;a href="https://github.com/Tthanh2k6/ai-automation-and-rag-toolkit" rel="noopener noreferrer"&gt;&lt;code&gt;ai-automation-and-rag-toolkit&lt;/code&gt;&lt;/a&gt;&lt;/strong&gt;.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;I'm &lt;strong&gt;Trung Thanh Tran&lt;/strong&gt;, a Python developer building small automation tools, Apify Actors and n8n workflows, and writing up what I learn.&lt;/em&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;em&gt;Want 3 ready-to-import &lt;strong&gt;n8n &amp;amp; Make.com JSON workflow blueprints&lt;/strong&gt; + 6 Python CLI scripts (including an n8n/Make workflow generator and an API bug-fix diagnostic script)? Check out the &lt;strong&gt;&lt;a href="https://trungthanh16.gumroad.com/l/python-ai-automation-n8n-scraper-toolkit-pro" rel="noopener noreferrer"&gt;Python, AI &amp;amp; n8n Scraper Toolkit Pro on Gumroad&lt;/a&gt;&lt;/strong&gt;.&lt;/em&gt;&lt;/li&gt;
&lt;li&gt;&lt;em&gt;Need help debugging broken n8n/Make scenarios, FastAPI microservices, or RAG pipelines? Connect with me on &lt;strong&gt;&lt;a href="https://www.upwork.com/freelancers/trungthanhtrandev" rel="noopener noreferrer"&gt;Upwork&lt;/a&gt;&lt;/strong&gt; or &lt;strong&gt;&lt;a href="https://www.fiverr.com/trungthnhtrn244" rel="noopener noreferrer"&gt;Fiverr&lt;/a&gt;&lt;/strong&gt;.&lt;/em&gt;&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>ai</category>
      <category>javascript</category>
      <category>python</category>
      <category>automation</category>
    </item>
  </channel>
</rss>
