<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Nan</title>
    <description>The latest articles on DEV Community by Nan (@eye_java_420f1faa10ae8b86).</description>
    <link>https://dev.to/eye_java_420f1faa10ae8b86</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4142919%2F40b26578-b7dd-4d20-85a0-379f7587396f.png</url>
      <title>DEV Community: Nan</title>
      <link>https://dev.to/eye_java_420f1faa10ae8b86</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/eye_java_420f1faa10ae8b86"/>
    <language>en</language>
    <item>
      <title>The Form That Failed Me for a Mouse</title>
      <dc:creator>Nan</dc:creator>
      <pubDate>Thu, 19 Nov 2026 12:37:34 +0000</pubDate>
      <link>https://dev.to/eye_java_420f1faa10ae8b86/the-form-that-failed-me-for-a-mouse-1edb</link>
      <guid>https://dev.to/eye_java_420f1faa10ae8b86/the-form-that-failed-me-for-a-mouse-1edb</guid>
      <description>&lt;p&gt;&lt;em&gt;This is a field report from November 2026, based on one debugging session on 6 November. The site involved can change its checks at any time, so treat the specifics as a snapshot of that session, not a stable description of how the site behaves now.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;Every evening, one of my scheduled jobs pitches a few of my sites to third-party listing and catalog sites through their public contact forms. On 2 November, a directory called edit.org started failing. The form began accepting submissions again only after I fixed three separate blockers: a consent control my selector never clicked, a mousemove check that only counts trusted events, and a minimum-time check that rejects fast submits. The rest of this post is the four days it took to find them.&lt;/p&gt;

&lt;h2&gt;
  
  
  The failure looked like their outage, so I blamed their outage
&lt;/h2&gt;

&lt;p&gt;The frustrating part of this class of bug is that it produces no error. No HTTP 403, no CAPTCHA challenge, no "access denied" page. The form just never fires. For four days my pipeline logged FAIL for this channel, and because the failure was silent, I filed it under "site's problem, maybe their form is broken." I have written about this exact shape of mistake before — &lt;a href="https://javaeye.dev/blog/silent-failure-unattended-automation/" rel="noopener noreferrer"&gt;silent failures in unattended automation&lt;/a&gt; — and I still made it again. When a scheduled job fails the same way twice, the first question should be "what is the page doing?", not "what is wrong with the site?"&lt;/p&gt;

&lt;p&gt;Separately, that same week, I had to roll back falsely recorded successes on a different channel after its post-submit check turned out to match static page text. Two channels, opposite failure directions, same root cause family: my own checks were asserting things the page never confirmed.&lt;/p&gt;

&lt;h2&gt;
  
  
  My button-only selector missed the consent anchor
&lt;/h2&gt;

&lt;p&gt;The page greets you with a cookie consent banner that overlays the lower part of the viewport, and the contact form sits behind it. For days my script's dismiss step ran without error and did nothing. The reason: my selector only searched for a &lt;code&gt;&amp;lt;button&amp;gt;&lt;/code&gt;, and the banner's accept control is an &lt;code&gt;&amp;lt;a&amp;gt;&lt;/code&gt; element. The lookup found zero elements, and the code treated zero matches as "nothing to dismiss" instead of "something is wrong." The banner stayed up for the whole session.&lt;/p&gt;

&lt;p&gt;The fix was to drop the tag restriction: find any element whose text matches &lt;code&gt;/accept cookies$/i&lt;/code&gt; and click that. One line. The banner disappeared and the form became fully visible. But the submission still did not go through, because the page had a second blocker waiting.&lt;/p&gt;

&lt;h2&gt;
  
  
  The mouse check only counted events the browser considered trusted
&lt;/h2&gt;

&lt;p&gt;The page carries an inline anti-bot check built around a variable I will paraphrase as &lt;code&gt;mouseMoved&lt;/code&gt;. The submit handler asks, in effect: did this visitor's mouse actually move across the document while the form was open?&lt;/p&gt;

&lt;p&gt;Two things make this check nastier than it sounds. First, it only counts events with &lt;code&gt;isTrusted === true&lt;/code&gt;. That flag is set by the browser for input the browser itself generated, and JavaScript running on the page cannot set it — &lt;code&gt;dispatchEvent()&lt;/code&gt; always produces untrusted events, by design. My old script synthesized mousemoves by dispatching JS events on the document. They ran, they were just permanently untrusted, so the flag stayed false. No error, no hint, just a form that declines to send.&lt;/p&gt;

&lt;p&gt;Second, even the "real input" path had a trap in my setup. I drive the browser over the Chrome DevTools Protocol, and CDP's &lt;code&gt;Input.dispatchMouseEvent&lt;/code&gt; does produce events the browser treats as trusted. But I run several channels through one browser instance, and the edit.org tab sat in the background while other tabs worked. In that session, my document-level mousemove listener did not register the dispatched trajectory until I brought the tab forward with &lt;code&gt;Page.bringToFront&lt;/code&gt; and replayed the mouse path again. Whether background tabs drop these events in general depends on the browser version and event type; what I can attest to is this session, this setup, before and after foregrounding.&lt;/p&gt;

&lt;p&gt;The working sequence is now fixed in the script: bring the tab to the front, then dispatch a short mouse trajectory across the document. Only then does the flag flip.&lt;/p&gt;

&lt;h2&gt;
  
  
  Seven seconds was too early; 25 seconds went through
&lt;/h2&gt;

&lt;p&gt;Even with the banner gone and the mouse flag set, quick submissions still bounced. The page has an inline dwell check: it records a start time at load and compares the elapsed time against a minimum at submit. Rushing the form earns a "✋ Wait…" message and a refusal.&lt;/p&gt;

&lt;p&gt;That evening I tested two points instead of guessing: a submit 7 seconds after page load was rejected, and the same submission at 25 seconds was accepted. With the other checks satisfied, those two tests bracket a fixed dwell threshold between the rejected 7-second attempt and the accepted 25-second attempt; they do not identify the exact cutoff. Since that run, I have simply waited 30 seconds before submitting. If I ever need the exact number, the inline script on the page has it — that is where the start time is read from, not from any server round trip.&lt;/p&gt;

&lt;h2&gt;
  
  
  The revised script reached the confirmation page that evening
&lt;/h2&gt;

&lt;p&gt;The timeline compressed to three minutes. At 21:47 on 6 November, the old script made one last attempt with synthetic mouse events and the form did not send. At 21:50, the revised script — tag-agnostic banner click, foreground tab, trusted mouse trajectory, 30-second dwell — submitted the same pitch for the same site and reached the directory's &lt;code&gt;?thanks=1&lt;/code&gt; confirmation URL, which I record as this channel's submission-success signal. I kept the confirmation screenshot and the ledger entry, and the sites that had queued behind this channel for days went out the same night.&lt;/p&gt;

&lt;p&gt;The other change I shipped that evening was diagnostic, and worth more than the fix: the pre-submit check now treats "selector matched zero elements" as a hard failure with a screenshot, instead of a silent skip. The four lost days were not caused by three clever anti-bot tricks. They were caused by my own code declining to tell me it had found nothing.&lt;/p&gt;




&lt;p&gt;Originally published on my blog: &lt;a href="https://javaeye.dev/blog/the-form-that-failed-me-for-a-mouse/" rel="noopener noreferrer"&gt;https://javaeye.dev/blog/the-form-that-failed-me-for-a-mouse/&lt;/a&gt;&lt;/p&gt;

</description>
      <category>testing</category>
      <category>automation</category>
      <category>debugging</category>
      <category>webdev</category>
    </item>
    <item>
      <title>The 0.00% My Tests Enforced and Math Refused</title>
      <dc:creator>Nan</dc:creator>
      <pubDate>Wed, 18 Nov 2026 12:27:58 +0000</pubDate>
      <link>https://dev.to/eye_java_420f1faa10ae8b86/the-000-my-tests-enforced-and-math-refused-53db</link>
      <guid>https://dev.to/eye_java_420f1faa10ae8b86/the-000-my-tests-enforced-and-math-refused-53db</guid>
      <description>&lt;p&gt;&lt;em&gt;This is a field report from November 2026. The simulation code it describes runs at build time with deterministic seeds, so every number in it is reproducible — but the specific claims belong to one site in my portfolio at one point in time. Verify from the seeds, not from my memory.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;Earlier this week, an automated review pass flagged a claim in a published data post on one of my tool sites: a history-aware team redraw that, per the post’s own FAQ, “collapsed” the repeat-pairing rate “from 4.7% to 0.0% across the entire simulation” — zero repeats in 9,999 consecutive draws. The flagged number was not merely wrong. It was impossible: with 24 players split into 4 teams of 6, zero repeated pairs across two consecutive draws cannot happen, and the mathematical floor is 8. The worse finding came after I opened the repo: my regression test had been asserting the impossible value as the expected output. This post is the re-measurement, the fix (commit &lt;code&gt;b004552&lt;/code&gt;), and what I changed so the pipeline can’t certify a lie again.&lt;/p&gt;

&lt;h2&gt;
  
  
  The claim that could not exist
&lt;/h2&gt;

&lt;p&gt;The site is &lt;a href="https://lineupdraw.com" rel="noopener noreferrer"&gt;lineupdraw.com&lt;/a&gt;, a seeded random team generator. Its blog carries data posts whose numbers are computed at build time by a small simulation module — 10,000 seeded draws of the same 24-name roster, every draw reproducible from its seed. Post 1, “Same-Team Odds in Random Splits, Computed,” answered three questions: how often a fixed pair lands on the same team, how often two independent draws in a row both group them together, and whether a history-aware redraw prevents repeat pairings.&lt;/p&gt;

&lt;p&gt;The third answer was the problem. The published FAQ stated that with the previous draw passed in as history, “the engine re-rolls internally (up to 20 tries) until it finds a split with zero repeated pairings,” and that “measured over 9,999 consecutive draws, the repeat rate collapsed from 4.7% to 0.0%.” The comparison table rendered the same claim as data: &lt;code&gt;0 repeats in 9,999 draws = 0.00%&lt;/code&gt;, next to a column that read &lt;code&gt;engine target: 0&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;A design target and a measurement, printed side by side, reading as one fact.&lt;/p&gt;

&lt;h2&gt;
  
  
  Eight pairs is the floor, and the arithmetic fits in one line
&lt;/h2&gt;

&lt;p&gt;Here is why zero was never on the table. Yesterday’s draw produced 4 teams of 6, which is 60 pairs in total (each 6-person team contributes 15). Today’s redraw scatters each of those 6-person groups across only 4 teams. Six people into four teams means at least one team receives at least two of them — pigeonhole — so each of yesterday’s four teams forces at least one repeated pair. Split 6 as evenly as possible (2, 2, 1, 1) and each yesterday-team contributes 2 repeated pairs, one per pair of members that landed together. Multiply by four teams: &lt;strong&gt;at least 8 of yesterday’s 60 pairs must recur in today’s draw, every single time.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;No redraw policy, however clever, can go below 8. A claim of 0 repeats in 9,999 draws is not an optimistic measurement — it refutes the arithmetic it sits on. One of my favorite failure modes in all of engineering is the impossibility argument, because it costs one sheet of paper and ends every debate. I had skipped the sheet of paper.&lt;/p&gt;

&lt;h2&gt;
  
  
  My test asserted the lie, so CI stayed green
&lt;/h2&gt;

&lt;p&gt;The post survived its own test suite, and that turned out to be part of the disease. The old regression test did not check whether the history-aware claim was plausible. It checked whether the page contained the expected string:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;assert.ok(html.includes('0.00%'))&lt;/code&gt; — with a Chinese comment declaring that the history-aware row must contain 0.00%.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Green CI proved the lie was present, faithfully rendered, byte for byte. It said nothing about truth. I have written before about checks that &lt;a href="https://javaeye.dev/blog/login-checks-that-lie/" rel="noopener noreferrer"&gt;lie in the other direction&lt;/a&gt; — alarms that scream while the underlying state is fine. This is the mirror image and it is scarier: a check that blesses the exact thing it should forbid, because someone (me) typed the wrong expected value once and the test enshrined it.&lt;/p&gt;

&lt;p&gt;A test that matches output against a hardcoded literal is a photograph of a claim, not a validation of it.&lt;/p&gt;

&lt;h2&gt;
  
  
  Three questions had been melted into one counter
&lt;/h2&gt;

&lt;p&gt;The old simulation function computed everything in a single loop: the single-draw rate, the independent back-to-back rate, and the history-aware rate all came out of one counter structure with shared state carried between iterations. Three genuinely different questions — “are they together today,” “are they together twice in a row by chance,” and “does history-help beat chance” — were melted into one measurement stream, and the FAQ narrated the engine’s design target (“re-roll until zero”) as if it were the measured output.&lt;/p&gt;

&lt;p&gt;The decision, written in the three-part form I owe every technical choice: I wanted one fairness number for the history feature. The engine actually has three distinct behaviors with three distinct denominators. So I split one simulation into three — independent single draws, independent consecutive draws, and history-aware consecutive draws — each with its own counter, its own seed namespace, and its own row in the table. No shared state, no ambiguity about what a row measures.&lt;/p&gt;

&lt;h2&gt;
  
  
  The re-measurement, with calibers separated
&lt;/h2&gt;

&lt;p&gt;All numbers below come from the rewritten build-time simulation: 10,000 deterministic seeded draws of the same 24-player roster, 9,999 consecutive-draw transitions, same group-assigning engine the site serves in production. Absolute counts and percentages are both shown; the simulation recomputes identically on every build.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Question&lt;/th&gt;
&lt;th&gt;Analytic answer&lt;/th&gt;
&lt;th&gt;Old post claim&lt;/th&gt;
&lt;th&gt;Re-measured (this fix)&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Fixed pair shares a team in one draw&lt;/td&gt;
&lt;td&gt;5 ÷ 23 = 21.74%&lt;/td&gt;
&lt;td&gt;21.8%&lt;/td&gt;
&lt;td&gt;22.07% (10,000 draws)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Same pair together in two independent draws in a row&lt;/td&gt;
&lt;td&gt;21.74%² = 4.73%&lt;/td&gt;
&lt;td&gt;4.7%&lt;/td&gt;
&lt;td&gt;5.08% (9,999 transitions)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Same pair, second draw is history-aware&lt;/td&gt;
&lt;td&gt;no exact value (constraint problem)&lt;/td&gt;
&lt;td&gt;0.0%&lt;/td&gt;
&lt;td&gt;3.15%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Repeated pairs per draw, all 60 yesterday-pairs&lt;/td&gt;
&lt;td&gt;floor: 8&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;13.1 independent → 8.6 history-aware&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The honest story is better than the fake one. The history-aware engine cannot reach zero — nothing can — but it grinds the average down from 13.1 repeated pairs per draw to 8.6, within touching distance of the 8-pair floor, at a cost of about 8.6 internal re-rolls per draw (it tries up to 20 and keeps the split with the fewest repeats). For the tracked pair specifically, back-to-back collisions fall from 5.08% to 3.15%. Team balance never wavered: across all 10,000 draws the largest gap between any two teams was 0.&lt;/p&gt;

&lt;p&gt;“3.15% instead of 5.08%, near the provable floor” is a real achievement. “0.0%” was a magic trick.&lt;/p&gt;

&lt;h2&gt;
  
  
  The review gate did what CI structurally cannot
&lt;/h2&gt;

&lt;p&gt;The number was caught by an automated review pass — a large-language-model reviewer re-reading both data posts during a site handover check, flagging Post 1’s zero claim as mathematically suspect and confirming it with the pigeonhole argument. Post 2, reviewed in the same pass, needed only wording-level fixes and its numbers stood.&lt;/p&gt;

&lt;p&gt;Notice what happened. The test compared the page against my expectation. The review gate compared the claim against reality — it ran the impossibility check I had skipped, arithmetic that costs nothing and tolerates no negotiation. A second agent with no stake in the original number was willing to say “this cannot be true” where my own tooling was designed to say “this matches what you typed last month.”&lt;/p&gt;

&lt;p&gt;This is the strongest argument I have yet collected for keeping an LLM review gate in front of automated publishing, and I say that as someone whose gate had just indicted him. I have measured plenty of quota and indexing walls on this blog (&lt;a href="https://javaeye.dev/blog/quota-alarm-read-the-page/" rel="noopener noreferrer"&gt;one alarm at a time&lt;/a&gt;); this one is about judgment, not quotas.&lt;/p&gt;

&lt;h2&gt;
  
  
  What changed so it cannot recur
&lt;/h2&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Calibers are separated in code, not only in prose.&lt;/strong&gt; Three simulation functions, three seed namespaces, and a table row per measured question. If a future number looks off, its denominator is one glance away.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The test now forbids the failure instead of photographing it.&lt;/strong&gt; The rewritten regression test asserts &lt;code&gt;histRate &amp;gt; 0&lt;/code&gt; with a comment stating the pigeonhole floor; asserts the history-aware rate is below the independent baseline (the real claim); asserts the page contains the phrase “eight-pair floor”; and asserts the literal string &lt;code&gt;0.00%&lt;/code&gt; appears nowhere in the post.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The copy states the floor before the feature.&lt;/strong&gt; The FAQ now leads with the impossibility — “It prevents as many as mathematics allows — and the ceiling is lower than intuition suggests” — and reports 8.6 against the floor of 8, not against zero.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Design targets stay out of the measurement column.&lt;/strong&gt; The table’s analytic column carries exact values or “no exact value — a constraint problem,” never the engine’s aspiration.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;After the rewrite: 65/65 tests green, drift check 40/40 clean, deployed the same morning. The post now says something true and, I would argue, more useful — a hard floor you can quote beats a soft zero you cannot.&lt;/p&gt;

&lt;p&gt;Next in this series: how the same review pass grades two posts in one batch, and why “only wording fixes” is a verdict worth logging on its own.&lt;/p&gt;




&lt;p&gt;Originally published on my blog: &lt;a href="https://javaeye.dev/blog/the-zero-percent-my-tests-enforced/" rel="noopener noreferrer"&gt;https://javaeye.dev/blog/the-zero-percent-my-tests-enforced/&lt;/a&gt;&lt;/p&gt;

</description>
      <category>debugging</category>
      <category>testing</category>
      <category>data</category>
      <category>automation</category>
    </item>
    <item>
      <title>I Submitted My Site Family to 57 Directories in One Day. The Wall Was 80 Characters.</title>
      <dc:creator>Nan</dc:creator>
      <pubDate>Mon, 16 Nov 2026 12:19:16 +0000</pubDate>
      <link>https://dev.to/eye_java_420f1faa10ae8b86/i-submitted-my-site-family-to-57-directories-in-one-day-the-wall-was-80-characters-31n0</link>
      <guid>https://dev.to/eye_java_420f1faa10ae8b86/i-submitted-my-site-family-to-57-directories-in-one-day-the-wall-was-80-characters-31n0</guid>
      <description>&lt;p&gt;&lt;em&gt;This is a field report from November 2026. Directory sites change their forms, validation rules, and policies without notice, and several of the channels named below will likely be different — or gone — by the time you read this. Treat it as a snapshot of one day's run, not a map of how these sites behave now.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;On Tuesday, November 3, I ran the biggest directory-submission day my small site family has had: &lt;strong&gt;57 verified submissions across four channels&lt;/strong&gt;, touching every site in the family, spread across the day. The headline number is less interesting than what produced it. One brand-new channel went from "never heard of it" to fully submitted in a single day. The day's failures clustered around two boring form behaviors. And five channels on my prospect list got written verdicts — three buried — which saved me more future time than the 57 submissions cost.&lt;/p&gt;

&lt;h2&gt;
  
  
  The day's fifty-seven submissions span four channels, each with its own evidence
&lt;/h2&gt;

&lt;p&gt;I run a family of small static tool sites and submit each one to web directories as one leg of off-site promotion. A "submission" here means one site offered to one channel through that channel's own form, and "verified" means the channel gave me something back I can point to: an HTTP 201 plus a per-site tracking key, a confirmation email, or a listing I can load. For channels already registered in the shared SQLite prospect database, the domain is claimed before any form gets touched (&lt;code&gt;claim --domain &amp;lt;d&amp;gt;&lt;/code&gt;) and recorded after a successful submit (&lt;code&gt;submit --domain &amp;lt;d&amp;gt; --site &amp;lt;s&amp;gt;&lt;/code&gt;); the database then keeps an already-registered directory from being offered again as a fresh prospect, and the claim keeps two concurrent runs from working the same target at once. I cap usage at 10 submissions per site per day.&lt;/p&gt;

&lt;p&gt;The November 3 run looked like this:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Channel&lt;/th&gt;
&lt;th&gt;Verified submissions&lt;/th&gt;
&lt;th&gt;Status after the run&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;listedai.com&lt;/td&gt;
&lt;td&gt;27&lt;/td&gt;
&lt;td&gt;Live, receipts reconciled&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;hyzenpro.com&lt;/td&gt;
&lt;td&gt;27&lt;/td&gt;
&lt;td&gt;Onboarded same day&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;edit.org&lt;/td&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;td&gt;Recovered from a broken form&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;aijumble.com&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;Daily quota nearly exhausted&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;every-ai.com&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;One failed submit, not counted&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;For hyzenpro, the new channel, verification meant an HTTP 201 from the real submit endpoint, a per-site tracking key (&lt;code&gt;HZ-xxx&lt;/code&gt;) in the response, and a terminal screenshot saved per submission — 27 screenshots on disk. For listedai it meant 26 confirmation emails reconciled line by line against the batch record; the 27th submission had no email, so I checked it against its live listing page instead. A submission that returned an ambiguous response did not go in the count.&lt;/p&gt;

&lt;h2&gt;
  
  
  The new channel went from zero to 27 submissions in one day
&lt;/h2&gt;

&lt;p&gt;I had never submitted to hyzenpro before that morning. The playbook was the usual one for a form-based channel: open the listing page, work out the field contract (which inputs are required, what format the description wants, where the submit endpoint actually posts to), then drive the form per site and capture the tracking key from the response.&lt;/p&gt;

&lt;p&gt;Two sites went through while I was still working out the form contract, before the tracked batches started. The morning pass then cleared 17 sites before its time window closed, and failed on 8. The evening fallback batch re-ran those 8 failures — all of them recovered — and by the 22:13 closeout the channel stood at 27/27, every site holding a tracking key.&lt;/p&gt;

&lt;h2&gt;
  
  
  Two boring form behaviors explain the failures the logs actually name
&lt;/h2&gt;

&lt;p&gt;The fallback batch's failure notes point to two failure modes. I can name the affected sites for each mode; whether the remaining failed attempts share them, the log doesn't say.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The first wall was 80 characters.&lt;/strong&gt; The form's description field enforces a &lt;em&gt;minimum&lt;/em&gt; of 80 characters. Two sites — refundplot and wheelroost — sat at 79 and 78 characters and failed validation on length alone. Lengthening the copy by a phrase put both through on the retry.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The second wall was a button that sometimes does nothing.&lt;/strong&gt; On wordroost the submit button accepted the click and the page gave no error, but no listing was created. Success took four rounds of click-and-check. No console error, no validation message, no visual difference between the attempts that did nothing and the one that worked. I never did diagnose the underlying bug, and I didn't need to: the working procedure is to verify the listing exists (or that a tracking key came back) before recording it, because "no error" and "submitted" are different states.&lt;/p&gt;

&lt;p&gt;Two boring form behaviors, named examples for each, and a workaround for both. That ratio is about as good as an ops day gets.&lt;/p&gt;

&lt;h2&gt;
  
  
  Five channels got dated verdicts: three buried, one blocklisted, one parked
&lt;/h2&gt;

&lt;p&gt;The same day's channel audit was worth more than the submissions:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;WeekHack&lt;/strong&gt; — buried. As of November 2026, the pricing page says a free launch earns a permanent dofollow backlink only by finishing in the weekly top three; otherwise the link goes out nofollow. The queue I was shown ran about 21 weeks. Google treats nofollow as a hint rather than a hard rule these days, so this is a prioritization call, not physics — but a five-month wait for a maybe-followed link loses to everything else on my list.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;startupproject.io and gptforthat.com&lt;/strong&gt; — buried after repeated failed attempts to get a submission through at all.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;SitePatent&lt;/strong&gt; — read-only from my side. The only contact I ever got from it was unsolicited "patent protection" mail about my brand name, with no prior relationship. I can't prove what the site is; I can prove I never asked, and the address goes on the blocklist either way.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;somi.ai&lt;/strong&gt; — parked until December 1, worth one revisit then rather than retries every day.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;This is the part of the job nobody puts in the growth-thread screenshots: a channel you correctly stop using stops costing you an attempt every week.&lt;/p&gt;

&lt;h2&gt;
  
  
  Twenty-seven submissions can't enter the dedup database without another role
&lt;/h2&gt;

&lt;p&gt;There is a loose end, and it's structural. Submissions live in a shared &lt;code&gt;backlinks.db&lt;/code&gt; so every role in the family reads the same dedup state. Claiming and recording a submission is open to the link-building role, but &lt;em&gt;adding a new channel to the prospect table&lt;/em&gt; is not — that permission belongs to a different role. So hyzenpro's 27 verified submissions are sitting in per-site log files, unable to enter the dedup database until the other role registers the channel. If that handoff slips, the prospect table will happily offer hyzenpro domains as fresh targets again, and a future run risks double-submitting.&lt;/p&gt;

&lt;p&gt;The lesson I take from this run: in a multi-agent setup, watch the permission seams between internal roles, not just the boundary between you and the outside world. My claim-first workflow protects against double-submitting to a channel &lt;em&gt;inside&lt;/em&gt; the database; it did nothing for a channel that hasn't reached the database yet.&lt;/p&gt;

&lt;h2&gt;
  
  
  Claim first, verify the listing, and write off dead channels
&lt;/h2&gt;

&lt;p&gt;Three things I'd repeat, in order of how much they mattered:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Claim before submit, for any channel the database knows.&lt;/strong&gt; One shared database row per registered domain, claimed before any form gets touched. It turned a multi-channel, whole-family day from a dedup nightmare into an append-only log — for registered channels. hyzenpro, not yet registered, fell outside that fence, which is exactly how the gap above happened.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Verify the listing, not the click.&lt;/strong&gt; The wordroost button taught me that a silent failure looks exactly like success until you check. A tracking key, a receipt email, or a 201 is what counts as evidence.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Write off dead channels with a dated verdict.&lt;/strong&gt; One line per channel ("buried: nofollow unless top 3, ~21-week queue, as of 2026-11") beats re-probing it out of optimism every week.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;57 verified submissions is a small number in the grand scheme of SEO — the same week's indexing work (covered in &lt;a href="https://javaeye.dev/blog/fifty-seven-directory-submissions-one-day//blog/google-request-indexing-quota/" rel="noopener noreferrer"&gt;my GSC quota measurements&lt;/a&gt;) moves more needles per hour. Each of the 57 has evidence attached, though 27 of them are still waiting on a database handoff before they're rows anywhere shared. And the day's other output — two named failure modes, five channels reviewed, one permission gap to close — should keep paying for a while.&lt;/p&gt;




&lt;p&gt;Originally published on my blog: &lt;a href="https://javaeye.dev/blog/fifty-seven-directory-submissions-one-day/" rel="noopener noreferrer"&gt;https://javaeye.dev/blog/fifty-seven-directory-submissions-one-day/&lt;/a&gt;&lt;/p&gt;

</description>
      <category>webdev</category>
      <category>seo</category>
      <category>automation</category>
      <category>marketing</category>
    </item>
    <item>
      <title>Error Code 8 Meant Two Different Failures</title>
      <dc:creator>Nan</dc:creator>
      <pubDate>Sun, 15 Nov 2026 12:04:58 +0000</pubDate>
      <link>https://dev.to/eye_java_420f1faa10ae8b86/error-code-8-meant-two-different-failures-2a68</link>
      <guid>https://dev.to/eye_java_420f1faa10ae8b86/error-code-8-meant-two-different-failures-2a68</guid>
      <description>&lt;p&gt;&lt;em&gt;This is a debugging note from November 2026, on a self-hosted Google Search Console request-indexing pipeline driven over HTTP. Console internals and undocumented error codes change without notice; treat what follows as a failure-mode checklist, not a map of Google's current behavior.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;Google Search Console's internal &lt;code&gt;batchexecute&lt;/code&gt; RPC answers a URL indexing request with a bare integer in a JSON array. For weeks my automation read &lt;code&gt;8&lt;/code&gt; as one thing: &lt;em&gt;daily quota exhausted, stop everything&lt;/em&gt;. On 2026-11-02 I learned it also means &lt;em&gt;this page indexed instantly during the live test — no quota was spent&lt;/em&gt;. Both readings are correct. The stop rule was not.&lt;/p&gt;

&lt;h2&gt;
  
  
  The whole pipeline is three shared accounts, one HTTP client, and a text file
&lt;/h2&gt;

&lt;p&gt;I run a fleet of small static sites (sixteen at the moment) and submit new URLs to Search Console through three shared Google accounts, one window per account (12:00, 13:00, 14:00 local, staggered to avoid tripping Google's own abuse heuristics). The submitter itself is pure HTTP: a &lt;code&gt;uTLS&lt;/code&gt; fingerprinted client carries a forged &lt;code&gt;batchexecute&lt;/code&gt; token plus fresh cookies, no browser involved after the mint step. Each accepted URL is appended to &lt;code&gt;submitted.txt&lt;/code&gt;, which had grown to 376 lines by the morning of the incident.&lt;/p&gt;

&lt;p&gt;The flow had passed final acceptance the day before: all three RPC stages returned 200, a 2,468-byte token blob was minted, and one real submission landed (&lt;code&gt;submitted.txt&lt;/code&gt; 373 → 374 on 2026-11-01 15:34).&lt;/p&gt;

&lt;h2&gt;
  
  
  ssh without -n silently ate four of my five queued URLs
&lt;/h2&gt;

&lt;p&gt;The 13:00 window runs its HTTP loop over ssh on a second machine. First live block claimed five URLs from the queue file. It ran one.&lt;/p&gt;

&lt;p&gt;The loop was &lt;code&gt;ssh host script.sh &amp;lt; urls.txt&lt;/code&gt; — without &lt;code&gt;-n&lt;/code&gt;, ssh quietly consumed the queue file as its own stdin. The script read four more lines, got EOF, and exited successfully. Four URLs were silently dropped: no error, no retry, no log line, and the queue believed they were done. A rehearsal run with a single-URL queue had not caught it, because eating one line of one line is invisible.&lt;/p&gt;

&lt;p&gt;I recovered all four from the raw &lt;code&gt;urls.txt&lt;/code&gt; pull and re-queued them, patched the script in place (every remote call now carries &lt;code&gt;-n&lt;/code&gt;), and pushed the fix to both ends of the tunnel. Embarrassing, but mechanical. The second mine was not.&lt;/p&gt;

&lt;h2&gt;
  
  
  Error code 8 covers both quota exhaustion and instant indexing, and my stop rule believed both
&lt;/h2&gt;

&lt;p&gt;After the fifth successful submission of the day, the next call returned the &lt;code&gt;Fj3Owf&lt;/code&gt; payload with &lt;code&gt;[8]&lt;/code&gt; in the result slot. Then another. Seven of them.&lt;/p&gt;

&lt;p&gt;The old rule fired on the first one: &lt;em&gt;error 8 = quota hit → halt the window&lt;/em&gt;. Which it did — and the 14:00 window, running a different Google account on a different machine, hit the exact same &lt;code&gt;[8]&lt;/code&gt; on the exact same URL (&lt;code&gt;tallypunch/8am-to-5pm/&lt;/code&gt;) and halted too. Two accounts, two machines, same error on the same page. That coincidence is what made me stop trusting the rule.&lt;/p&gt;

&lt;p&gt;I had to decide which failure the code meant, per URL, with only production to test against. The evidence looked like this:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Observation&lt;/th&gt;
&lt;th&gt;If code 8 = quota&lt;/th&gt;
&lt;th&gt;If code 8 = page-level instant index&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Same URL errors on two &lt;em&gt;different&lt;/em&gt; accounts&lt;/td&gt;
&lt;td&gt;Implausible coincidence&lt;/td&gt;
&lt;td&gt;Expected — the page state is per-URL&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Same account: adjacent URLs succeed, this one errors&lt;/td&gt;
&lt;td&gt;Contradiction&lt;/td&gt;
&lt;td&gt;Expected&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Token mint step shows no quota-exhausted wording&lt;/td&gt;
&lt;td&gt;Suggests not quota&lt;/td&gt;
&lt;td&gt;Consistent&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Check-chain fields identical between success and error responses&lt;/td&gt;
&lt;td&gt;Unexplained&lt;/td&gt;
&lt;td&gt;Consistent&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Four for four pointed the same way. The &lt;code&gt;[8]&lt;/code&gt; was a property of the page, not the account: apparently when the live test indexes the URL on the spot, the RPC still reports the quota-style code. There was even a precedent sitting in the runbook — a &lt;code&gt;vinunwrap&lt;/code&gt; URL that had indexed in seconds under the real UI months earlier, and later returned &lt;code&gt;[8]&lt;/code&gt; to the automation.&lt;/p&gt;

&lt;h2&gt;
  
  
  Per-URL strike counts, not a global stop rule, now disambiguate the code
&lt;/h2&gt;

&lt;p&gt;Keeping a stop rule that means two things is worse than no rule, because it trains you to ignore it. The replacement, now baked into the window script and the runbook:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Count &lt;code&gt;[8]&lt;/code&gt; per URL &lt;em&gt;across all channels&lt;/em&gt; — two strikes on the same URL and it moves to an &lt;code&gt;excluded&lt;/code&gt; ledger (&lt;code&gt;eight-counts.txt&lt;/code&gt;), permanently out of the queue. Two accounts agreeing on one page is signal; one account disagreeing with itself is noise.&lt;/li&gt;
&lt;li&gt;Only three &lt;em&gt;consecutive&lt;/em&gt; &lt;code&gt;[8]&lt;/code&gt;s on &lt;em&gt;distinct&lt;/em&gt; URLs halt a window. That is the quota signature.&lt;/li&gt;
&lt;li&gt;A halted window retries at its next scheduled slot, with the exclusion list loaded. No manual re-queueing.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Under the new rules, the day's tally was: 5 URLs submitted (&lt;code&gt;submitted.txt&lt;/code&gt; 376 → 381, ledger reconciled), 2 URLs excluded as page-level mines, zero quota consumed by errors.&lt;/p&gt;

&lt;h2&gt;
  
  
  One full-width comma crashed a failure path that had never executed
&lt;/h2&gt;

&lt;p&gt;While dissecting the acceptance run from the day before, I found the failure path of the submitter script had never once executed in testing — and its first real execution crashed. The notification line contained &lt;code&gt;$REPORT，&lt;/code&gt; — a full-width comma glued to the variable name. In the cron shell's C locale, bash 3.2 parsed the comma's first byte into the variable name and &lt;code&gt;set -u&lt;/code&gt; killed the script with &lt;code&gt;REPORT�: unbound variable&lt;/code&gt;. The verdict file was written; the cleanup never ran. A second, dormant instance (&lt;code&gt;$DONE）&lt;/code&gt;) waited on the same branch. Both are now &lt;code&gt;${REPORT}&lt;/code&gt; and &lt;code&gt;${DONE}&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;The lesson generalizes: any &lt;code&gt;$VAR&lt;/code&gt; immediately followed by a non-ASCII character in a shell script is a live mine under &lt;code&gt;set -u&lt;/code&gt;, and failure paths that have never fired have never been tested. I now grep for the pattern in every script that reports to me, including the one that caught this.&lt;/p&gt;

&lt;h2&gt;
  
  
  Cross-check every undocumented integer against a signal from a different layer
&lt;/h2&gt;

&lt;p&gt;If you are driving Search Console from automation: never let an undocumented integer be your only signal. Cross-check it against account identity, URL-level state, and the token-mint step before letting it stop a pipeline; log the full response body, because the day you need it is the day it is gone; and rehearse with a queue of &lt;em&gt;one&lt;/em&gt; only if you enjoy finding stdin-eating bugs in production. My earlier notes on &lt;a href="https://javaeye.dev/blog/gsc-indexing-quota-8-16-a-day-rolling-24-hours/" rel="noopener noreferrer"&gt;the quota itself&lt;/a&gt; and on &lt;a href="https://javaeye.dev/blog/quota-alarm-read-the-page/" rel="noopener noreferrer"&gt;detectors matching their own targets&lt;/a&gt; both trace back to the same root cause: a cheap signal believed too early.&lt;/p&gt;

&lt;p&gt;The next window runs 2026-11-03 13:00, exclusions loaded. If the 14:00 account submits one of today's &lt;code&gt;[8]&lt;/code&gt; URLs without complaint, that will be the page-level theory's final witness — and I will update this post with the result.&lt;/p&gt;

&lt;p&gt;More post-mortems like this one live on &lt;a href="https://javaeye.dev/blog/" rel="noopener noreferrer"&gt;the blog&lt;/a&gt;; the free tools we run are listed on &lt;a href="https://javaeye.dev/tools/" rel="noopener noreferrer"&gt;/tools&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Originally published on my blog: &lt;a href="https://javaeye.dev/blog/error-code-8-meant-two-failures/" rel="noopener noreferrer"&gt;https://javaeye.dev/blog/error-code-8-meant-two-failures/&lt;/a&gt;&lt;/p&gt;

</description>
      <category>automation</category>
      <category>debugging</category>
      <category>webdev</category>
      <category>seo</category>
    </item>
    <item>
      <title>A Dead Session Looks Exactly Like a Dead Proxy</title>
      <dc:creator>Nan</dc:creator>
      <pubDate>Sat, 14 Nov 2026 12:05:33 +0000</pubDate>
      <link>https://dev.to/eye_java_420f1faa10ae8b86/a-dead-session-looks-exactly-like-a-dead-proxy-54l3</link>
      <guid>https://dev.to/eye_java_420f1faa10ae8b86/a-dead-session-looks-exactly-like-a-dead-proxy-54l3</guid>
      <description>&lt;p&gt;This is a note from October 2026 about driving a remote headless Chrome over the DevTools protocol. The specific failure is YouTube-specific; the diagnostic pattern is not.&lt;/p&gt;

&lt;p&gt;At 20:43 last night my daily YouTube scan died in three places at once: transcript fetches returned HTTP 429, the low-bitrate audio download returned 403, and the recommendation feed was skipped entirely. Two of those are HTTP error statuses; the third is a stage my own guard refused to run. The obvious reading is that the exit node got flagged or throttled.&lt;/p&gt;

&lt;p&gt;It wasn’t the node — or at least, the evidence pointed past the node, at the login session. The two look identical until you split your probes by layer.&lt;/p&gt;

&lt;h2&gt;
  
  
  My pipeline depends on a signed-in browser, which is the risk
&lt;/h2&gt;

&lt;p&gt;Every evening a cron job opens a headless Chrome on a small home server — a Mac Mini on the same LAN — through an SSH tunnel, and runs a fixed scan: pull 12 channels’ RSS feeds for new uploads, read video metadata, fetch captions, download low-bitrate audio, transcribe it. The last three steps go through a browser that has been logged into YouTube for months, because the endpoints my scraper hits rate-limit or refuse anonymous traffic far more aggressively than the public surfaces do.&lt;/p&gt;

&lt;p&gt;That night the metadata stage was perfectly healthy — all 12 feeds returned 200 with fresh entries — while every stage behind the login failed.&lt;/p&gt;

&lt;h2&gt;
  
  
  RSS succeeding through the same node argued against the IP
&lt;/h2&gt;

&lt;p&gt;When several authenticated calls fail in one run, my first instinct is the proxy: shared Clash node, someone else’s traffic got the IP throttled, rotate the node and retry. That’s the standard playbook, and I’ve used it before. But there was one observation sitting in the way — the RSS stage, running minutes earlier through the exact same exit node, returned all 200s.&lt;/p&gt;

&lt;p&gt;That doesn’t exonerate the IP by definition; different surfaces on the same site enforce blocking differently. But it made “the node is dead” a weaker hypothesis than “the thing that differs between those stages is dead” — and the thing that differed was the session, not the route.&lt;/p&gt;

&lt;h2&gt;
  
  
  Three cookies is not a session
&lt;/h2&gt;

&lt;p&gt;So before touching the node, I dumped the browser’s cookies over CDP — &lt;code&gt;Network.getAllCookies&lt;/code&gt; against the browser context, which unlike &lt;code&gt;document.cookie&lt;/code&gt; also sees &lt;code&gt;HttpOnly&lt;/code&gt; ones. What came back was three cookies. Total.&lt;/p&gt;

&lt;p&gt;Three. A live YouTube session carries dozens — &lt;code&gt;SID&lt;/code&gt;, &lt;code&gt;HSID&lt;/code&gt;, &lt;code&gt;SSID&lt;/code&gt;, &lt;code&gt;APISID&lt;/code&gt;, &lt;code&gt;SAPISID&lt;/code&gt;, and a pile of &lt;code&gt;__Secure-&lt;/code&gt; prefixed ones. &lt;code&gt;SAPISID&lt;/code&gt; is the basis of the &lt;code&gt;SAPISIDHASH&lt;/code&gt; value the player API uses to authorize requests. I wouldn’t treat any single cookie as a definitive session test — Google has fallback auth schemes — but single digits where a live session stores dozens is a strong signal, and it matched what I was seeing.&lt;/p&gt;

&lt;p&gt;The browser itself had no idea anything was wrong. The tab would have rendered a normal signed-out homepage, which looks almost identical to a signed-in one. The failure mode was silent at the UI layer and loud only at the API layer — 429, 403, and a skipped stage, three rungs down from where anyone would notice.&lt;/p&gt;

&lt;h2&gt;
  
  
  The three failures each matched the signed-out behavior I know
&lt;/h2&gt;

&lt;p&gt;Once I had the cookie count, the failures sorted themselves as observations, not counterfactuals:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;Captions, 429. The caption endpoints my pipeline hits are the ones that throttled me hardest in past anonymous testing. Signed in, I rarely see 429 there; signed out, it’s the first wall I hit. I can’t claim this is universal — it’s the pattern on the surfaces I use.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Audio, 403. The low-bitrate audio endpoint I fetch refused the request. Whether anonymous requests are always refused I can’t prove; what I can say is that in my pipeline, a request without the session’s credentials got 403 that night, every time.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Recommendation feed, skipped. That stage reads the signed-in homepage. With no session it would have returned generic content, which the pipeline would have wasted quota parsing, so the script’s guard kicked in and skipped it.&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;If I’d rotated the proxy and retried blind, the 403 would most likely have persisted and I’d have burned twenty minutes chasing an IP that the evidence already argued against. I say “most likely” because I didn’t run that experiment — I had a cheaper probe with a clearer answer.&lt;/p&gt;

&lt;h2&gt;
  
  
  The probe order I now use before any retry
&lt;/h2&gt;

&lt;p&gt;The general rule I extracted, and the reason I’m writing this down: when authenticated calls fail and anonymous calls succeed through the same exit node, check the credentials before the network. Not because the IP is proven innocent — because the credential layer is cheaper to inspect and, in my setup, fails silently far more often than the route does.&lt;/p&gt;

&lt;p&gt;The cheapest probe order, most to least informative per second spent:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;Dump cookies over CDP and count them, scoped to the site. Single digits on a site that stores dozens = session death, case closed enough to act on.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;One anonymous request through the same node (RSS, a public API, a status page). If it succeeds, the node routes fine.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Only then consider the node — and even then, rotate with a hypothesis, not as a reflex.&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Step 1 is nearly free and I now run it before any retry. Three cookies versus dozens is a one-glance verdict.&lt;/p&gt;

&lt;h2&gt;
  
  
  Health reported at the wrong layer is the recurring theme
&lt;/h2&gt;

&lt;p&gt;This is the same class of failure I wrote about in &lt;a href="https://javaeye.dev/blog/read-the-profile-before-you-drive-the-browser/" rel="noopener noreferrer"&gt;read the profile before you drive the browser&lt;/a&gt; — automation that reports health at the wrong layer. A tab being alive tells you the process is alive. It tells you nothing about the session, and on this stack the session is the part you actually depend on.&lt;/p&gt;

&lt;h2&gt;
  
  
  Two cheap changes came out of it
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;The scan now dumps cookie count as a preflight. Below threshold, it logs &lt;code&gt;session dead&lt;/code&gt; and stops before burning any quota — no more 429/403 archaeology.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;The node-rotation playbook now has an explicit precondition: don’t rotate while an anonymous probe through the same node still succeeds and the credential layer is uninspected. Credentials first, network second.&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The underlying death was unglamorous: the browser profile’s cookie store had stopped persisting (the file on disk was gone while the process kept running from memory), so a restart would have surfaced it immediately. The recovery is one manual re-login — a human task, logged as such, not something to engineer around.&lt;/p&gt;

&lt;p&gt;One session, two HTTP statuses and a skipped stage, zero useful logs from the thing that died. The probes you layer in front of a flaky external dependency are only as good as their ability to tell “the network is broken” from “your credentials evaporated”. Count the cookies first.&lt;/p&gt;

&lt;p&gt;More post-mortems like this one live on &lt;a href="https://javaeye.dev/blog/" rel="noopener noreferrer"&gt;the blog&lt;/a&gt;; the free tools we run are listed on &lt;a href="https://javaeye.dev/tools/" rel="noopener noreferrer"&gt;/tools&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Originally published on my blog: &lt;a href="https://javaeye.dev/blog/dead-session-looks-like-dead-proxy/" rel="noopener noreferrer"&gt;https://javaeye.dev/blog/dead-session-looks-like-dead-proxy/&lt;/a&gt;&lt;/p&gt;

</description>
      <category>automation</category>
      <category>debugging</category>
      <category>networking</category>
      <category>webdev</category>
    </item>
    <item>
      <title>My Runner Failed 16 Times With Zero Evidence, and Every Health Check Was Green</title>
      <dc:creator>Nan</dc:creator>
      <pubDate>Fri, 13 Nov 2026 13:14:59 +0000</pubDate>
      <link>https://dev.to/eye_java_420f1faa10ae8b86/my-runner-failed-16-times-with-zero-evidence-and-every-health-check-was-green-4hpg</link>
      <guid>https://dev.to/eye_java_420f1faa10ae8b86/my-runner-failed-16-times-with-zero-evidence-and-every-health-check-was-green-4hpg</guid>
      <description>&lt;p&gt;Everything below was measured on my own machines in October 2026: a Mac mini running a shared Perplexity research runner (browser automation over CDP, port 9226) and a scheduling side on a laptop. The Perplexity page fields I cite will rot when their UI changes. The failure model and the evidence rules won't.&lt;/p&gt;

&lt;p&gt;For four days my health checks said the channel was fine, and they were mostly right. The channel was fine. What had been dying was the work flowing through it. On the afternoon of October 28, the shared research runner logged sixteen uncaught-exception failures in two clusters, each with an empty note field and no error text anywhere on disk. The account had quota. The page loaded. The evidence simply did not exist.&lt;/p&gt;

&lt;p&gt;This post is the post-mortem: the three-layer model of what my checks actually covered, the per-day numbers, root causes ranked by how sure I am, and the two code changes that make the next death explainable. It is the sibling of an earlier post about the job that ran on time and did nothing. That one was a single job with no one reading its log. This one is a shared channel where every layer's health check passed while the pipeline was dead.&lt;/p&gt;

&lt;h2&gt;
  
  
  What the three layers actually covered
&lt;/h2&gt;

&lt;p&gt;One shared runner script drives a logged-in browser to run deep research. Three scheduled jobs plus ad-hoc research calls all funnel through it, serialized by a single file lock. Health is policed by a daily verify job that opens the site and checks the page structure. Three layers, each checking itself:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Layer&lt;/th&gt;
&lt;th&gt;What it checks&lt;/th&gt;
&lt;th&gt;What it cannot see&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Page/quota (daily verify, 09:10)&lt;/td&gt;
&lt;td&gt;Does the page render, is the sign-in chip present&lt;/td&gt;
&lt;td&gt;Whether logged-in state has rotted — its pass criterion is literally "not signed in → OK"&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Task (runner + file lock)&lt;/td&gt;
&lt;td&gt;Exits 4, 12, 13, 14, each with a named reason in the log&lt;/td&gt;
&lt;td&gt;Exit 1: an uncaught exception — a code the script never defines; it is Node's default&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Scheduling (jobs + shared ledger)&lt;/td&gt;
&lt;td&gt;Per-day call counts in a shared JSON Lines file&lt;/td&gt;
&lt;td&gt;Which job issued a given call — the ledger had no owner field&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The verify job ran green on October 27 and 28 even though the account's login state was gone on both days: it printed &lt;code&gt;login:false&lt;/code&gt; and still returned OK, because a page that renders unauthenticated is, to that check, a healthy page. All four documented exit codes carry named reasons: CDP unreachable, lock wait or quota, not logged in, output too short. Code 1 carries nothing: the ledger recorded it with an empty note, no screenshot, no stderr capture.&lt;/p&gt;

&lt;p&gt;Two properties that read like small log-hygiene details are the whole bug: failures left no body, and no ledger row said whose job it was.&lt;/p&gt;

&lt;h2&gt;
  
  
  The channel never went down. It stopped leaving bodies.
&lt;/h2&gt;

&lt;p&gt;My ledger, per day:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Day&lt;/th&gt;
&lt;th&gt;Calls&lt;/th&gt;
&lt;th&gt;Successes&lt;/th&gt;
&lt;th&gt;Dominant failure&lt;/th&gt;
&lt;th&gt;What I can prove&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Oct 25&lt;/td&gt;
&lt;td&gt;145&lt;/td&gt;
&lt;td&gt;25&lt;/td&gt;
&lt;td&gt;84x exit 12 (lock wait)&lt;/td&gt;
&lt;td&gt;A lock-storm day; most callers gave up after the wait window&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Oct 26&lt;/td&gt;
&lt;td&gt;28&lt;/td&gt;
&lt;td&gt;11&lt;/td&gt;
&lt;td&gt;16x exit 12&lt;/td&gt;
&lt;td&gt;Load cooled, same pattern&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Oct 27&lt;/td&gt;
&lt;td&gt;37&lt;/td&gt;
&lt;td&gt;22&lt;/td&gt;
&lt;td&gt;8x exit 4&lt;/td&gt;
&lt;td&gt;CDP connection lost; one deep run finished at 11:41 only via a manual re-run&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Oct 28&lt;/td&gt;
&lt;td&gt;26&lt;/td&gt;
&lt;td&gt;3&lt;/td&gt;
&lt;td&gt;16x exit 1, clustered&lt;/td&gt;
&lt;td&gt;5 failures at 15:07, 7 at 18:06–18:07; every note empty&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Oct 29&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;—&lt;/td&gt;
&lt;td&gt;By 00:17, two of the three jobs had been disabled, no record of who or why&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The last successful deep run on October 28 was 13:38. After that the channel went quiet: one call on the 29th, succeeded at 02:24, then nothing. Two checks that would have caught the rot early cost almost nothing. Tailing the ledger and counting empty-note exit-1 rows per day — five minutes, no new infrastructure.&lt;/p&gt;

&lt;h2&gt;
  
  
  Every root cause I could prove was about missing evidence
&lt;/h2&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Confirmed — failures left no evidence.&lt;/strong&gt; The exit-1 rows had empty note fields, no stderr on disk, no screenshot. This is the exact mechanism behind the complaint that started the investigation: "the jobs stopped working, and I can't say why."&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Confirmed — the shared ledger has no owner.&lt;/strong&gt; 145 calls on the 25th, 26 on the 28th: I cannot say which job issued any of them, so I cannot even say whose backlog died.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Confirmed — the verify job's blind spot is structural.&lt;/strong&gt; It failed on October 29 only because my laptop's network dropped for six minutes around 10:02 and its 3600-second probe timed out. It sailed through two days of lost login state. It alarmed when it shouldn't and stayed silent when it should have. Both directions, same root cause.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Inferred, not confirmed — the retirement mechanism.&lt;/strong&gt; A done marker for one job had existed since the 27th at 11:41, yet its retry job kept spinning on 30-minute cycles. Two jobs were disabled at 00:17 with no reason recorded. I cannot separate "the gate failed" from "a different consumer was issuing those calls" — which is exactly the consequence of root cause 2. So I stopped calling it a root cause.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The ranking is the deliverable. Three confirmed, one explicitly marked inference. A claim I cannot ground in something on disk is a hypothesis, and the incident write-up writes it down as one.&lt;/p&gt;

&lt;h2&gt;
  
  
  The fix targets the next death, not this one
&lt;/h2&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Catch the uncaught path.&lt;/strong&gt; The runner now installs process-level handlers for uncaught exceptions and unhandled rejections: on a crash, a one-line summary of the error lands in the ledger's note field instead of the field staying empty. The next exit-1 row reads &lt;code&gt;uncaught: &amp;lt;message&amp;gt;&lt;/code&gt;. Someone will be able to say what died.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Put an owner on every call.&lt;/strong&gt; The ledger row now records which job issued the call. The next time a channel dies, the backlog that died has a name.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;And one decision that is not code: I am not re-enabling any runner. Nothing is queued. Restarting a consumer to keep a channel warm is how the 145-call lock-storm day happened. The channel earns its keep on demand.&lt;/p&gt;

&lt;p&gt;A dashboard that cannot distinguish "no work pending" from "the worker is dead" is a green light on a dead channel. The rule I keep from this incident: when a failure cannot be explained with evidence on disk, the defect is in the evidence path — and fixing the path, not guessing the cause, is what ends the incident.&lt;/p&gt;

&lt;p&gt;More post-mortems like this one live on &lt;a href="https://javaeye.dev/blog/" rel="noopener noreferrer"&gt;the blog&lt;/a&gt;; the free tools we run are listed on &lt;a href="https://javaeye.dev/tools/" rel="noopener noreferrer"&gt;/tools&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Originally published on my blog: &lt;a href="https://javaeye.dev/blog/sixteen-unexplained-failures-zero-evidence/" rel="noopener noreferrer"&gt;https://javaeye.dev/blog/sixteen-unexplained-failures-zero-evidence/&lt;/a&gt;&lt;/p&gt;

</description>
      <category>monitoring</category>
      <category>automation</category>
      <category>debugging</category>
      <category>logging</category>
    </item>
    <item>
      <title>We Ran 13,103 Keyword Probes: 72.6% Empty</title>
      <dc:creator>Nan</dc:creator>
      <pubDate>Thu, 12 Nov 2026 12:25:42 +0000</pubDate>
      <link>https://dev.to/eye_java_420f1faa10ae8b86/we-ran-13103-keyword-probes-726-empty-32k4</link>
      <guid>https://dev.to/eye_java_420f1faa10ae8b86/we-ran-13103-keyword-probes-726-empty-32k4</guid>
      <description>&lt;p&gt;Across 13,103 keyword tool probes run between 2026-10-13 and 2026-10-25, 72.6% returned nothing: zero rows, no volume, no keyword difficulty. The single strongest pattern we found: engineered long-tail phrases built as &lt;code&gt;&amp;lt;category&amp;gt; for &amp;lt;condition&amp;gt;&lt;/code&gt; fail 95-97% of the time, while plain spoken category words almost always return data.&lt;/p&gt;

&lt;h2&gt;
  
  
  What percentage of keyword probes return zero results?
&lt;/h2&gt;

&lt;p&gt;72.6% of the 13,103 probes returned nothing — no volume, no keyword difficulty, zero rows. Each probe was one phrase sent through Semrush Keyword Magic Tool; each empty result is a CSV with a header and no data rows. In total the non-empty probes returned 80,674 matched keywords, which means the empty results were not rare edge cases. They were the majority of the dataset.&lt;/p&gt;

&lt;p&gt;The probes ran in a dual-gate setup. Gate A required volume of at least 100 and keyword difficulty of 20 or less; gate B required volume of at least 500 with no difficulty filter. A phrase counts as empty only when both gates return zero rows, so the 72.6% rate cannot be blamed on one strict filter.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Word shape&lt;/th&gt;
&lt;th&gt;Probes&lt;/th&gt;
&lt;th&gt;Empty rate&lt;/th&gt;
&lt;th&gt;Note&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Engineered long-tail — &lt;code&gt;&amp;lt;category&amp;gt; for &amp;lt;condition&amp;gt;&lt;/code&gt; / &lt;code&gt;by &amp;lt;parameter&amp;gt;&lt;/code&gt;
&lt;/td&gt;
&lt;td&gt;~700&lt;/td&gt;
&lt;td&gt;95-97%&lt;/td&gt;
&lt;td&gt;Consistent across three consecutive batches; the hardest finding in this dataset&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Spoken category words — &lt;code&gt;&amp;lt;category&amp;gt; calculator&lt;/code&gt; / &lt;code&gt;cost&lt;/code&gt; / &lt;code&gt;size&lt;/code&gt; / &lt;code&gt;how many … do i need&lt;/code&gt;
&lt;/td&gt;
&lt;td&gt;~1,900&lt;/td&gt;
&lt;td&gt;22-71%&lt;/td&gt;
&lt;td&gt;Same tool, same database; the word shape changes everything&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Control batch, all filters removed (any entry counts)&lt;/td&gt;
&lt;td&gt;140&lt;/td&gt;
&lt;td&gt;91%&lt;/td&gt;
&lt;td&gt;Only 9% of phrases have any entries at all&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Total&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;13,103&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;72.6%&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;80,674 matched keywords returned&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;A rate this high is not in any tool's public documentation, and we could not find another public probe-level dataset — most teams run a handful of phrases and stop. Our 49 batches over 13 days exist because we fund the probes as a cost of picking directions, not as research.&lt;/p&gt;

&lt;h2&gt;
  
  
  Which word shapes actually have volume?
&lt;/h2&gt;

&lt;p&gt;Split the 13,103 probes by word shape and the 72.6% average hides two opposite worlds. Plain spoken category words — &lt;code&gt;&amp;lt;category&amp;gt; calculator&lt;/code&gt;, &lt;code&gt;&amp;lt;category&amp;gt; cost&lt;/code&gt;, &lt;code&gt;&amp;lt;category&amp;gt; size&lt;/code&gt;, &lt;code&gt;how many &amp;lt;category&amp;gt; do i need&lt;/code&gt; — returned data 29-78% of the time depending on the category. Engineered long-tail phrases built as &lt;code&gt;&amp;lt;category&amp;gt; for &amp;lt;condition&amp;gt;&lt;/code&gt; or &lt;code&gt;by &amp;lt;parameter&amp;gt;&lt;/code&gt; returned data only 3-5% of the time.&lt;/p&gt;

&lt;p&gt;Same tool, same database, same two weeks of October 2026. Roughly 1,900 of the probes were spoken category words; roughly 700 were engineered long-tail. Both groups went through identical gates, so the gap is not a filtering artifact — it is in the phrases themselves.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why do "for / by / with" conditional phrases fail?
&lt;/h2&gt;

&lt;p&gt;Because almost nobody types them. These phrases live in content briefs and AI-generated topic lists, not in search boxes. Three consecutive batches in mid-October 2026 produced empty rates of 95-97% for this shape — the most consistent result in the whole dataset. When a seed pattern fails at that level across independent batches, the pattern is the problem, not the tool.&lt;/p&gt;

&lt;h2&gt;
  
  
  Does a low-KD filter cause the empty results?
&lt;/h2&gt;

&lt;p&gt;No, and we ran the control to prove it. On 2026-10-22, in batch 41, we re-ran 140 phrases with every filter removed — no minimum volume, no difficulty ceiling, unfiltered phrase match. 91% of them still returned zero entries; only 9% had any keyword data at all. A strict gate hides weak keywords, but it cannot hide keywords that do not exist.&lt;/p&gt;

&lt;p&gt;This control rules out the cheapest explanation for the 72.6% headline. If the empty rate were caused by our volume and difficulty gates, the unfiltered control would have filled most of those rows. Instead, on 2026-10-22, 128 of 140 phrases came back empty with every gate removed.&lt;/p&gt;

&lt;h2&gt;
  
  
  How should you write keyword seeds, then?
&lt;/h2&gt;

&lt;p&gt;Start from the category word a person would say out loud, then attach one modifier with task intent: calculator, cost, size, or how many. We covered the category-versus-long-tail gap from the selection side in &lt;a href="https://javaeye.dev/blog/keyword-matrix-gap-analysis/" rel="noopener noreferrer"&gt;our keyword matrix analysis&lt;/a&gt;. And if you build pages from seeds like these, the next bottleneck is getting them indexed — we &lt;a href="https://javaeye.dev/blog/gsc-indexing-quota-measured/" rel="noopener noreferrer"&gt;measured Search Console's indexing quota&lt;/a&gt; the same way, one logged attempt at a time.&lt;/p&gt;

&lt;h2&gt;
  
  
  How was this measured?
&lt;/h2&gt;

&lt;p&gt;Every number on this page comes from the same archive, and the archive is kept raw — no sampled screenshots, no reconstructed logs. If a claim in the table above cannot be traced to a dated entry in that archive, it does not appear on this page. The pipeline, exactly as run:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Tool: Semrush Keyword Magic Tool, Phrase match mode, US database, USD.&lt;/li&gt;
&lt;li&gt;Gate A: volume at least 100 and keyword difficulty at most 20. Gate B: volume at least 500, no difficulty limit. Gate C: no filters at all — control batch only.&lt;/li&gt;
&lt;li&gt;Empty result: the phrase returns zero data rows, which in the archive is a CSV containing only the header line.&lt;/li&gt;
&lt;li&gt;Sample: 13,103 probes in 49 batches, run between 2026-10-13 and 2026-10-25.&lt;/li&gt;
&lt;li&gt;Counting: one probe is one phrase checked in both gates; a probe is empty only if both gates return zero rows.&lt;/li&gt;
&lt;li&gt;Verification: after this page was drafted on 2026-10-26, we re-scanned the raw archive independently — every probe CSV across all 56 round directories. That re-scan found 13,988 probe files with the same 72.6% empty rate, and the 2026-10-22 control batch re-counted at 140 probes, 91.4% empty. The small difference in totals is re-probe deduplication in the published aggregate; the rate is identical either way.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  What does this data not tell you?
&lt;/h2&gt;

&lt;p&gt;Three honest limits, stated because a data page should know its own edges. First, this is one tool, one regional database, and one match mode — another tool or another region may return different numbers. Second, an empty result means the tool has no data for the phrase; it does not prove nobody searches for it, and the 2026-10-22 control exists precisely because empty and nonexistent are different claims.&lt;/p&gt;

&lt;p&gt;Third, our probe set was chosen to pick site directions, so it over-represents the word shapes we test most. Treat the category-level rates as the robust finding, not any single batch.&lt;/p&gt;

&lt;h2&gt;
  
  
  Frequently asked questions
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;What counts as an empty keyword probe?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;An empty probe is a phrase that returns zero data rows after server-side filtering — in our archive, a CSV file that contains only the column header. Each probe is one phrase checked under two gates: volume at least 100 with keyword difficulty at most 20, and volume at least 500 with no difficulty limit. If both return zero rows, the probe counts as empty. The 72.6% rate is counted this way across all 13,103 probes; the unfiltered control batch is reported separately from that headline number.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Is the 72.6% empty rate specific to one keyword tool?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Yes. This page measures Semrush Keyword Magic Tool with phrase match on the US database, and the number should not be transplanted to other tools. What transfers is the method: run your own seeds through your own tool, write down the empty ones, and aggregate by word shape. We would expect any volume database to show a similar failure pattern for engineered &lt;code&gt;&amp;lt;category&amp;gt; for &amp;lt;condition&amp;gt;&lt;/code&gt; phrases — not because every tool agrees, but because no tool can have data for phrases nobody searches.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Does an empty result mean nobody searches for the phrase?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;No. An empty result means the tool's database has no entries for that phrase under the filters used. On 2026-10-22 we ran the control: 140 phrases with every filter removed, and 91% still came back empty. That tells us the phrases lack data in this database — not that human demand is zero. Search demand exists outside any single keyword database, especially for new or spoken-language topics that seed tools have not caught up with.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Which keyword seeds are most likely to return volume data?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Plain spoken category words with a task modifier: &lt;code&gt;&amp;lt;category&amp;gt; calculator&lt;/code&gt;, &lt;code&gt;&amp;lt;category&amp;gt; cost&lt;/code&gt;, &lt;code&gt;&amp;lt;category&amp;gt; size&lt;/code&gt;, or &lt;code&gt;how many &amp;lt;category&amp;gt; do i need&lt;/code&gt;. In our sample of about 1,900 such probes the empty rate ranged from 22% to 71% depending on the category — high variance, but every category returned data at least some of the time. Engineered conditional phrases of the form &lt;code&gt;&amp;lt;category&amp;gt; for &amp;lt;condition&amp;gt;&lt;/code&gt; returned data only 3-5% of the time across about 700 probes in the same archive.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;How often should seed rules be re-measured?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Whenever the tool's database updates or your niche drifts — in practice, once per selection round rather than on a calendar. Our rates come from 49 batches between 2026-10-13 and 2026-10-25, and we treat them as per-round facts, not permanent constants. A cheap habit that keeps this honest: keep every probe CSV, re-count empties at the end of each round, and retire any seed pattern whose empty rate climbs for three consecutive batches.&lt;/p&gt;

&lt;p&gt;More measurement notes like this one live on &lt;a href="https://javaeye.dev/blog/" rel="noopener noreferrer"&gt;the blog&lt;/a&gt;; the free tools we run are listed on &lt;a href="https://javaeye.dev/tools/" rel="noopener noreferrer"&gt;/tools&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Originally published on my blog: &lt;a href="https://javaeye.dev/blog/keyword-tool-empty-results-measured/" rel="noopener noreferrer"&gt;https://javaeye.dev/blog/keyword-tool-empty-results-measured/&lt;/a&gt;&lt;/p&gt;

</description>
      <category>analytics</category>
      <category>data</category>
      <category>seo</category>
      <category>keywords</category>
    </item>
    <item>
      <title>GSC Indexing Quota: 8-16 a Day, Rolling 24 Hours</title>
      <dc:creator>Nan</dc:creator>
      <pubDate>Wed, 11 Nov 2026 12:07:51 +0000</pubDate>
      <link>https://dev.to/eye_java_420f1faa10ae8b86/gsc-indexing-quota-8-16-a-day-rolling-24-hours-1g4a</link>
      <guid>https://dev.to/eye_java_420f1faa10ae8b86/gsc-indexing-quota-8-16-a-day-rolling-24-hours-1g4a</guid>
      <description>&lt;p&gt;Between 2026-10-07 and 2026-10-25 we logged every Search Console request-indexing submission on one account: 154 successful submissions across 19 days, with the daily cap floating between 8 and 16. The quota is account-wide, shared across every property, and it resets on a rolling 24-hour window — not at midnight.&lt;/p&gt;

&lt;h2&gt;
  
  
  How many URL inspection requests can you submit per day?
&lt;/h2&gt;

&lt;p&gt;Between 8 and 16 per day, and the honest planning number is 11. On days when we measured the cap from a fresh pool it landed on 9, 11, 13, or 16. From 2026-10-20 to 2026-10-25 the cap was exactly 11 on six consecutive days, each ending with the 12th click rejected. The folklore number "about ten" is not wrong, but it hides the range — and the range is what breaks batch plans.&lt;/p&gt;

&lt;p&gt;This blog published a first three-day measurement on 2026-10-12 in &lt;a href="https://javaeye.dev/blog/google-request-indexing-quota/" rel="noopener noreferrer"&gt;an earlier post&lt;/a&gt;; every claim in it still holds after the larger sample below.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Date (2026)&lt;/th&gt;
&lt;th&gt;Successful submissions&lt;/th&gt;
&lt;th&gt;Rejected at&lt;/th&gt;
&lt;th&gt;Notes&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;10-07&lt;/td&gt;
&lt;td&gt;4&lt;/td&gt;
&lt;td&gt;5th&lt;/td&gt;
&lt;td&gt;Pool partly consumed before our batch&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10-08&lt;/td&gt;
&lt;td&gt;13&lt;/td&gt;
&lt;td&gt;14th&lt;/td&gt;
&lt;td&gt;Recovered next morning before Pacific midnight&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10-09&lt;/td&gt;
&lt;td&gt;16&lt;/td&gt;
&lt;td&gt;17th&lt;/td&gt;
&lt;td&gt;Highest sample; spread across four properties; 17:21 rejection&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10-10&lt;/td&gt;
&lt;td&gt;11&lt;/td&gt;
&lt;td&gt;12th&lt;/td&gt;
&lt;td&gt;17:52-18:00&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10-11&lt;/td&gt;
&lt;td&gt;11&lt;/td&gt;
&lt;td&gt;12th&lt;/td&gt;
&lt;td&gt;18:24-18:31; identical shape to 10-10&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10-13&lt;/td&gt;
&lt;td&gt;9&lt;/td&gt;
&lt;td&gt;10th&lt;/td&gt;
&lt;td&gt;16:18-16:25; first clicks showed a low-pool error signature&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10-17&lt;/td&gt;
&lt;td&gt;13&lt;/td&gt;
&lt;td&gt;14th&lt;/td&gt;
&lt;td&gt;19:43 rejection; the 13:42 flag was a detector false positive — see method&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10-18 morning&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;2nd&lt;/td&gt;
&lt;td&gt;13:38 success, 13:39 rejection; rolling window not yet released&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10-18 evening&lt;/td&gt;
&lt;td&gt;10&lt;/td&gt;
&lt;td&gt;—&lt;/td&gt;
&lt;td&gt;After release; stopped at our own batch cap, day total 11&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10-19&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;1st&lt;/td&gt;
&lt;td&gt;13:45 first click rejected; rolling window from 10-18 21:54 not released&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10-20&lt;/td&gt;
&lt;td&gt;11&lt;/td&gt;
&lt;td&gt;12th&lt;/td&gt;
&lt;td&gt;19:18-19:49; rejection at 19:49:47&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10-21&lt;/td&gt;
&lt;td&gt;11&lt;/td&gt;
&lt;td&gt;12th&lt;/td&gt;
&lt;td&gt;19:57-20:05; rejection at 20:06:23&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10-22&lt;/td&gt;
&lt;td&gt;11&lt;/td&gt;
&lt;td&gt;12th&lt;/td&gt;
&lt;td&gt;20:18-20:36&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10-23&lt;/td&gt;
&lt;td&gt;11&lt;/td&gt;
&lt;td&gt;12th&lt;/td&gt;
&lt;td&gt;Rejection at 20:38:42&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10-24&lt;/td&gt;
&lt;td&gt;11&lt;/td&gt;
&lt;td&gt;12th&lt;/td&gt;
&lt;td&gt;20:39-20:48; rejection at 20:49:05&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10-25&lt;/td&gt;
&lt;td&gt;11&lt;/td&gt;
&lt;td&gt;12th&lt;/td&gt;
&lt;td&gt;20:50-20:59; rejection at 21:00:12&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h2&gt;
  
  
  Is the quota per account or per property?
&lt;/h2&gt;

&lt;p&gt;Per account. Every property under one Google account draws from the same daily pool. On 2026-10-09 we logged sixteen successful submissions spread across four different properties — one, six, eight, and one — and the seventeenth request at 17:21 was rejected regardless of which property it targeted.&lt;/p&gt;

&lt;p&gt;Switching sites changes nothing because switching sites is not switching accounts. In our tests, changing IP address or using an incognito window made no difference either — the only action that draws from a different pool is logging into a different Google account.&lt;/p&gt;

&lt;h2&gt;
  
  
  When does the quota reset?
&lt;/h2&gt;

&lt;p&gt;On a rolling 24-hour window, one submission at a time — not at midnight, and not at any fixed hour. On 2026-10-16 our last submission went through at 19:11:38. The next afternoon, on 2026-10-17, a 13:42 request was still rejected, while submissions at 19:30 and 19:31 succeeded.&lt;/p&gt;

&lt;p&gt;The popular "wait for Pacific midnight" advice fails in our data. So does the fixed-hour Beijing-time claim we tested on 2026-10-08: a 14:31 request was rejected, a 15:00:54 re-test was still rejected, and the pool only came back at 17:05. A rolling window has no reset hour; each successful submission frees its slot exactly 24 hours later.&lt;/p&gt;

&lt;h2&gt;
  
  
  What does hitting the limit look like?
&lt;/h2&gt;

&lt;p&gt;A specific banner, and two operational details that matter. The rejection appears after the click, not before, and its wording is unambiguous — the page says the daily quota has been exceeded and asks you to try again tomorrow. A rejected click consumes nothing: on 2026-10-19 our first click was rejected at 13:45 and the account's total consumption for that day stayed at zero.&lt;/p&gt;

&lt;p&gt;The signature is consistent enough to plan around. When a batch ends with a rejection, stop the batch — the remaining URLs go to the next rolling window, and probing later the same day costs nothing but time. We screenshot every rejection; the table above is built from those screenshots, not from memory.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why do some days allow 11 and others 16?
&lt;/h2&gt;

&lt;p&gt;We do not know, and we will not pretend to. Sixteen went through on 2026-10-09; from 2026-10-20 through 2026-10-25 the cap was 11 every single day. Request pacing, account history, and property mix are hypotheses we have not controlled for, and our sample is one account over three weeks.&lt;/p&gt;

&lt;p&gt;What we can say safely: the cap is stable within a stretch of days but floats across weeks. Any submission schedule built on an exact number — including "11", our own planning value — will eventually break, which is why the batch rule is "stop at the cap, roll the rest," not "submit N per day."&lt;/p&gt;

&lt;h2&gt;
  
  
  How was this measured?
&lt;/h2&gt;

&lt;p&gt;Every number in the table above is a logged click in the URL Inspection interface, recorded on the day it happened, with a screenshot for every rejection. Nothing in this page is reconstructed from memory or copied from someone else's account. The protocol, exactly as run:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Channel: manual and browser-automation clicks in the Search Console URL Inspection UI, one URL at a time. No API was used.&lt;/li&gt;
&lt;li&gt;Cap signal: only the exact daily-quota wording counts as a rejection. Nothing else — no transient errors, no progress dialogs.&lt;/li&gt;
&lt;li&gt;Counting: one click is one submission attempt. Rejected clicks are not counted as successes and, per the 2026-10-19 observation, consume no quota.&lt;/li&gt;
&lt;li&gt;Sample: one Google account, 154 successful submissions and 15 rejections between 2026-10-07 and 2026-10-25. This blog and several small tool properties sit under the same account.&lt;/li&gt;
&lt;li&gt;Detector honesty: on 2026-10-17 our automation flagged a quota rejection that was not one — the inspected URL's slug happened to contain the word "quota", and the screenshot showed a progress dialog, not the quota banner. We tightened the matcher to the exact platform wording. We publish this because a quota dataset with silent false positives is worse than a small one.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  What does this data not prove?
&lt;/h2&gt;

&lt;p&gt;Everything above is one account, three weeks, manual submissions. Google does not document this quota, does not promise it, and can change it without notice — these numbers describe October 2026, not a policy. If you schedule around them, re-measure on your own account first.&lt;/p&gt;

&lt;p&gt;And if your bottleneck is producing pages worth submitting, the waste usually starts earlier: &lt;a href="https://javaeye.dev/blog/keyword-tool-empty-results-measured/" rel="noopener noreferrer"&gt;72.6% of keyword probes return nothing&lt;/a&gt;, so seed choice decides how many URLs you even need in the queue. More measurement notes live on &lt;a href="https://javaeye.dev/blog/" rel="noopener noreferrer"&gt;the blog&lt;/a&gt;; the tools we run are on &lt;a href="https://javaeye.dev/tools/" rel="noopener noreferrer"&gt;/tools&lt;/a&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Frequently asked questions
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;How many URLs can you submit to Google Search Console per day?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;In our measurement, between 8 and 16, with 11 as the practical planning number. Across 19 days of logged submissions on one account, from 2026-10-07 to 2026-10-25, the cap landed on 9, 11, 13, or 16 when measured from a fresh pool — and from 2026-10-20 onward it was 11 on six consecutive days, each ending with the 12th request rejected. Treat the quota as a range, plan batches for the low end, and stop the moment the banner appears.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Does the Search Console indexing quota reset at midnight?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;No. Our data shows a rolling 24-hour window in which each submission frees its slot exactly 24 hours later. After our last 2026-10-23 submission at 20:37:54, the first 2026-10-24 click succeeded at 20:39:06. Waiting for Pacific midnight, or for any fixed hour, failed every time we tested it — including the 15:00 Beijing-time claim we falsified on 2026-10-08, when the pool only returned at 17:05.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Is the request indexing quota per site or per account?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Per account, shared across every property the account owns. On 2026-10-09 we logged sixteen successful submissions spread over four different properties under one account, and the seventeenth was rejected no matter which property it targeted. Changing your IP address or using an incognito window made no difference in our tests — the only way to draw from a different pool is a different Google account.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Do rejected indexing requests use up quota?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;No. On 2026-10-19 our first click of the day was rejected at 13:45, and the account's total consumption for that day stayed at zero — later checks confirmed nothing was deducted. A rejection is the quota telling you the pool is empty; it costs you a click, not a slot. That is worth knowing operationally, because you can probe whether a window has opened without spending anything.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Can you get more indexing quota with a second Google account?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Sometimes, but not a full fresh pool. Spare accounts usually own properties of their own, so their daily budget is partly spent before you arrive — in our 2026-10-08 tests, three spare accounts hit the cap on their first or fifth request. If you keep a second account for submissions, check what else it owns and measure its actual cap before counting on it. An account that already carries several properties is not a fresh account.&lt;/p&gt;




&lt;p&gt;Originally published on my blog: &lt;a href="https://javaeye.dev/blog/gsc-indexing-quota-measured/" rel="noopener noreferrer"&gt;https://javaeye.dev/blog/gsc-indexing-quota-measured/&lt;/a&gt;&lt;/p&gt;

</description>
      <category>google</category>
      <category>seo</category>
      <category>indexing</category>
    </item>
    <item>
      <title>The Queue Claimer That Ate Its Own Index Files</title>
      <dc:creator>Nan</dc:creator>
      <pubDate>Mon, 26 Oct 2026 13:15:29 +0000</pubDate>
      <link>https://dev.to/eye_java_420f1faa10ae8b86/the-queue-claimer-that-ate-its-own-index-files-10d8</link>
      <guid>https://dev.to/eye_java_420f1faa10ae8b86/the-queue-claimer-that-ate-its-own-index-files-10d8</guid>
      <description>&lt;p&gt;On 24 October, two scheduled Search Console channels were supposed to move about twenty URLs between them. Both logged one line and stopped. The next day I fixed what I believed was the cause and then found two more defects underneath it: my queue picker globbed every text file in the queue folder, so it claimed a 19-domain site list and a 14-line submitted log as fresh work, and the submitter reported success for ten pages it had only inspected. The backlog that night was 160 URLs.&lt;/p&gt;

&lt;p&gt;None of the three defects threw an error. That is the part worth writing down.&lt;/p&gt;

&lt;h2&gt;
  
  
  All three defects hid inside one claim-and-submit path
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;160 URLs&lt;/strong&gt; sat queued across nine per-site files, from 89 on the largest site to 1 on the smallest.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;19 bare domains&lt;/strong&gt; and &lt;strong&gt;14 already-submitted URLs&lt;/strong&gt; lived in the same directory as the work, one &lt;code&gt;glob()&lt;/code&gt; call away from being picked up.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;2 nights&lt;/strong&gt; produced zero submitted URLs; the second night's skip was correct behaviour and the log could not tell me that.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;10 lines claimed, 0 submitted&lt;/strong&gt; on the first repaired run: every URL came back &lt;code&gt;indexed&lt;/code&gt;, meaning the script inspected it, saw it was already in Google, and never clicked anything.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  The lock error was real, and it was the least interesting part
&lt;/h2&gt;

&lt;p&gt;The two channels are shell scripts. One runs on a Windows box at 20:50, one on this Mac at 22:00, and each claims URLs from a shared folder by deleting the lines it takes. To keep a manual run from colliding with a scheduled one, both take a lock at the top of the script. The original code was the usual idiom:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nb"&gt;exec &lt;/span&gt;9&amp;gt;/tmp/gsc.lock
&lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="o"&gt;!&lt;/span&gt; flock &lt;span class="nt"&gt;-n&lt;/span&gt; 9&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="k"&gt;then
  &lt;/span&gt;log &lt;span class="s2"&gt;"another instance is running → skip"&lt;/span&gt;
  &lt;span class="nb"&gt;exit &lt;/span&gt;0
&lt;span class="k"&gt;fi&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The problem is that &lt;code&gt;flock&lt;/code&gt; is not present on macOS. &lt;code&gt;which flock&lt;/code&gt; returns nothing here — I checked again while writing this. When the command does not exist, the shell returns 127, &lt;code&gt;!&lt;/code&gt; inverts that to true, and the guard fires on every single run. The script exits 0 with a message that reads like a reasonable decision. The scheduler sees a successful run. Nothing in the pipeline is in a position to notice, because "I skipped because someone else is working" is a legitimate state.&lt;/p&gt;

&lt;p&gt;A guard that cannot fail is worse than no guard, because it looks like protection and it converts every run into a no-op. I replaced the file-descriptor idiom with an atomic &lt;code&gt;mkdir&lt;/code&gt; lock: the lock directory holds the owner's pid, a live pid makes the second run give way, a dead pid or a directory with no pid file makes the lock reclaimable after five seconds of waiting, and a trap releases it on exit. That reclaim path is the weak part and I know it: &lt;code&gt;mkdir&lt;/code&gt; and the pid write are two separate operations, so there is a window in which a second process can find a lock directory with no owner yet, wait five seconds, and take it away. I chose that over the alternative of a lock that never blocks anything, and the window is five seconds wide rather than the entire night.&lt;/p&gt;

&lt;p&gt;I tested four cases in isolation before trusting the replacement — first acquisition, concurrent refusal, stale lock with a dead pid, lock directory with no pid file — and only then pointed the scripts at it.&lt;/p&gt;

&lt;h2&gt;
  
  
  Night two was contention, and the log named the wrong machine
&lt;/h2&gt;

&lt;p&gt;On 24 October that channel's log holds a single dry-run at 21:19 and then a skip message at 21:40, an hour after its scheduled slot; I have not established whether the 20:50 slot fired and logged nothing or never fired at all. The Mac channel logged the same sentence at 22:00, with a problem: the Mac script's message said the &lt;em&gt;other machine&lt;/em&gt; was holding the lock. The two scripts had been forked from one another and the string had never been updated, so a skip on this Mac pointed at a computer that was idle at the time. That string was corrected later that night.&lt;/p&gt;

&lt;p&gt;The reason for the Mac skip was real, and it was me. I had a manual round running in an interactive terminal, holding this machine's lock, submitting URLs one at a time. A scheduled window deferring to an interactive session is exactly what a lock is for. Deferring was correct.&lt;/p&gt;

&lt;p&gt;What was not correct was the reporting. A skip, a failure, and an empty queue all reached my end-of-day check as "zero submissions". Three states, one number. The check now requires the claim count to appear in the log (&lt;code&gt;claimed=N&lt;/code&gt;), and treats a missing claim line as a different condition from a claim of zero.&lt;/p&gt;

&lt;h2&gt;
  
  
  The picker globbed every text file in the folder
&lt;/h2&gt;

&lt;p&gt;This is the defect I did not go looking for. While verifying that the lock fix had landed, I read the claim step properly for the first time. Here is what it did:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;f&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="nf"&gt;sorted&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;pathlib&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Path&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;qdir&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;glob&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;*.txt&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)):&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;claimed&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;continue&lt;/span&gt;
    &lt;span class="bp"&gt;...&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The directory holds one file per site: &lt;code&gt;gradeplot.com.txt&lt;/code&gt; with 89 URLs, &lt;code&gt;expectcalc.com.txt&lt;/code&gt;, &lt;code&gt;payofftable.com.txt&lt;/code&gt; and six more. It also holds two files that are not work at all. &lt;code&gt;sites.txt&lt;/code&gt; is the list of my 19 domains, one bare hostname per line, no scheme. &lt;code&gt;submitted.txt&lt;/code&gt; is the append-only audit trail of everything already submitted — 14 lines at that moment, 24 by the end of the next day.&lt;/p&gt;

&lt;p&gt;The filter skipped any file with &lt;code&gt;claimed&lt;/code&gt; in its name. It did not skip those two, because they do not contain that substring. So on every claim round, the picker was entitled to take bare hostnames as URLs and to re-take URLs that had already been submitted and logged. The separator between work and record was a filename convention that the code half-implemented.&lt;/p&gt;

&lt;p&gt;I reproduced it in a sandbox copy rather than reasoning about it, with the queue directory pointed at a scratch folder holding both kinds of file. Before the fix, the claimed list contained &lt;code&gt;bracketree.com&lt;/code&gt;, &lt;code&gt;expectcalc.com&lt;/code&gt; and &lt;code&gt;https://x.com/already-submitted/&lt;/code&gt;. After the fix, it contained only real queue URLs.&lt;/p&gt;

&lt;p&gt;The fix is two rules, and only one of them is a whitelist:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Skip any file named &lt;code&gt;sites.txt&lt;/code&gt;, &lt;code&gt;submitted.txt&lt;/code&gt; or &lt;code&gt;excluded.txt&lt;/code&gt; — a name blacklist, the weaker half.&lt;/li&gt;
&lt;li&gt;Keep only lines that begin with &lt;code&gt;http&lt;/code&gt;, so a bare hostname can never enter the loop regardless of which file it came from — a scheme whitelist.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;I also renamed one of the two site lists in the same pass, and immediately discovered why name-based exclusion is fragile: a third file, &lt;code&gt;excluded.txt&lt;/code&gt;, had to be added to the skip list the same afternoon. The list of names I remember is not the list of names that will exist next month. The structural fix is one directory for work and another for records, or a suffix convention like &lt;code&gt;*.urls.txt&lt;/code&gt; that the picker accepts; that is on my list instead of another name.&lt;/p&gt;

&lt;h2&gt;
  
  
  Exit code zero covered two different outcomes
&lt;/h2&gt;

&lt;p&gt;The submit step runs a Node script against a DevTools-attached browser. Zero means the run finished, not that a submission happened. The script first runs Search Console's URL Inspection and reads the verdict; when the verdict is "URL is on Google", it stops there and never clicks Request Indexing, because resubmitting an already-indexed URL spends indexing quota for nothing. The inspection itself draws on a separate daily inspection allowance, which is why the wrapper still runs it only once per URL. It reports that short circuit as status &lt;code&gt;indexed&lt;/code&gt;. That logic is correct. The shell wrapper around it logged &lt;code&gt;OK&lt;/code&gt;, counted it, and moved on.&lt;/p&gt;

&lt;p&gt;The first repaired round made the distinction visible. Ten URLs were claimed from the expectcalc file, the round finished in three minutes nine seconds, and every one of the ten came back &lt;code&gt;indexed&lt;/code&gt;, with the tool's detail line saying the URL was already in Google. Ten &lt;code&gt;OK&lt;/code&gt; lines, zero quota spent, zero clicks on the button. Those ten pages had been queued by an automatic lastmod-based top-up and were already inside the index, so the queue had been carrying ten tasks that could not produce anything.&lt;/p&gt;

&lt;p&gt;Left alone, that failure mode repeats indefinitely: the URLs are appended to &lt;code&gt;submitted.txt&lt;/code&gt; when they exit successfully, the submitted list was not being read by anything except me, and the queue file gets refilled by the same automation the next day. Every line that is both in a queue file and in &lt;code&gt;submitted.txt&lt;/code&gt; is a line that will be retried tomorrow with no possible effect.&lt;/p&gt;

&lt;p&gt;The wrapper now reads the status field out of the submitter's output and counts &lt;code&gt;indexed&lt;/code&gt;, &lt;code&gt;dry-ready&lt;/code&gt; and &lt;code&gt;submitted&lt;/code&gt; separately, then reports all three. The line "10 claimed, 0 submitted, 10 already indexed" describes that night; "ok=10" did not.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Guard&lt;/th&gt;
&lt;th&gt;What it looked like it guaranteed&lt;/th&gt;
&lt;th&gt;How it actually failed&lt;/th&gt;
&lt;th&gt;What I check now&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Lock&lt;/td&gt;
&lt;td&gt;Only one submitter runs at a time&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;flock&lt;/code&gt; absent → &lt;code&gt;if !&lt;/code&gt; always true → every run exits 0&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;mkdir&lt;/code&gt; lock with pid; four isolated tests&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Skip message&lt;/td&gt;
&lt;td&gt;Which channel held the lock&lt;/td&gt;
&lt;td&gt;Message named the other machine&lt;/td&gt;
&lt;td&gt;Channel name printed from one variable&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Queue claim&lt;/td&gt;
&lt;td&gt;Only URLs are claimed&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;glob("*.txt")&lt;/code&gt; + name blacklist → domains and audit rows claimed&lt;/td&gt;
&lt;td&gt;Name blacklist, &lt;code&gt;http&lt;/code&gt; whitelist, split directories&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Exit code&lt;/td&gt;
&lt;td&gt;A submission happened&lt;/td&gt;
&lt;td&gt;0 also for inspected-but-skipped pages&lt;/td&gt;
&lt;td&gt;Status field parsed and counted per outcome&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h2&gt;
  
  
  The same guard now exists in two copies
&lt;/h2&gt;

&lt;p&gt;One more thing I found while verifying. The window script for the Mac channel exists twice on this machine — once under &lt;code&gt;~/.hermes/scripts/&lt;/code&gt; and once under the scheduler profile's &lt;code&gt;scripts/&lt;/code&gt; directory. At 19:48 today, after a separate incident where a browser port turned out to be logged into the wrong Google account, I added a ten-line login check to that script.&lt;/p&gt;

&lt;p&gt;The two files no longer have the same md5. &lt;code&gt;diff&lt;/code&gt; shows the login check in one and not in the other. Whichever copy the scheduler resolves, the other one is now a file that looks like the production script and behaves like last week's. Duplicated guards drift, and the drift is invisible because both copies still run.&lt;/p&gt;

&lt;p&gt;I have not consolidated them yet.&lt;/p&gt;

&lt;h2&gt;
  
  
  Five changes replaced name filters with structural separation
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Split records from work in the filesystem, not in a filter.&lt;/strong&gt; The picker reads every &lt;code&gt;.txt&lt;/code&gt; in that directory, so any new file placed there becomes work.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Make the claim step announce its own numbers.&lt;/strong&gt; &lt;code&gt;claimed=10&lt;/code&gt; at the top of a run is a cheap assertion; the absence of that line is now itself a failure signal.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Treat skip, failure and empty as three states.&lt;/strong&gt; My daily check reads the claim count before it reads anything else.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Test a guard by trying to make it fail.&lt;/strong&gt; The &lt;code&gt;mkdir&lt;/code&gt; lock got four negative tests before it got a single real run. The &lt;code&gt;flock&lt;/code&gt; version got none, and it never worked once.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Do not trust &lt;code&gt;exit 0&lt;/code&gt; from a wrapper around a UI.&lt;/strong&gt; Read the status the interface actually reported, and count outcomes, not attempts.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The queue holds 160 URLs tonight, and the two channels can move about 22 a day when nothing is queued ahead of them, which puts the backlog at roughly a week of windows. The open question is whether the remaining lines can produce anything at all: ten of last night's were already in the index before they were queued, and the only way I have to find the rest is to diff the queue files against &lt;code&gt;submitted.txt&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;A note on scope: this describes two shell scripts and one Node script driving the Search Console web interface in October 2026. Interface labels and quota behaviour change without notice; treat every number here as a snapshot of my own account.&lt;/em&gt;&lt;/p&gt;




&lt;p&gt;Originally published on my blog: &lt;a href="https://javaeye.dev/blog/the-queue-claimer-that-ate-its-own-index-files/" rel="noopener noreferrer"&gt;https://javaeye.dev/blog/the-queue-claimer-that-ate-its-own-index-files/&lt;/a&gt;&lt;/p&gt;

</description>
      <category>debugging</category>
      <category>programming</category>
      <category>softwareengineering</category>
    </item>
    <item>
      <title>My Unattended Job Ran on Time and Did Nothing</title>
      <dc:creator>Nan</dc:creator>
      <pubDate>Fri, 23 Oct 2026 07:54:46 +0000</pubDate>
      <link>https://dev.to/eye_java_420f1faa10ae8b86/my-unattended-job-ran-on-time-and-did-nothingmy-unattended-job-ran-on-time-and-did-nothing-aef</link>
      <guid>https://dev.to/eye_java_420f1faa10ae8b86/my-unattended-job-ran-on-time-and-did-nothingmy-unattended-job-ran-on-time-and-did-nothing-aef</guid>
      <description>&lt;p&gt;&lt;em&gt;This post describes the behavior of specific tool versions on my machine — macOS with bash 3.2 and the Claude Code CLI as of October 2026. Argument-parsing details change; verify against your own versions before copying any of this.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;For two nights in a row, my nightly automation fired exactly on schedule and exited one second later, having done nothing. The scheduler was never the problem. The prompt I fed to the agent CLI happened to start with a line of three dashes — the YAML frontmatter of my runbook file — and the CLI parsed the entire 12-kilobyte prompt as one long command-line option, rejected it, and quit with exit code 1. Here is the failure, the one-line fix, and the check I now run on every unattended job.&lt;/p&gt;

&lt;h2&gt;
  
  
  The job fired on time and failed in one second
&lt;/h2&gt;

&lt;p&gt;I run a small unattended pipeline on my Mac. Every night at 21:00, a launchd agent executes a shell script. The script checks that a logged-in browser is reachable, then hands a runbook file to an AI coding agent in print mode, which executes the night's tasks: a few community replies, a read-only check of one account, and a blog post for this site.&lt;/p&gt;

&lt;p&gt;On the morning of the third day I opened the logs. Both nights looked like this (the two Chinese lines translated — the script logs in Chinese, everything else verbatim):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;===== nightly-yanghao start 2026-10-16 21:00:05 =====
[Fri Oct 16 21:00:05 CST 2026] CDP preflight OK (localhost:9223)
error: unknown option '---
created: 2026-10-15
type: runbook
...
[Fri Oct 16 21:00:05 CST 2026] claude exit rc=1
===== nightly-yanghao end 2026-10-16 21:00:06 rc=1 =====
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The timeline table, both nights:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Night&lt;/th&gt;
&lt;th&gt;launchd fired&lt;/th&gt;
&lt;th&gt;Preflight&lt;/th&gt;
&lt;th&gt;Agent runtime&lt;/th&gt;
&lt;th&gt;Work done&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Oct 15&lt;/td&gt;
&lt;td&gt;21:00:05&lt;/td&gt;
&lt;td&gt;Passed&lt;/td&gt;
&lt;td&gt;1 second&lt;/td&gt;
&lt;td&gt;Zero&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Oct 16&lt;/td&gt;
&lt;td&gt;21:00:05&lt;/td&gt;
&lt;td&gt;Passed&lt;/td&gt;
&lt;td&gt;1 second&lt;/td&gt;
&lt;td&gt;Zero&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The scheduler did its job perfectly. The script did its preflight correctly. The agent CLI then rejected its instructions as a malformed flag, and the whole run was over before a human heartbeat's worth of time had passed. Two nights, six tasks per night, twelve missed executions, and the system was green on every dashboard I had — because I had no dashboard, only a log file nobody read.&lt;/p&gt;

&lt;p&gt;One thing worth crediting: macOS itself kept receipts I never looked at. launchd records the last exit status of every job it manages — &lt;code&gt;launchctl list&lt;/code&gt; shows it in the Status column, where a &lt;code&gt;1&lt;/code&gt; had been sitting since the first night. And the plist format has native &lt;code&gt;StandardOutPath&lt;/code&gt; and &lt;code&gt;StandardErrorPath&lt;/code&gt; keys that persist a job's output to files without any wrapper script. My wrapper did capture stderr into its own log, so nothing was lost — but between launchd's exit-status table and my log, the failure was recorded twice and read zero times. The observability was there. The observing was not.&lt;/p&gt;

&lt;h2&gt;
  
  
  The prompt was valid Markdown and an invalid command line
&lt;/h2&gt;

&lt;p&gt;My runbook is a Markdown file, and like most of my Markdown files it starts with YAML frontmatter:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="nn"&gt;---&lt;/span&gt;
&lt;span class="na"&gt;created&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;2026-10-15&lt;/span&gt;
&lt;span class="na"&gt;type&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;runbook&lt;/span&gt;
&lt;span class="nn"&gt;---&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;My script passed the file's contents to the CLI as a positional argument, roughly like this:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;claude &lt;span class="nt"&gt;-p&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="si"&gt;$(&lt;/span&gt;&lt;span class="nb"&gt;cat &lt;/span&gt;runbook.md&lt;span class="si"&gt;)&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The CLI's own help text describes print mode plainly: &lt;code&gt;use -p/--print for non-interactive output&lt;/code&gt;. What the help text does not need to say — because it is a convention every CLI follows — is that argument parsing happens before your positional argument is interpreted. A positional argument that begins with a dash is not a positional argument at all. It is an option, by definition, at the layer where the shell's words meet the program's parser.&lt;/p&gt;

&lt;p&gt;This convention has a name and a number. POSIX.1's Utility Syntax Guidelines define &lt;code&gt;--&lt;/code&gt; as the delimiter that ends option parsing (Guideline 10): everything after it is an operand, even if it begins with a dash. Every mainstream argument parser — glibc's getopt, Python's argparse, the Node and Go flag libraries — implements some flavor of this rule. That means two things. First, my failure was not a quirk of one particular CLI; any standards-following parser would have rejected the input the same way. Second, there was always an escape hatch I did not know I needed:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;claude &lt;span class="nt"&gt;-p&lt;/span&gt; &lt;span class="nt"&gt;--&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="si"&gt;$(&lt;/span&gt;&lt;span class="nb"&gt;cat &lt;/span&gt;runbook.md&lt;span class="si"&gt;)&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt;   &lt;span class="c"&gt;# parses fine — everything after -- is an operand&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;So the parser saw &lt;code&gt;---\ncreated: 2026-10-15\ntype: runbook\n...&lt;/code&gt;, tried to match it against its known flags, failed, and printed &lt;code&gt;error: unknown option '---...'&lt;/code&gt; with the entire runbook echoed back as the offending "option". Every layer of my pipeline was working as designed. The design was wrong.&lt;/p&gt;

&lt;h2&gt;
  
  
  My preflight checked everything except the thing that runs
&lt;/h2&gt;

&lt;p&gt;This is the part that stung, because I had been careful. The script has a &lt;code&gt;--check&lt;/code&gt; mode. Before I loaded the launchd job, I ran the preflight and watched it print &lt;code&gt;PREFLIGHT-OK&lt;/code&gt;. I verified the schedule, the working directory, the browser probe, the log directory.&lt;/p&gt;

&lt;p&gt;What the preflight validated: the environment around the task. What it never validated: the task itself. The one line that actually invoked the agent was exercised for the first time at 21:00 on the first night, in production, unattended. Loading a schedule is not running a job. A green preflight means the runway is clear; it says nothing about whether the plane can fly.&lt;/p&gt;

&lt;p&gt;There is a subtler version of this trap worth naming. Even if my preflight had invoked the CLI, it would have invoked it with a fixed, known-good test string — while the real run reads its input from an external file. A preflight with hardcoded inputs proves the binary starts; it says nothing about the contract between the binary and the data you feed it at runtime. The honest fix is a smoke test with the real inputs, end to end: after enabling any unattended job, run it once by hand, exactly as the scheduler will. If that is too expensive, read the first real run's log the next morning before you assume the system exists.&lt;/p&gt;

&lt;h2&gt;
  
  
  A notification is not monitoring
&lt;/h2&gt;

&lt;p&gt;One detail deserves honesty, and a harsher framing than I first gave it. My script did not stay silent: on a non-zero exit it fired a macOS desktop notification — something went wrong, see the log. It fired on both nights. I never saw either one, because notifications on this machine are where focus-mode banners go to die.&lt;/p&gt;

&lt;p&gt;My first draft of this section asked "why did the notification fail?" The better question is why a desktop banner was the observability layer of an unattended job in the first place. A banner is volatile, silenced by Do Not Disturb, and requires a human to be looking at the screen within seconds. Rob Ewaschuk's classic essay &lt;em&gt;My Philosophy on Alerting&lt;/em&gt; (reprinted in Google’s &lt;em&gt;Site Reliability Engineering&lt;/em&gt;) argues that passive channels like email are of very limited value precisely because they get buried; an alert that does not demand immediate human action is noise. A transient GUI banner on an unattended machine is worse than email — it pages no one, ever. That is not a notification problem. It is an architectural anti-pattern: a desktop UI component carrying a production observability responsibility.&lt;/p&gt;

&lt;p&gt;The deeper issue is the direction of the signal. Mine was push-on-failure: the system tries to announce its own death. But anything announced by the dying component shares a failure domain with that component — if the Mac loses power, hangs, or the process is killed before the trap runs, the announcement dies with it. Absolute silence, indistinguishable from success. The robust pattern inverts the direction: the job sends a success heartbeat to an external service when it completes, and the service — not the job — raises the alarm when the heartbeat stops arriving. This is the dead man's switch, or inverted heartbeat, and it is the entire business model of services like Healthchecks.io, Cronitor, and Dead Man's Snitch. Google's SRE book devotes a chapter of its cron discussion to exactly this: for periodic batch jobs, "the expected output did not appear" is the symptom worth alerting on, not any signal the job itself emits.&lt;/p&gt;

&lt;p&gt;Every mature scheduler has some version of this lesson built in. Classic cron's &lt;code&gt;MAILTO&lt;/code&gt; captures whatever the job printed and mails it — which is why &lt;code&gt;&amp;gt; /dev/null 2&amp;gt;&amp;amp;1&lt;/code&gt; in a crontab silently disarms it. systemd has &lt;code&gt;OnFailure=&lt;/code&gt;, which activates a separate diagnostic unit when a service fails, so the reporting path is a different process from the failing one. macOS gave me the exit-status table I never read. My cheap version, until I build a real external heartbeat, is a rule instead of a tool: the first morning after any new unattended job goes live, reading its log is a scheduled task, not a vibe.&lt;/p&gt;

&lt;h2&gt;
  
  
  The fix was one pipe, with three caveats
&lt;/h2&gt;

&lt;p&gt;The corrected invocation strips the frontmatter and feeds the prompt through standard input, where no option parsing applies:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nb"&gt;awk&lt;/span&gt; &lt;span class="s1"&gt;'/^---$/{c++; next} c&amp;gt;=2'&lt;/span&gt; runbook.md | claude &lt;span class="nt"&gt;-p&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;stdin beats the positional argument here for two reasons, and only one of them is option parsing. The other is size: positional arguments live in the fixed-size buffer &lt;code&gt;execve&lt;/code&gt; copies into the new process, capped by &lt;code&gt;ARG_MAX&lt;/code&gt; (1 MB on macOS). A prompt that grows past that ceiling fails with &lt;code&gt;E2BIG&lt;/code&gt; before the program starts at all. A pipe is a stream with no such ceiling, so the fix removes the parsing ambiguity and the size limit in one move.&lt;/p&gt;

&lt;p&gt;Three caveats make this production-grade rather than a snippet:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Pipeline exit codes lie by default.&lt;/strong&gt; A pipeline's exit status is its last command's. If awk crashed mid-stream and the CLI treated empty input as success, the failure would vanish. &lt;code&gt;set -o pipefail&lt;/code&gt; makes the pipeline report any stage's failure.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;launchd does not load your shell's locale.&lt;/strong&gt; A job started by launchd runs without the locale environment your terminal has; byte-oriented tools like awk can misparse multi-byte UTF-8 content unless you set &lt;code&gt;LC_CTYPE&lt;/code&gt; explicitly in the script. My runbooks contain Chinese — this one is not theoretical.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;---&lt;/code&gt; is also Markdown syntax.&lt;/strong&gt; In CommonMark, three dashes are a thematic break — a horizontal rule. My one-liner counts &lt;em&gt;every&lt;/em&gt; line of three dashes, so a horizontal rule in the runbook body would be silently deleted from the prompt: a parsing crash traded for silent data corruption. The hardened filter only strips a frontmatter block at line 1 and stops at its first closing delimiter:
&lt;/li&gt;
&lt;/ol&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nb"&gt;awk&lt;/span&gt; &lt;span class="s1"&gt;'NR==1 &amp;amp;&amp;amp; /^---$/ {f=1; next} f &amp;amp;&amp;amp; /^---$/ {f=0; next} !f'&lt;/span&gt; runbook.md
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;I tested the pipe with a two-word prompt before trusting it, which is what I should have done on day one. The next scheduled run executed all six tasks end to end.&lt;/p&gt;

&lt;p&gt;The total cost of the bug was two quiet nights. The total cost of not noticing could have been weeks — nothing in the system would ever have gotten louder on its own. That asymmetry is the real lesson, and it is why the incident earned a permanent entry in the family's incident log rather than a quick fix and a shrug: unattended automation fails silently by default, and silence only breaks where you have built a place for it to break.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I changed after the incident
&lt;/h2&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Strip metadata before handing files to a CLI.&lt;/strong&gt; Any file that might carry frontmatter gets filtered before its content becomes an argument; stdin beats positional arguments for anything that can start with a dash; and where an argument is unavoidable, &lt;code&gt;--&lt;/code&gt; ends option parsing before it begins.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Smoke-test the invocation with real inputs, not just the environment.&lt;/strong&gt; A preflight proves the surroundings are ready; only a real run with real data proves the pipeline runs.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Check the first-run log the next morning.&lt;/strong&gt; New unattended jobs get a scheduled human review of their first execution, before trust is granted.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Treat silence as the alarm.&lt;/strong&gt; Until a real external heartbeat exists, the interim rule is explicit: no news is bad news, and something — eventually an external service, for now a calendar entry — is responsible for noticing.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Write the incident down where future runs will read it.&lt;/strong&gt; My nightly agent now greps the incident log before starting and appends new entries before finishing, so a mistake is expensive exactly once.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;None of this is clever. That is the point. Unattended systems do not need cleverness; they need the dull discipline of checking that the thing you built is the thing that ran.&lt;/p&gt;







&lt;p&gt;Originally published on my blog: &lt;a href="https://javaeye.dev/blog/silent-failure-unattended-automation/" rel="noopener noreferrer"&gt;https://javaeye.dev/blog/silent-failure-unattended-automation/&lt;/a&gt;&lt;/p&gt;

</description>
      <category>agents</category>
      <category>ai</category>
      <category>automation</category>
      <category>cli</category>
    </item>
    <item>
      <title>Keeping New Sites Visible to AI Search Crawlers</title>
      <dc:creator>Nan</dc:creator>
      <pubDate>Wed, 21 Oct 2026 00:35:34 +0000</pubDate>
      <link>https://dev.to/eye_java_420f1faa10ae8b86/keeping-new-sites-visible-to-ai-search-crawlers-57p5</link>
      <guid>https://dev.to/eye_java_420f1faa10ae8b86/keeping-new-sites-visible-to-ai-search-crawlers-57p5</guid>
      <description>&lt;p&gt;&lt;em&gt;AI crawler policies, bot lists, and platform defaults change quickly. This is a snapshot from October 2026, based on what I run on my own sites today. Re-check the official lists before you copy any of it.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;Since July 2025, Cloudflare blocks AI crawlers on newly onboarded domains by default. If you do nothing, a new site on Cloudflare is simply absent from ChatGPT, Claude, and Perplexity — and nothing errors, nothing warns you, and your analytics stay silent because the traffic never arrives. Here is the four-step checklist I now run on every new static site before I consider it launched.&lt;/p&gt;

&lt;h2&gt;
  
  
  New Cloudflare domains hide from AI crawlers by default
&lt;/h2&gt;

&lt;p&gt;This is the trap, and it is silent. Cloudflare turned on default AI-crawler blocking for new domains on 2025-07-01 (Cloudflare, 2025). An existing site keeps its settings; a &lt;em&gt;new&lt;/em&gt; domain inherits the block. Your pages render fine, Google Search Console shows nothing unusual, and the first symptom is that AI assistants simply never mention your site — a symptom with no dashboard.&lt;/p&gt;

&lt;p&gt;The crawl access layer is worth treating as launch infrastructure, same as DNS and SSL. I missed this on exactly zero sites only because the family SOP caught it before launch — the checklist below is that SOP.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 1: Explicitly allow search-class crawlers in Cloudflare AI Crawl Control
&lt;/h2&gt;

&lt;p&gt;Cloudflare's AI Crawl Control is the switch that matters most. Go in and explicitly allow the &lt;strong&gt;search-class&lt;/strong&gt; crawlers — the bots that fetch pages to answer user queries: &lt;code&gt;OAI-SearchBot&lt;/code&gt; (ChatGPT search), &lt;code&gt;ClaudeBot&lt;/code&gt; (Claude), and &lt;code&gt;PerplexityBot&lt;/code&gt; (Perplexity). These are the ones that make your content quotable inside AI answers.&lt;/p&gt;

&lt;p&gt;Also allow the &lt;strong&gt;user-triggered fetchers&lt;/strong&gt; — bots that fetch a page because a user explicitly asked an AI to read it. Google's list (updated 2025-10/11) includes &lt;code&gt;Google-NotebookLM&lt;/code&gt;, &lt;code&gt;Google-CWS&lt;/code&gt; (Chrome's AI features), and &lt;code&gt;Google-Pinpoint&lt;/code&gt;. Being read on demand is the same value as being read by search bots; block it and "summarize this page for me" fails for your domain.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Crawler class&lt;/th&gt;
&lt;th&gt;Examples&lt;/th&gt;
&lt;th&gt;Why it fetches&lt;/th&gt;
&lt;th&gt;My setting&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Search-class AI bots&lt;/td&gt;
&lt;td&gt;OAI-SearchBot, ClaudeBot, PerplexityBot&lt;/td&gt;
&lt;td&gt;Answer user queries with your content&lt;/td&gt;
&lt;td&gt;Allow&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;User-triggered fetchers&lt;/td&gt;
&lt;td&gt;Google-NotebookLM, Google-CWS, Google-Pinpoint&lt;/td&gt;
&lt;td&gt;A user asked an AI to read this page&lt;/td&gt;
&lt;td&gt;Allow&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Training-class crawlers&lt;/td&gt;
&lt;td&gt;GPTBot, Google-Extended&lt;/td&gt;
&lt;td&gt;Model training corpora&lt;/td&gt;
&lt;td&gt;Allow (optional — I allow them)&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h2&gt;
  
  
  Step 2: Name the bots in robots.txt — and skip invented fields
&lt;/h2&gt;

&lt;p&gt;robots.txt still works, but only for documented fields. Google has said since 2024-10 that it ignores undocumented self-invented directives like &lt;code&gt;NoAI&lt;/code&gt; (Google, 2024). Writing &lt;code&gt;NoAI: false&lt;/code&gt; or similar into robots.txt controls nothing anywhere. Control goes through the official mechanisms: Cloudflare AI Crawl Control for the edge, and named user-agent groups in robots.txt for everything else.&lt;/p&gt;

&lt;p&gt;On my own sites the robots file names the AI crawlers explicitly — ten user-agent groups, every one of them allowed — and a contract test fails the build if any group gains a &lt;code&gt;Disallow&lt;/code&gt;. Named groups beat wildcards here: you can see exactly who you admitted, and a diff on the file is a readable audit trail.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 3: Keep llms.txt, but do not count it as a Google lever
&lt;/h2&gt;

&lt;p&gt;Google's official guidance across three documents (2025-05 generative AI content guidance, 2025-12 AI features documentation, and the 2026-05 AI optimization guide) settles two questions: there is &lt;strong&gt;no separate AI index&lt;/strong&gt; to optimize for, and llms.txt is &lt;strong&gt;not counted&lt;/strong&gt; by Google Search. GEO, in Google's own framing, is traditional SEO done well — crawlable pages with citable, quotable content.&lt;/p&gt;

&lt;p&gt;I still ship &lt;code&gt;/llms.txt&lt;/code&gt; on every site, with this blog's &lt;a href="https://javaeye.dev/llms.txt" rel="noopener noreferrer"&gt;own copy&lt;/a&gt; covering the hub and main pages. It costs almost nothing and non-Google consumers use it. But I keep it out of the ROI story: for Google, its expected value is zero, and pretending otherwise would distort what I measure.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 4: Reconcile with the GSC Generative AI report every month
&lt;/h2&gt;

&lt;p&gt;Since 2026-06, Search Console has a Generative AI report showing how often your content appears in AI experiences (Google, 2026). Two things to know before you read it: it reports &lt;strong&gt;impressions, not clicks&lt;/strong&gt;, and it has no query dimension. You can see that AI surfaces showed your page, not which question triggered it.&lt;/p&gt;

&lt;p&gt;This report is now part of my monthly inspection routine, next to classic queries and pages. One related setting worth leaving alone: the "Search generative AI" participation control (opened globally 2026-08-31) is not a ranking lever in either direction — opting out does not protect rankings and opting in does not boost them. I keep participation on, because for a content site AI visibility is the point.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I verified on my own sites, and what I did not
&lt;/h2&gt;

&lt;p&gt;On this blog the full stack went live on 2026-09-29: robots.txt with ten named AI-crawler groups all allowed, &lt;code&gt;/llms.txt&lt;/code&gt; covering the hub and main pages, a Cloudflare zone audit showing no AI blocking, and curl checks confirming every rule served what I intended. Contract tests lock the robots file so a regression fails the build instead of slipping out. The same SOP now runs across my family of static tool sites — &lt;a href="https://wordroost.com" rel="noopener noreferrer"&gt;the newest of them&lt;/a&gt; went through the same checklist on launch day.&lt;/p&gt;

&lt;p&gt;The honest boundary: all of this verifies &lt;strong&gt;crawl access&lt;/strong&gt;, not outcomes. I can prove the bots are admitted; I cannot yet point at traffic and say the checklist caused it. That part needs months of the GSC Generative AI report before anyone — including me — should claim it works.&lt;/p&gt;

&lt;h2&gt;
  
  
  The one-line version
&lt;/h2&gt;

&lt;p&gt;Allow search-class and user-triggered AI crawlers in Cloudflare AI Crawl Control, name them in robots.txt with no invented fields, keep llms.txt as a free extra with zero Google ROI, and reconcile against the GSC Generative AI report monthly.&lt;/p&gt;

&lt;p&gt;Originally published on my blog: &lt;a href="https://javaeye.dev/blog/ai-crawler-governance-checklist/" rel="noopener noreferrer"&gt;https://javaeye.dev/blog/ai-crawler-governance-checklist/&lt;/a&gt;&lt;/p&gt;

</description>
      <category>seo</category>
      <category>indiehackers</category>
      <category>webdev</category>
      <category>marketing</category>
    </item>
    <item>
      <title>Google's Request Indexing Quota: What I Measured</title>
      <dc:creator>Nan</dc:creator>
      <pubDate>Sun, 18 Oct 2026 13:42:03 +0000</pubDate>
      <link>https://dev.to/eye_java_420f1faa10ae8b86/googles-request-indexing-quota-what-i-measured-6ce</link>
      <guid>https://dev.to/eye_java_420f1faa10ae8b86/googles-request-indexing-quota-what-i-measured-6ce</guid>
      <description>&lt;p&gt;&lt;em&gt;Search Console limits, labels, and quotas change without notice — the quota here is not documented at all. Every number below is a snapshot from early October 2026, measured on my own Google account. Re-measure on yours before you plan around it.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;Google Search Console's "Request Indexing" has a daily quota. The help documentation confirms that a limit exists and never says how large it is; the number people repeat in forums — ten URLs a day — has no official source I could find. I spent three days measuring mine. The short version: the quota is shared by every property under one Google account rather than counted per site, the daily number floats, and it recovers hours before Pacific midnight. Here is what I actually logged.&lt;/p&gt;

&lt;h2&gt;
  
  
  The quota belongs to the account, not to the site
&lt;/h2&gt;

&lt;p&gt;This is the finding that changed how I work, and it is the one I got wrong first. My earliest notes said the limit was counted per account, per property, per day — I assumed each site had its own budget. Cross-property testing disproved it. A small family of free tool sites is what I actually run — the eight of them are listed on my &lt;a href="https://dev.to/tools"&gt;/tools&lt;/a&gt; page, and &lt;a href="https://wordroost.com" rel="noopener noreferrer"&gt;WordRoost&lt;/a&gt; is the newest — plus this blog, and when one site under my account had spent the day's quota, a different site under the same account could not submit either.&lt;/p&gt;

&lt;p&gt;The clearest single day is 2026-10-09. Sixteen requests succeeded that day, and they were spread across four different properties under one account: one site took one, another took six, this blog took eight, and a fourth took one. Property boundaries reset nothing. The pool was the account's.&lt;/p&gt;

&lt;p&gt;Two things that do &lt;em&gt;not&lt;/em&gt; change the pool, which I tested specifically because both are common advice: the IP address you submit from, and whether you use an incognito window. Neither made any difference. The only action that gets you a different pool is a different Google account.&lt;/p&gt;

&lt;h2&gt;
  
  
  The daily number floats, so plan for a range
&lt;/h2&gt;

&lt;p&gt;I have two clean samples of a full day's capacity: 11 and 13. On a third day, 16 requests went through and the 17th was rejected. So "the quota" is not a fixed number you can memorise. Treat it as a range somewhere in the low teens, and stop guessing at an exact figure.&lt;/p&gt;

&lt;p&gt;The shape of a batch tells you which end of that range you are on. On 2026-10-07 my blog's batch submitted four URLs and the fifth was rejected — the account had already been used that day, so four was what remained rather than the day's total. The next full batch, on 2026-10-09, submitted eight URLs with nothing left over. Same account, same tool, very different amounts available, because the pool depends on whatever else the account did that day.&lt;/p&gt;

&lt;h2&gt;
  
  
  Recovery happens before Pacific midnight
&lt;/h2&gt;

&lt;p&gt;The advice I kept reading was to wait for midnight in Pacific time. My logs disagree. After one morning cap, the next day's first submissions at 13:19 Beijing time went straight through. That is hours earlier than Pacific midnight, so I moved my working window to "any time after 13:00 Beijing" and stopped waiting until the evening.&lt;/p&gt;

&lt;p&gt;I also tested the other popular claim, that the quota resets at 15:00 Beijing. It does not. On one day the first request at 14:31 was already rejected, a re-test at 15:00:54 was still rejected, and the pool only came back at 17:05. Recovery is not always generous either: one recovery gave exactly one submission before the cap returned.&lt;/p&gt;

&lt;h2&gt;
  
  
  The first click is a usable signal for how much is left
&lt;/h2&gt;

&lt;p&gt;This is the small observation I rely on most now. When the quota is nearly gone, the first click on a URL returns a transient error — a generic message that there was a problem submitting the indexing request — and the &lt;em&gt;second&lt;/em&gt; click succeeds. When the pool is fresh, the first click succeeds and there is no error at all.&lt;/p&gt;

&lt;p&gt;Both batches fit the pattern exactly. The 2026-10-07 batch, running against a partly used pool, had a first-click error on all four URLs. The 2026-10-09 batch, running against a fresh pool, had zero errors across eight.&lt;/p&gt;

&lt;p&gt;So if you are working through a batch and every first attempt fails, that is not a broken tool. It is the quota telling you it is nearly empty — and it is a good moment to write down where you stopped.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I do now: stop at the cap, roll the rest, keep the screenshots
&lt;/h2&gt;

&lt;p&gt;Three rules, all of them forced by these runs:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Stop the moment the quota is hit.&lt;/strong&gt; The remaining URLs go to the next day. Switching accounts to keep going is not worth it, for the reason in the next section.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Screenshot every confirmation.&lt;/strong&gt; The success state is the line saying the URL has been added to the priority crawl queue. A line you typed into your own log is not evidence; a screenshot is.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Re-check read-only.&lt;/strong&gt; After a batch, checking whether a URL is already indexed does not consume quota. Only the submit action does.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;For cost: a four-URL batch ran about five minutes end to end, and an eight-URL batch about eight minutes. The clicking is not the bottleneck. The queue is.&lt;/p&gt;

&lt;h2&gt;
  
  
  A second account is not a second full quota
&lt;/h2&gt;

&lt;p&gt;The obvious workaround is to keep a spare Google account and switch when the first one runs dry. I tried it, and it mostly does not work. Quota is attached to the account, and my spare accounts already carried properties of their own — so their budget for the day was partly spent before I arrived. On the worst day, a spare account's very first request was rejected.&lt;/p&gt;

&lt;p&gt;Keep a spare if you want one, but look at what else that account does before you count on it. An account with several properties under it is not a fresh account.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I would tell anyone batching indexing requests
&lt;/h2&gt;

&lt;p&gt;The quota is real, it is smaller than the folklore suggests, and it is not the thing worth optimising. What matters is knowing which pool you are drawing from and stopping cleanly when it is empty. Everything above is one account's behaviour in one week of October 2026, not a documented Google promise — so measure your own before you build a schedule on it.&lt;/p&gt;

&lt;p&gt;Originally published on my blog: &lt;a href="https://javaeye.dev/blog/google-request-indexing-quota/" rel="noopener noreferrer"&gt;https://javaeye.dev/blog/google-request-indexing-quota/&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;This article was created with the help of AI. AI was not used to write the content; it assisted only with translation and grammar checks.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>seo</category>
      <category>indiehackers</category>
      <category>webdev</category>
      <category>marketing</category>
    </item>
  </channel>
</rss>
