<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: fei mai</title>
    <description>The latest articles on DEV Community by fei mai (@mediadownloaderguides).</description>
    <link>https://dev.to/mediadownloaderguides</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4168281%2F94a4437c-a608-434d-830d-1ba393d1e994.png</url>
      <title>DEV Community: fei mai</title>
      <link>https://dev.to/mediadownloaderguides</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/mediadownloaderguides"/>
    <language>en</language>
    <item>
      <title>Our Best Page Is the One Nothing Links To</title>
      <dc:creator>fei mai</dc:creator>
      <pubDate>Tue, 17 Nov 2026 08:18:27 +0000</pubDate>
      <link>https://dev.to/mediadownloaderguides/our-best-page-is-the-one-nothing-links-to-cal</link>
      <guid>https://dev.to/mediadownloaderguides/our-best-page-is-the-one-nothing-links-to-cal</guid>
      <description>&lt;p&gt;Fourteen URLs. Seventeen internal links. One audit that started as a bug report and ended as something else entirely.&lt;/p&gt;

&lt;p&gt;I crawled jobradarly.com page by page on 17 November 2026, and the number that came back is not the one I expected.&lt;/p&gt;

&lt;h2&gt;
  
  
  How the crawl worked
&lt;/h2&gt;

&lt;p&gt;All 14 URLs from the sitemap, one request each, 200 on all of them. Then every &lt;code&gt;&amp;lt;a href&amp;gt;&lt;/code&gt; in the saved HTML was regexed out, relative paths resolved, anchors and mailto dropped, same-domain targets kept.&lt;/p&gt;

&lt;p&gt;Seventeen unique internal targets. That's the whole link graph of a fourteen-page site.&lt;/p&gt;

&lt;p&gt;Each target was re-requested with redirects disabled, so a 301 surfaces as a 301 instead of resolving quietly to something else. Every non-200 got two more attempts before being recorded. Those retries have caught four false dead-link reports for me this year.&lt;/p&gt;

&lt;h2&gt;
  
  
  The table
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Target&lt;/th&gt;
&lt;th&gt;Status&lt;/th&gt;
&lt;th&gt;Pages linking to it&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;/&lt;/td&gt;
&lt;td&gt;200&lt;/td&gt;
&lt;td&gt;13&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;/about&lt;/td&gt;
&lt;td&gt;200&lt;/td&gt;
&lt;td&gt;10&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;/contact&lt;/td&gt;
&lt;td&gt;200&lt;/td&gt;
&lt;td&gt;12&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;/guide/&lt;/td&gt;
&lt;td&gt;200&lt;/td&gt;
&lt;td&gt;10&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;/guide/h1b-sponsorship-jobs/&lt;/td&gt;
&lt;td&gt;200&lt;/td&gt;
&lt;td&gt;7&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;/guide/recently-posted-jobs/&lt;/td&gt;
&lt;td&gt;200&lt;/td&gt;
&lt;td&gt;6&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;/guide/how-long-do-job-postings-stay-up/&lt;/td&gt;
&lt;td&gt;200&lt;/td&gt;
&lt;td&gt;4&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;/guide/how-many-jobs-should-i-apply-to/&lt;/td&gt;
&lt;td&gt;200&lt;/td&gt;
&lt;td&gt;4&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;/guide/best-job-aggregator-sites/&lt;/td&gt;
&lt;td&gt;200&lt;/td&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;/guide/new-grad-jobs-2027/&lt;/td&gt;
&lt;td&gt;200&lt;/td&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;/guide/limits/&lt;/td&gt;
&lt;td&gt;200&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;/pricing&lt;/td&gt;
&lt;td&gt;200&lt;/td&gt;
&lt;td&gt;11&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;/privacy&lt;/td&gt;
&lt;td&gt;200&lt;/td&gt;
&lt;td&gt;11&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;/terms&lt;/td&gt;
&lt;td&gt;200&lt;/td&gt;
&lt;td&gt;11&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;/login&lt;/td&gt;
&lt;td&gt;200&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;jobradarly.com (no slash)&lt;/td&gt;
&lt;td&gt;200&lt;/td&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;/cdn-cgi/l/email-protection&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;404&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;13&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h2&gt;
  
  
  Three findings that broke my first draft
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;One: the only 404 belongs to Cloudflare.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;code&gt;/cdn-cgi/l/email-protection&lt;/code&gt; appears on thirteen of fourteen pages because Cloudflare's email obfuscation rewrites mailto links into scripted ones. Browser: fine. Raw HTML with JavaScript off: a link to nothing.&lt;/p&gt;

&lt;p&gt;So the site has zero broken internal links. My first draft said thirteen, and it was wrong in the most embarrassing direction — the checker was reporting its own blindness.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Two: the most substantial page is the least linked.&lt;/strong&gt;&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Guide&lt;/th&gt;
&lt;th&gt;Body words&lt;/th&gt;
&lt;th&gt;Inbound from other guides&lt;/th&gt;
&lt;th&gt;FAQ questions&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;/guide/limits/&lt;/td&gt;
&lt;td&gt;2,852&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;0&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;12&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;/guide/how-long-do-job-postings-stay-up/&lt;/td&gt;
&lt;td&gt;2,144&lt;/td&gt;
&lt;td&gt;3&lt;/td&gt;
&lt;td&gt;5&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;/guide/h1b-sponsorship-jobs/&lt;/td&gt;
&lt;td&gt;1,747&lt;/td&gt;
&lt;td&gt;6&lt;/td&gt;
&lt;td&gt;6&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;/guide/recently-posted-jobs/&lt;/td&gt;
&lt;td&gt;1,522&lt;/td&gt;
&lt;td&gt;5&lt;/td&gt;
&lt;td&gt;5&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;/guide/new-grad-jobs-2027/&lt;/td&gt;
&lt;td&gt;1,481&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;5&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;/guide/how-many-jobs-should-i-apply-to/&lt;/td&gt;
&lt;td&gt;1,431&lt;/td&gt;
&lt;td&gt;3&lt;/td&gt;
&lt;td&gt;5&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;/guide/best-job-aggregator-sites/&lt;/td&gt;
&lt;td&gt;1,250&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;5&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Length and links run in opposite directions. The 2,852-word page about what the product can't do gets nothing from its six siblings; the 1,250-word page gets one.&lt;/p&gt;

&lt;p&gt;Nineteen sibling links across the cluster. Eleven of those nineteen, 58%, land on two pages.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Three: 54 FAQ questions are marked up for a feature that no longer shows.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Twelve on the limits page, eight on pricing, five or six per guide, three on about. No duplicates anywhere — this was hand-written, carefully, 54 times.&lt;/p&gt;

&lt;p&gt;FAQ rich results have been restricted to government and health sites since 2023. All 54 questions are currently worth zero pixels in a search result.&lt;/p&gt;

&lt;p&gt;And on the same eight pages: zero Article blocks declare an &lt;code&gt;image&lt;/code&gt;, which is the property that still gets rewarded.&lt;/p&gt;

&lt;h2&gt;
  
  
  What this audit can't see
&lt;/h2&gt;

&lt;p&gt;Rankings and indexing. I don't have Search Console for this domain open, so I can't tell you whether these 14 URLs are indexed, whether the cluster ranks for anything, or whether the link imbalance costs a single visit.&lt;/p&gt;

&lt;p&gt;I didn't test a JavaScript-rendering crawler either. Everything here is the raw HTML one request returns.&lt;/p&gt;

&lt;p&gt;And I can't prove the pattern is accidental. The limits page may be deliberately quiet because it's the least commercial page on the site. If so it's a decision, and not mine to overrule.&lt;/p&gt;

&lt;h2&gt;
  
  
  A correction we owe
&lt;/h2&gt;

&lt;p&gt;In October I wrote that this site had zero JobPosting markup and no job detail URLs, and used it as evidence it "skips structured data".&lt;/p&gt;

&lt;p&gt;Half of that was wrong. Eight of these pages carry three JSON-LD blocks each — Article, BreadcrumbList, FAQPage — with dates that match the sitemap's &lt;code&gt;lastmod&lt;/code&gt; exactly. I checked expecting a mismatch and found none.&lt;/p&gt;

&lt;p&gt;The JobPosting half still holds. The other half was sloppy writing, and sloppy writing is what gets quoted back at you.&lt;/p&gt;

&lt;h2&gt;
  
  
  Disclosure
&lt;/h2&gt;

&lt;p&gt;JobRadarly is one of the sites we build. All fourteen URLs crawled here are ours, and nothing was changed to steer the result.&lt;/p&gt;

&lt;p&gt;Site: &lt;a href="https://jobradarly.com/" rel="noopener noreferrer"&gt;jobradarly.com&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Left hanging
&lt;/h2&gt;

&lt;p&gt;Four sentences of HTML and ten minutes fixes the inverted links.&lt;/p&gt;

&lt;p&gt;Should I do it? A page that honestly lists what your product can't do is either the most trust-building page you own or the fastest way to lose a signup. I have one data point on that, and it's my own opinion.&lt;/p&gt;

</description>
      <category>seo</category>
      <category>webdev</category>
      <category>webscraping</category>
    </item>
    <item>
      <title>I Read 39 Arguments About Whether to Buy Something. One Field Predicted Every Verdict.</title>
      <dc:creator>fei mai</dc:creator>
      <pubDate>Tue, 17 Nov 2026 08:18:14 +0000</pubDate>
      <link>https://dev.to/mediadownloaderguides/i-read-39-arguments-about-whether-to-buy-something-one-field-predicted-every-verdict-3kpj</link>
      <guid>https://dev.to/mediadownloaderguides/i-read-39-arguments-about-whether-to-buy-something-one-field-predicted-every-verdict-3kpj</guid>
      <description>&lt;p&gt;Thirty-nine people argued for or against a purchase in front of strangers. I read all of them on 17 November 2026, then I read the JSON behind them.&lt;/p&gt;

&lt;p&gt;The JSON was more interesting than the arguments.&lt;/p&gt;

&lt;h2&gt;
  
  
  The data source
&lt;/h2&gt;

&lt;p&gt;No browser, no scraping loop. WalletJury's front end loads cases from a public endpoint, so the audit is one request:&lt;/p&gt;

&lt;p&gt;&lt;code&gt;GET https://walletjury.com/api/list?sort=new&amp;amp;limit=200&amp;amp;offset=0&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;39 cases came back with &lt;code&gt;next: null&lt;/code&gt;. Null means no more pages, so it's the entire database rather than a sample — nothing filtered, nothing skipped, 29,402 bytes.&lt;/p&gt;

&lt;p&gt;Then we dumped every key on every case. Twenty-one keys.&lt;/p&gt;

&lt;h2&gt;
  
  
  What the fields actually contain
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Field&lt;/th&gt;
&lt;th&gt;Filled&lt;/th&gt;
&lt;th&gt;Detail&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;item&lt;/td&gt;
&lt;td&gt;39/39&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;price&lt;/td&gt;
&lt;td&gt;39/39&lt;/td&gt;
&lt;td&gt;20 to 12,999&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;currency&lt;/td&gt;
&lt;td&gt;39/39&lt;/td&gt;
&lt;td&gt;USD 37, CNY 2&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;defense&lt;/td&gt;
&lt;td&gt;39/39&lt;/td&gt;
&lt;td&gt;32–375 characters, median 244&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;use&lt;/td&gt;
&lt;td&gt;35/39&lt;/td&gt;
&lt;td&gt;Daily 19, Weekly 8, Not sure 7, few times a year 1&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;afford&lt;/td&gt;
&lt;td&gt;39/39&lt;/td&gt;
&lt;td&gt;stretch 22, easily 10, not really 7&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;anon&lt;/td&gt;
&lt;td&gt;39/39&lt;/td&gt;
&lt;td&gt;all true&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;authorName&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;0/39&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;never once&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;hasImage&lt;/td&gt;
&lt;td&gt;1/39&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;ai.lean&lt;/td&gt;
&lt;td&gt;39/39&lt;/td&gt;
&lt;td&gt;guilty 36, not guilty 3&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;ai.opinion&lt;/td&gt;
&lt;td&gt;39/39&lt;/td&gt;
&lt;td&gt;20–121 characters, median 52&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;outcome&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;0/39&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;the thing the product promises&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;votes&lt;/td&gt;
&lt;td&gt;46 total&lt;/td&gt;
&lt;td&gt;spread over 17 cases only&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;replies&lt;/td&gt;
&lt;td&gt;51 total&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Twenty-one fields. Seven carry real variation: item, price, currency, defense, use, afford, and the AI's one-line opinion.&lt;/p&gt;

&lt;h2&gt;
  
  
  Three things I did not expect
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;One: 29 out of 29.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Every case where the poster picked "It's a stretch" (22 cases) or "Not really" (7 cases) came back Guilty. All 29, no exceptions across the whole database.&lt;/p&gt;

&lt;p&gt;The only three Not Guilty verdicts sit inside the ten "Easily" cases. A 30% acquittal rate in one bucket, 0% in the other two.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Two: the vote count doesn't spread.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;22 of 39 cases have zero votes. The other 17 have exactly 3 votes (12 cases) or exactly 2 (5 cases). Nothing between the two, nothing above three.&lt;/p&gt;

&lt;p&gt;46 votes have been cast on this database, all inside those 17 cases, and every one landed on the same two numbers. That is not a distribution. That is a shape.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Three: three fields have never done anything.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;code&gt;authorName&lt;/code&gt; is empty on all 39. &lt;code&gt;hasImage&lt;/code&gt; is true on one. &lt;code&gt;outcome&lt;/code&gt; is null on all 39 — the field that would record what the jury decided.&lt;/p&gt;

&lt;p&gt;Meanwhile &lt;code&gt;afford&lt;/code&gt;, the field nobody thinks about, splits the dataset cleanly in two.&lt;/p&gt;

&lt;h2&gt;
  
  
  What we could not measure
&lt;/h2&gt;

&lt;p&gt;Why. We can see the output and not the reasoning. Three readings fit this table:&lt;/p&gt;

&lt;p&gt;The model reads &lt;code&gt;afford&lt;/code&gt; and treats "I can easily afford this" as a reason to acquit. Or it ignores &lt;code&gt;afford&lt;/code&gt; and reads the text — people who pick "easily" write differently than people who pick "stretch". Or it is self-selection: if you can afford the thing, you only post the genuinely borderline ones, and borderline cases get acquitted.&lt;/p&gt;

&lt;p&gt;Telling those three apart needs controlled submissions. I am not going to post fake cases into a live product to get a prettier chart. The correlation is 29 for 29 and that is the whole of what I can defend.&lt;/p&gt;

&lt;p&gt;We also didn't render the site in a browser for this piece. Everything above is the API payload, read as text.&lt;/p&gt;

&lt;h2&gt;
  
  
  A correction to our own notes
&lt;/h2&gt;

&lt;p&gt;On 14 November I wrote that this site had 39 cases and 46 votes. It is 17 November and it is still 39 and 46.&lt;/p&gt;

&lt;p&gt;I wrote that assuming the next pull would show movement. The newest case was created 23 October, 13:08 UTC, and nothing has arrived in the 25 days since. The first version should have said "unchanged since 23 October".&lt;/p&gt;

&lt;h2&gt;
  
  
  Disclosure
&lt;/h2&gt;

&lt;p&gt;WalletJury is one of the sites we build and run. These numbers come from its own public endpoint, and nothing on the site was changed to produce them.&lt;/p&gt;

&lt;p&gt;All 39 cases are public: &lt;a href="https://walletjury.com/" rel="noopener noreferrer"&gt;walletjury.com&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  One open question
&lt;/h2&gt;

&lt;p&gt;If you were building this, what do you do with a field that predicts the verdict 29 times out of 29?&lt;/p&gt;

&lt;p&gt;Keep it, because it works. Drop it, because a jury that reads your bank balance before your argument is not a jury.&lt;/p&gt;

&lt;p&gt;I genuinely don't know which answer is right.&lt;/p&gt;

</description>
    </item>
    <item>
      <title>The Filename Said "models". 80% of the File Wasn't.</title>
      <dc:creator>fei mai</dc:creator>
      <pubDate>Mon, 16 Nov 2026 08:17:14 +0000</pubDate>
      <link>https://dev.to/mediadownloaderguides/the-filename-said-models-80-of-the-file-wasnt-305e</link>
      <guid>https://dev.to/mediadownloaderguides/the-filename-said-models-80-of-the-file-wasnt-305e</guid>
      <description>&lt;p&gt;&lt;code&gt;sitemap-models.xml&lt;/code&gt; on our appliance site is 1.4 MB. I parsed it, saw 8,400 &lt;code&gt;&amp;lt;loc&amp;gt;&lt;/code&gt; entries, and wrote "8,400 model pages" into a note. Then I built half a chart out of that number before checking what the URLs actually pointed at.&lt;/p&gt;

&lt;p&gt;817 brand-category pages. 6,756 detail sub-pages. 827 model pages.&lt;/p&gt;

&lt;p&gt;The number I wanted was three percent of the file.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why I checked at all
&lt;/h2&gt;

&lt;p&gt;We had a second sitemap — &lt;code&gt;sitemap-core.xml&lt;/code&gt;, 1,668 URLs. Two files, one site, and the obvious question is how much they overlap. The answer is zero. Not "a little" — the intersection of the two URL sets is the empty set.&lt;/p&gt;

&lt;p&gt;That's allowed. A sitemap index can point at disjoint shards. But disjoint shards with names that describe neither shard is how you end up quoting 8,400 model pages in a meeting.&lt;/p&gt;

&lt;h2&gt;
  
  
  What the two files hold
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;File&lt;/th&gt;
&lt;th&gt;URLs&lt;/th&gt;
&lt;th&gt;Model pages&lt;/th&gt;
&lt;th&gt;Brand-category&lt;/th&gt;
&lt;th&gt;Detail sub-pages&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;sitemap-core.xml&lt;/td&gt;
&lt;td&gt;1,668&lt;/td&gt;
&lt;td&gt;1,655&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;sitemap-models.xml&lt;/td&gt;
&lt;td&gt;8,400&lt;/td&gt;
&lt;td&gt;827&lt;/td&gt;
&lt;td&gt;817&lt;/td&gt;
&lt;td&gt;6,756&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Overlap&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;—&lt;/td&gt;
&lt;td&gt;—&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;code&gt;core&lt;/code&gt; is 16 brands. &lt;code&gt;models&lt;/code&gt; is 111 brands, 101 of which appear nowhere in &lt;code&gt;core&lt;/code&gt;. So the split isn't curated-versus-generated, it's big-brands-in-one-file and everything-else-in-the-other, with a filename that claims otherwise.&lt;/p&gt;

&lt;h2&gt;
  
  
  Grouping the detail pages
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Suffix&lt;/th&gt;
&lt;th&gt;Count&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;/manual/&lt;/td&gt;
&lt;td&gt;1,689&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;/specs/&lt;/td&gt;
&lt;td&gt;1,689&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;/parts/&lt;/td&gt;
&lt;td&gt;1,688&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;/problems/&lt;/td&gt;
&lt;td&gt;1,688&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;/error-codes/&lt;/td&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Four suffixes cover 1,689 models. Total model root pages: 2,482. Which leaves 793 models declared with nothing behind them.&lt;/p&gt;

&lt;p&gt;We fetched &lt;code&gt;/manual/&lt;/code&gt; on four of those 793 at random. All four returned 200.&lt;/p&gt;

&lt;h2&gt;
  
  
  Three things I didn't want to find
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Roughly a third of the site is undeclared.&lt;/strong&gt; 793 × 4 = 3,172 pages that answer 200 and appear in neither sitemap file. If you count our pages from the sitemaps you get 6,756 detail pages. There are probably 9,928.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;87% of the gap is one category.&lt;/strong&gt; 693 of the 793 undeclared models are refrigerators, the second-biggest category we have. Manual, specs, parts and problems run across eight categories and skip the one with the most models. Nobody wrote that down as a decision.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Two page types, split 1,688 to 2.&lt;/strong&gt; &lt;code&gt;/problems/&lt;/code&gt; and &lt;code&gt;/error-codes/&lt;/code&gt; are the same idea with two names. &lt;code&gt;/problems/&lt;/code&gt; has 1,688 URLs. &lt;code&gt;/error-codes/&lt;/code&gt; has two: a Maytag MVWB965H and a Whirlpool WTW8000D, both washers. Our homepage title tag says "Appliance Manuals, Specs, Parts &amp;amp; Error Codes," which is a promise backed by two URLs.&lt;/p&gt;

&lt;p&gt;So I opened four &lt;code&gt;/problems/&lt;/code&gt; pages. Each one states there's no public code table for that model. None of the four contains a code.&lt;/p&gt;

&lt;h2&gt;
  
  
  Where the note went wrong
&lt;/h2&gt;

&lt;p&gt;I counted a sub-page as a model page because the parent file said models. That's a naming bug in the artifact, not a parsing bug in my code — and it's the more dangerous kind, because the parse looked correct. Renaming the file wouldn't fix the site, but it would have cost me nothing and saved the hour.&lt;/p&gt;

&lt;h2&gt;
  
  
  What this doesn't tell you
&lt;/h2&gt;

&lt;p&gt;Whether any of the 3,172 undeclared pages got indexed. Sitemaps are a hint. Plenty of unlisted URLs get crawled through internal links and plenty of listed ones never get looked at. That's Search Console coverage data, and I didn't pull it.&lt;/p&gt;

&lt;p&gt;And four out of 1,688 is enough to say "these pages exist and say this." It's not enough to say "all 1,688 are empty." I'm not saying that.&lt;/p&gt;

&lt;h2&gt;
  
  
  Disclosure
&lt;/h2&gt;

&lt;p&gt;Our site: &lt;a href="https://modelnumberfinder.com/" rel="noopener noreferrer"&gt;modelnumberfinder.com&lt;/a&gt;. Both sitemaps are public and every figure here comes from a parse you can rerun with &lt;code&gt;xml.etree&lt;/code&gt; and about ten lines.&lt;/p&gt;

&lt;p&gt;The fix is a sitemap generated from the route table instead of a maintained list, and one page type where there are two. Both are boring. That's why they're still open.&lt;/p&gt;

</description>
      <category>analytics</category>
      <category>seo</category>
      <category>webscraping</category>
    </item>
    <item>
      <title>I Wrote a 40-Line Script to Count Title Tags and Got the Answer Wrong Twice</title>
      <dc:creator>fei mai</dc:creator>
      <pubDate>Mon, 16 Nov 2026 08:17:00 +0000</pubDate>
      <link>https://dev.to/mediadownloaderguides/i-wrote-a-40-line-script-to-count-title-tags-and-got-the-answer-wrong-twice-504p</link>
      <guid>https://dev.to/mediadownloaderguides/i-wrote-a-40-line-script-to-count-title-tags-and-got-the-answer-wrong-twice-504p</guid>
      <description>&lt;p&gt;The script is 40 lines. It takes a sitemap, fetches every URL one at a time, pulls &lt;code&gt;&amp;lt;title&amp;gt;&lt;/code&gt;, and prints a length. I ran it against a flashcard site we run and got 160 pages over 60 characters.&lt;/p&gt;

&lt;p&gt;The real number is 144. Both errors were mine, and both are the kind that make a data post useless if you don't catch them.&lt;/p&gt;

&lt;h2&gt;
  
  
  Error one: entity length
&lt;/h2&gt;

&lt;p&gt;&lt;code&gt;&amp;amp;amp;&lt;/code&gt; is one character of title and five characters of source. Twenty pages in that set have an ampersand in the book title. I counted the source. Fix: &lt;code&gt;html.unescape()&lt;/code&gt; before &lt;code&gt;len()&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Sixteen pages flipped. That's 8% of the site, from one missing function call.&lt;/p&gt;

&lt;h2&gt;
  
  
  Error two: threads
&lt;/h2&gt;

&lt;p&gt;The first version used a thread pool of eight. It returned 28 errors on a 191-page site and I spent twenty minutes looking for missing pages that weren't missing — they were timeouts that surfaced as failed requests, indistinguishable from a real negative answer.&lt;/p&gt;

&lt;p&gt;The rewrite is single-threaded with a 250 ms sleep. 191 pages, 48 seconds, zero errors, and a 404 now looks like a 404.&lt;/p&gt;

&lt;h2&gt;
  
  
  What the 191 pages actually look like
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Page type&lt;/th&gt;
&lt;th&gt;Pages&lt;/th&gt;
&lt;th&gt;Mean title&lt;/th&gt;
&lt;th&gt;Shortest&lt;/th&gt;
&lt;th&gt;Longest&lt;/th&gt;
&lt;th&gt;Over 60 chars&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Book-card pages&lt;/td&gt;
&lt;td&gt;156&lt;/td&gt;
&lt;td&gt;83 chars&lt;/td&gt;
&lt;td&gt;52&lt;/td&gt;
&lt;td&gt;136&lt;/td&gt;
&lt;td&gt;133&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Blog posts&lt;/td&gt;
&lt;td&gt;23&lt;/td&gt;
&lt;td&gt;58 chars&lt;/td&gt;
&lt;td&gt;44&lt;/td&gt;
&lt;td&gt;64&lt;/td&gt;
&lt;td&gt;10&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Landing, legal, index&lt;/td&gt;
&lt;td&gt;11&lt;/td&gt;
&lt;td&gt;50 chars&lt;/td&gt;
&lt;td&gt;26&lt;/td&gt;
&lt;td&gt;64&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;All pages&lt;/td&gt;
&lt;td&gt;191&lt;/td&gt;
&lt;td&gt;78 chars&lt;/td&gt;
&lt;td&gt;26&lt;/td&gt;
&lt;td&gt;136&lt;/td&gt;
&lt;td&gt;144&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Every URL returned 200 on the first attempt, so there's no dead-link noise in this dataset.&lt;/p&gt;

&lt;h2&gt;
  
  
  The part I expected to be bad and wasn't
&lt;/h2&gt;

&lt;p&gt;I assumed the head tags were a mess. They aren't. Zero pages are missing a description. Zero of the 191 descriptions collide with another. Every canonical points at its own page. Exactly one H1 per page, no stray &lt;code&gt;robots&lt;/code&gt; meta. Nine of ten checks pass.&lt;/p&gt;

&lt;p&gt;One field is broken and it's the one nobody looks at, because the checker tool most people use shows you the title length and you think "I'll get to it."&lt;/p&gt;

&lt;h2&gt;
  
  
  The boilerplate tax
&lt;/h2&gt;

&lt;p&gt;All 156 book pages end with the identical string:&lt;/p&gt;

&lt;p&gt;&lt;code&gt;Flashcards: Key Concepts &amp;amp; Review | WakeCards&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;Forty-six characters, on every page, before you write anything about the book. Which leaves fourteen characters of budget. The median book title on the site is 37.&lt;/p&gt;

&lt;p&gt;Strip the brand suffix and the mean drops 78 → 66, and pages over 60 drop from 144 to 98. So half of it is a suffix, and half of it is that we wrote a sentence where a noun phrase belonged.&lt;/p&gt;

&lt;h2&gt;
  
  
  What the script can't tell you
&lt;/h2&gt;

&lt;p&gt;Whether Google truncates any of these. Truncation is pixel-width at a font size I don't set and can't query. Google also rewrites titles it doesn't like, and the rate is a Search Console number I didn't pull. Sixty characters is a convention from tooling, not a fact about Google.&lt;/p&gt;

&lt;p&gt;What I can measure is the input: 144 pages carry 25.8 characters more than that convention allows.&lt;/p&gt;

&lt;h2&gt;
  
  
  Disclosure
&lt;/h2&gt;

&lt;p&gt;The site is &lt;a href="https://wakecards.com/" rel="noopener noreferrer"&gt;wakecards.com&lt;/a&gt;, one of ours. The script and the page list are in this pass only — no sampling, all 191 URLs.&lt;/p&gt;

&lt;p&gt;The fix is 156 rewrites with 156 different tails. No template. That's the part that doesn't fit in 40 lines.&lt;/p&gt;

</description>
      <category>debugging</category>
      <category>programming</category>
      <category>python</category>
      <category>webscraping</category>
    </item>
    <item>
      <title>The Answer Checker Explains the Missing Question</title>
      <dc:creator>fei mai</dc:creator>
      <pubDate>Sun, 15 Nov 2026 08:11:24 +0000</pubDate>
      <link>https://dev.to/mediadownloaderguides/the-answer-checker-explains-the-missing-question-3aa9</link>
      <guid>https://dev.to/mediadownloaderguides/the-answer-checker-explains-the-missing-question-3aa9</guid>
      <description>&lt;p&gt;Our practice page grades answers with one line:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="nf"&gt;parse&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$(&lt;/span&gt;&lt;span class="s1"&gt;'#practice-answer'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;value&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;eq&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;answer&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;If &lt;code&gt;answer&lt;/code&gt; is &lt;code&gt;undefined&lt;/code&gt;, that line throws. Not "marks it wrong" — throws, &lt;code&gt;Cannot read properties of undefined (reading 'n')&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;That one line is why our question bank has no vertical-line question, and why three of our articles teach a case the tool will never test. I found it by running the bank through the site's own function instead of reading it.&lt;/p&gt;

&lt;h2&gt;
  
  
  How we measured
&lt;/h2&gt;

&lt;p&gt;Two files off the live site: &lt;code&gt;/assets/app.mjs&lt;/code&gt; (38,731 bytes, holds the question array) and &lt;code&gt;/assets/math.mjs&lt;/code&gt; (9,985 bytes, holds &lt;code&gt;slope()&lt;/code&gt; and the exact-fraction &lt;code&gt;parse()&lt;/code&gt;).&lt;/p&gt;

&lt;p&gt;I copied the array verbatim — ten &lt;code&gt;[x1, y1, x2, y2]&lt;/code&gt; string arrays — and fed each one to the exported &lt;code&gt;slope()&lt;/code&gt;. Node 22, ESM import, no DOM. Then I forced a vertical pair through the same function to see what the page would do with it.&lt;/p&gt;

&lt;p&gt;For the grader I called the same &lt;code&gt;parse().eq()&lt;/code&gt; chain the page calls, with eight inputs, and caught the throws instead of letting them print.&lt;/p&gt;

&lt;p&gt;Two things I had to fix in my own harness along the way: &lt;code&gt;JSON.stringify&lt;/code&gt; dies on BigInt, so I stringified coordinates by hand, and my first pass reported the vertical case as "m=undefined" without proving anything — the useful result was the thrown error, which only shows up if you actually call &lt;code&gt;.eq()&lt;/code&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  The ten questions, in full
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;#&lt;/th&gt;
&lt;th&gt;From&lt;/th&gt;
&lt;th&gt;To&lt;/th&gt;
&lt;th&gt;Rise&lt;/th&gt;
&lt;th&gt;Run&lt;/th&gt;
&lt;th&gt;Slope&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;(1, 2)&lt;/td&gt;
&lt;td&gt;(5, 4)&lt;/td&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;td&gt;4&lt;/td&gt;
&lt;td&gt;1/2&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;td&gt;(0, 5)&lt;/td&gt;
&lt;td&gt;(2, 1)&lt;/td&gt;
&lt;td&gt;−4&lt;/td&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;td&gt;−2&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;3&lt;/td&gt;
&lt;td&gt;(−2, 3)&lt;/td&gt;
&lt;td&gt;(4, 3)&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;6&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;4&lt;/td&gt;
&lt;td&gt;(−1, −2)&lt;/td&gt;
&lt;td&gt;(3, 6)&lt;/td&gt;
&lt;td&gt;8&lt;/td&gt;
&lt;td&gt;4&lt;/td&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;5&lt;/td&gt;
&lt;td&gt;(2, 1)&lt;/td&gt;
&lt;td&gt;(5, 2)&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;3&lt;/td&gt;
&lt;td&gt;1/3&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;6&lt;/td&gt;
&lt;td&gt;(0, 0)&lt;/td&gt;
&lt;td&gt;(4, 3)&lt;/td&gt;
&lt;td&gt;3&lt;/td&gt;
&lt;td&gt;4&lt;/td&gt;
&lt;td&gt;3/4&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;7&lt;/td&gt;
&lt;td&gt;(−3, 4)&lt;/td&gt;
&lt;td&gt;(1, 0)&lt;/td&gt;
&lt;td&gt;−4&lt;/td&gt;
&lt;td&gt;4&lt;/td&gt;
&lt;td&gt;−1&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;8&lt;/td&gt;
&lt;td&gt;(1, −2)&lt;/td&gt;
&lt;td&gt;(5, −2)&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;4&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;9&lt;/td&gt;
&lt;td&gt;(−2, −3)&lt;/td&gt;
&lt;td&gt;(2, 5)&lt;/td&gt;
&lt;td&gt;8&lt;/td&gt;
&lt;td&gt;4&lt;/td&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10&lt;/td&gt;
&lt;td&gt;(1, 4)&lt;/td&gt;
&lt;td&gt;(7, 5)&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;6&lt;/td&gt;
&lt;td&gt;1/6&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Ten questions. Eight distinct answers.&lt;/p&gt;

&lt;h2&gt;
  
  
  Three things this set gets wrong
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Not one of the ten has the same x on both points.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;That means the set never asks for an undefined slope. Zero out of ten.&lt;/p&gt;

&lt;p&gt;Our blog has 43 articles. Three of them are specifically about this: "When is a slope considered undefined?", "Undefined slope vs zero slope", and "When is a slope undefined? Two equal x-values". We teach the case in three places and refuse to test it in the only place a reader can act.&lt;/p&gt;

&lt;p&gt;Here is why, and this is the part that matters. Feed a vertical pair into the page's own function and it returns &lt;code&gt;mode: 'vertical'&lt;/code&gt; with no &lt;code&gt;m&lt;/code&gt; at all. The render line then builds the explanation string by hand:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Slope = (6) / (0) = undefined
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;And the answer check calls &lt;code&gt;parse(input).eq(answer)&lt;/code&gt; where &lt;code&gt;answer&lt;/code&gt; is &lt;code&gt;undefined&lt;/code&gt;. That throws. Not "marks wrong" — throws. I ran it: &lt;code&gt;Cannot read properties of undefined (reading 'n')&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;So the practice cannot ask a question it cannot grade, and the grader was never taught the one case the articles promise. The bank is not missing a hard question by accident. It is missing the question that would crash.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Two of the ten have the same answer as each other, twice over.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Answer 0 shows up at question 3 and again at question 8. Answer 2 shows up at question 4 and again at question 9. Different points, different lines, so nothing is technically repeated — but eight distinct answers spread over ten questions means two answers cover 20% of the bank each. A student who has not understood the sign rule yet can hit two right by guessing the round numbers.&lt;/p&gt;

&lt;p&gt;And the set cycles between exactly two versions. The restart handler does &lt;code&gt;set = (set + 1) % 2&lt;/code&gt;. Ten questions, two sets of five, then back to the start. Anyone who practises twice has seen everything, forever.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The public JavaScript ships our build notes in Chinese.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;code&gt;app.mjs&lt;/code&gt;: 589 lines, 90 of them contain Chinese characters, 82 of those are comments. &lt;code&gt;math.mjs&lt;/code&gt;: 265 lines, 61 with Chinese, 54 comments. That is 151 comment lines going out to every visitor of an English-language site.&lt;/p&gt;

&lt;p&gt;One of them, dated 2026-11-10, explains a bug we hit in production:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;判断「这页有练习题」不能只看 #practice —— 文章正文里的 &lt;code&gt;&amp;lt;h2 id="practice"&amp;gt;&lt;/code&gt; 会命中同一个 selector…&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Roughly: don't detect "this page has practice questions" by looking for &lt;code&gt;#practice&lt;/code&gt;, because a heading in an article body matches the same selector and then the code crashes on a null form. Someone spent an evening on that. It is now public, in Chinese, on a site for American students.&lt;/p&gt;

&lt;p&gt;The site is English. Our notes are not. We minify the CSS and version it with a hash, and then ship the reasoning behind every function in a language our readers do not read.&lt;/p&gt;

&lt;h2&gt;
  
  
  One thing I wrote wrong this morning
&lt;/h2&gt;

&lt;p&gt;Before I opened &lt;code&gt;app.mjs&lt;/code&gt;, I wrote in our own notes that the practice page had 173 words and was the thinnest page on the site.&lt;/p&gt;

&lt;p&gt;173 is what the HTML ships before the module runs. The page a reader actually sees has five questions, a hint button, a show-answer button and a score. The page is not thin. It has ten questions in total and cannot say so out loud. That is worse than thin, and I only found it because the first number I wrote down was wrong.&lt;/p&gt;

&lt;h2&gt;
  
  
  What this does not tell you
&lt;/h2&gt;

&lt;p&gt;Whether ten questions is too few.&lt;/p&gt;

&lt;p&gt;I have no completion data. The page fires an analytics event when someone asks for a hint and when someone gets one wrong. It does not fire anything when someone finishes, and it does not record whether a restart was a second attempt or a fifth. So I can count the bank and I can count the articles. I cannot tell you whether anyone is bored by question five, because we never measured that.&lt;/p&gt;

&lt;p&gt;I also tested eight input formats for one answer. That is not proof that the grader accepts every reasonable way to type a slope. It is proof that it accepts eight of them, including &lt;code&gt;2/4&lt;/code&gt; for &lt;code&gt;1/2&lt;/code&gt; and &lt;code&gt;.5&lt;/code&gt; for &lt;code&gt;0.5&lt;/code&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Disclosure
&lt;/h2&gt;

&lt;p&gt;I maintain &lt;a href="https://slopecalculatoronline.com/" rel="noopener noreferrer"&gt;Slope Calculator&lt;/a&gt;, which owns both files this post is about. The Chinese comments are mine, the 99.4%-identical pages on the other site are mine, and the checker that throws on &lt;code&gt;undefined&lt;/code&gt; is mine. Publishing the audit is cheaper than pretending the numbers are not there.&lt;/p&gt;

&lt;p&gt;Three articles about undefined slope, and the practice set will not touch it. Fixing that means teaching the grader what "undefined" is, which means changing the answer type, which means the next version of this page is not a content fix.&lt;/p&gt;

&lt;p&gt;That is the order it has to happen in. I got it backwards for three weeks.&lt;/p&gt;

</description>
      <category>bug</category>
      <category>debugging</category>
      <category>javascript</category>
      <category>testing</category>
    </item>
    <item>
      <title>I Generated 76 OBD-II Pages. Two of Them Are the Same Page.</title>
      <dc:creator>fei mai</dc:creator>
      <pubDate>Sun, 15 Nov 2026 08:11:11 +0000</pubDate>
      <link>https://dev.to/mediadownloaderguides/i-generated-76-obd-ii-pages-two-of-them-are-the-same-page-21dp</link>
      <guid>https://dev.to/mediadownloaderguides/i-generated-76-obd-ii-pages-two-of-them-are-the-same-page-21dp</guid>
      <description>&lt;p&gt;Small car-specs site, generated pages, one per diagnostic trouble code. The build script decides what goes in. Nobody reads the output.&lt;/p&gt;

&lt;p&gt;So I read it. All 76 pages, fetched like a stranger would, and the thing that fell out is not a coverage gap — it is that P0303 and P0304 are P0302 with the digit swapped. 99.4% character similarity on the body text. Eight difference blocks. Every block is the numeral.&lt;/p&gt;

&lt;p&gt;That is what happens when a template owns the writing and a &lt;code&gt;for&lt;/code&gt; loop owns the variation.&lt;/p&gt;

&lt;h2&gt;
  
  
  How we measured
&lt;/h2&gt;

&lt;p&gt;Pulled &lt;code&gt;sitemap-obd.xml&lt;/code&gt;, 76 URLs. Fetched with Python 3.13, &lt;code&gt;urllib&lt;/code&gt;, single thread, 300ms apart, three attempts each before declaring a failure. All 76 returned 200.&lt;/p&gt;

&lt;p&gt;Per page: strip script, style, nav, footer; count words; regex the labelled fields out of the HTML (system, standard, severity, symptom count, cause count, diagnosis length, sibling list).&lt;/p&gt;

&lt;p&gt;For the duplicate check I used two measures on purpose. &lt;code&gt;difflib&lt;/code&gt; character similarity on the body text between the system label and the sibling list — that boundary matters, because the sibling list changes on every page and would drag the score down for a reason that is not duplication. And Jaccard on 6-word shingles as a cross-check.&lt;/p&gt;

&lt;p&gt;For the record: my first pass compared whole pages and reported 60–83% similarity. That number was contaminated by the sibling lists. Restricted to the body, the true figure for P0303 is 99.4%.&lt;/p&gt;

&lt;h2&gt;
  
  
  What is in there
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Prefix&lt;/th&gt;
&lt;th&gt;Codes&lt;/th&gt;
&lt;th&gt;What the prefix covers&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;P&lt;/td&gt;
&lt;td&gt;70&lt;/td&gt;
&lt;td&gt;Powertrain&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;U&lt;/td&gt;
&lt;td&gt;6&lt;/td&gt;
&lt;td&gt;Network / communication&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;B&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;Body (airbags, restraints, comfort)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;C&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;Chassis (ABS, steering, suspension)&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;System&lt;/th&gt;
&lt;th&gt;Codes&lt;/th&gt;
&lt;th&gt;Number range&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Fuel &amp;amp; Air Metering&lt;/td&gt;
&lt;td&gt;33&lt;/td&gt;
&lt;td&gt;P0100–P0299&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Ignition &amp;amp; Misfire&lt;/td&gt;
&lt;td&gt;12&lt;/td&gt;
&lt;td&gt;P0300–P0352&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Emissions Control&lt;/td&gt;
&lt;td&gt;9&lt;/td&gt;
&lt;td&gt;P0401–P0456&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Transmission&lt;/td&gt;
&lt;td&gt;8&lt;/td&gt;
&lt;td&gt;P0700–P0755&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Network&lt;/td&gt;
&lt;td&gt;6&lt;/td&gt;
&lt;td&gt;U0001–U0155&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Electrical&lt;/td&gt;
&lt;td&gt;5&lt;/td&gt;
&lt;td&gt;P0562–P0606&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Speed &amp;amp; Idle Control&lt;/td&gt;
&lt;td&gt;3&lt;/td&gt;
&lt;td&gt;P0500–P0507&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Severity label&lt;/th&gt;
&lt;th&gt;Codes&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Repair soon&lt;/td&gt;
&lt;td&gt;45&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Low urgency&lt;/td&gt;
&lt;td&gt;22&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Do not keep driving&lt;/td&gt;
&lt;td&gt;9&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Measure&lt;/th&gt;
&lt;th&gt;Value&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Words per page, min&lt;/td&gt;
&lt;td&gt;382&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Median&lt;/td&gt;
&lt;td&gt;426&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Max&lt;/td&gt;
&lt;td&gt;478&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Standard deviation&lt;/td&gt;
&lt;td&gt;17.1&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Pages listing every peer in their own system&lt;/td&gt;
&lt;td&gt;14 of 76&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Pages listing exactly 6 peers&lt;/td&gt;
&lt;td&gt;62 of 76&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h2&gt;
  
  
  Three things I did not expect
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;The misfire run stops at cylinder six.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;P0301 through P0306 exist. All six carry the "Do not keep driving" label, which is the right call — an unburnt-fuel misfire will cook a catalytic converter.&lt;/p&gt;

&lt;p&gt;P0307 and P0308 return 404. I retried both twice.&lt;/p&gt;

&lt;p&gt;So we tell the owner of a V6 that a misfire means stop driving, and we tell the owner of a V8 nothing at all. The most common vehicles on American roads are V8 trucks. The coverage stops exactly where the biggest engines begin.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Two of those pages are literally the same page with the digit changed.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;I compared the body text only — everything between the system label and the sibling list — so the "other codes" block could not inflate the score.&lt;/p&gt;

&lt;p&gt;P0303 against P0302: 99.4% character similarity. Eight difference blocks. Every one of the eight is the numeral &lt;code&gt;2&lt;/code&gt; replaced by a &lt;code&gt;3&lt;/code&gt;. Nothing else.&lt;/p&gt;

&lt;p&gt;P0304 against P0302: 99.4%. Eight blocks. Every one is &lt;code&gt;2&lt;/code&gt; replaced by &lt;code&gt;4&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;That is not two pages about two different faults. That is one page, twice, with the cylinder number stamped in. Same cause list, same diagnosis paragraph, same FAQ. Google compares text, not intent.&lt;/p&gt;

&lt;p&gt;P0305 and P0306 are the honest ones, and the contrast is the useful part. 81% similarity, not 99%. The sentences were actually rewritten — different cause ordering, different phrasing in the diagnosis, plus one detail worth keeping: cylinder numbering differs between V6 and V8 engines, so confirm which cylinder you are actually chasing. Whoever wrote those two did the work. Whoever generated 3 and 4 did not.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Navigation gives up at six, no matter how many peers exist.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;62 of the 76 pages render exactly six "other codes in this system" links. Fuel &amp;amp; Air holds 33 codes and shows six of them. Ignition holds 12 and shows six. Six is a &lt;code&gt;slice(0, 6)&lt;/code&gt; in the template, not a decision about how many neighbours a reader can use — and nothing on the page says the list is partial.&lt;/p&gt;

&lt;p&gt;Six is not a design decision about how many links a reader wants. It is a &lt;code&gt;slice(0, 6)&lt;/code&gt;. The result: 62 pages cannot reach two-thirds of their own neighbours, and no page in the site tells you which six you are missing.&lt;/p&gt;

&lt;p&gt;And every page is the same length. 382 to 478 words, standard deviation 17, whether the page sits in a system with 3 codes or 33. The template decides the length, not the fault.&lt;/p&gt;

&lt;h2&gt;
  
  
  One thing we wrote wrong
&lt;/h2&gt;

&lt;p&gt;Every one of the 76 pages carries the site footer: "All figures sourced from NHTSA and the U.S. Department of Energy (fueleconomy.gov)."&lt;/p&gt;

&lt;p&gt;That sentence is true for vehicle specs. It is false on every OBD page. Code numbers, system names and definitions on these pages come from the SAE J2012 / ISO 15031-6 generic code standard — which the sourcing box on the same page says, three inches above the footer. Two boxes, one page, contradicting each other, 76 times.&lt;/p&gt;

&lt;p&gt;We are fixing the footer template this week. The OBD pages are not NHTSA data and should never have said so.&lt;/p&gt;

&lt;h2&gt;
  
  
  What this measurement cannot tell you
&lt;/h2&gt;

&lt;p&gt;Which of the missing codes anyone actually looks up.&lt;/p&gt;

&lt;p&gt;I have no keyword data for this site and I did not buy any for a blog post. There are 167 empty numbering slots in the Fuel &amp;amp; Air block and most of them probably correspond to nothing anyone has ever typed into a search box. The honest statement is: P0307 and P0308 are gaps I can prove matter, because a reader asked about one. The other 160-odd gaps I can only count, not rank.&lt;/p&gt;

&lt;p&gt;I also did not test any of this against a live vehicle. The symptom and cause lists are written from the generic definition, not from a scan of a real car. If your P0171 turned out to be a vacuum leak and our page said intake gasket, that is the gap between a generic code and your specific engine, and no amount of page-building closes it.&lt;/p&gt;

&lt;h2&gt;
  
  
  Disclosure
&lt;/h2&gt;

&lt;p&gt;I build &lt;a href="https://carspecdata.com/" rel="noopener noreferrer"&gt;CarSpecData&lt;/a&gt;, which is the site doing the publishing here. Every number above came out of my own pages and my own templates. The duplicate pair is mine, the footer error is mine, and the &lt;code&gt;slice(0, 6)&lt;/code&gt; is mine.&lt;/p&gt;

&lt;p&gt;If you own a V8 and you are chasing a misfire right now, we do not have your page yet. That is the state of it. The template fix goes in first; the missing cylinders go in after.&lt;/p&gt;

</description>
      <category>automation</category>
      <category>programming</category>
      <category>python</category>
    </item>
    <item>
      <title>My Blog Index Is the Only Thing Linking to Fifteen of My Posts</title>
      <dc:creator>fei mai</dc:creator>
      <pubDate>Sat, 14 Nov 2026 08:14:01 +0000</pubDate>
      <link>https://dev.to/mediadownloaderguides/my-blog-index-is-the-only-thing-linking-to-fifteen-of-my-posts-1c11</link>
      <guid>https://dev.to/mediadownloaderguides/my-blog-index-is-the-only-thing-linking-to-fifteen-of-my-posts-1c11</guid>
      <description>&lt;p&gt;Fifteen articles on my own site have exactly one internal link pointing at them. The blog index. Nothing else on the internet I control points at those pages.&lt;/p&gt;

&lt;p&gt;I found out by building the link graph instead of trusting the sitemap.&lt;/p&gt;

&lt;p&gt;The site is a proportion calculator with a content section: 56 URLs in the sitemap, 40 of them blog posts. I fetched all 56 with curl through a local proxy — every one returned 200 first pass, no retries needed — then parsed 1,617 &lt;code&gt;&amp;lt;a&amp;gt;&lt;/code&gt; tags with a regex, resolved relative hrefs to absolute, dropped the external hosts, deduped repeats inside each page, and built the in/out table in Python. Server-rendered HTML, so what I saved is the page.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Measure&lt;/th&gt;
&lt;th&gt;Value&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;URLs in sitemap&lt;/td&gt;
&lt;td&gt;56, all HTTP 200&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Canonical equals own URL&lt;/td&gt;
&lt;td&gt;56 of 56&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;code&gt;&amp;lt;a&amp;gt;&lt;/code&gt; tags parsed&lt;/td&gt;
&lt;td&gt;1,617&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Unique internal links after dedupe&lt;/td&gt;
&lt;td&gt;944&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Blog posts among those 56 pages&lt;/td&gt;
&lt;td&gt;40&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Internal links pointing at any blog post&lt;/td&gt;
&lt;td&gt;220&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Of those, held by the 3 newest posts&lt;/td&gt;
&lt;td&gt;129 (58.6%)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Posts reachable by exactly one link&lt;/td&gt;
&lt;td&gt;15&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Posts with two links or fewer&lt;/td&gt;
&lt;td&gt;22&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Median links per post&lt;/td&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Links leaving a post that go to nav or legal pages&lt;/td&gt;
&lt;td&gt;360 of 700&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Correlation, word count vs inbound links&lt;/td&gt;
&lt;td&gt;r = −0.10&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Clicks from home to any post&lt;/td&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;People assume a link audit finds broken links. It did not. Every URL resolved, every canonical pointed at itself, and the site is technically clean. The problem is arithmetic, not plumbing.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The "latest three" block is doing almost all the work.&lt;/strong&gt; Forty of the 56 pages render a block with the three newest articles, and each block links all three. That is 129 links aimed at 3 URLs — 58.6% of all internal links into blog content, and 13.7% of every internal link on the domain. Remove that block and those three pages fall back to near-zero. It is not a related-posts system. It is a clock.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Length buys nothing.&lt;/strong&gt; Pearson r between word count and inbound links is −0.10, noise leaning slightly negative. The site's longest article, 1,969 words on ratios as decimals, holds 2 links. A 1,239-word grade-6 examples page holds 44. If you write longer hoping the site notices, the site does not notice.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Half of every article's outbound links point at furniture.&lt;/strong&gt; The 40 posts emit 700 unique internal links; 360 go to the homepage, /learn/, /blog/, /practice/ and five legal pages. Only 340 point at other content, roughly 8.5 per post, and most of those land on the same three pages as everything else.&lt;/p&gt;

&lt;p&gt;Something I did not go hunting for: 46 anchors begin with a literal date string, "2026-11-14 ↗ How to find x in a ratio". Anchor text that mutates daily is a timestamp, not an anchor.&lt;/p&gt;

&lt;p&gt;What I cannot measure: whether that block builds at deploy time or renders per request. I fetched each page once, on one afternoon, and they all agreed. If it rotates on a schedule, tomorrow's crawl gives a different distribution and my table is one snapshot of one Thursday. Ranking flow I cannot measure at all — that graph belongs to Google.&lt;/p&gt;

&lt;p&gt;Correction, because the first version of this audit was wrong in a way worth publishing. My initial crawl reported 41 orphan pages with zero inbound links. It was a parser bug. I had stripped &lt;code&gt;/blog/&lt;/code&gt; out of every path while extracting filenames, so the graph asked whether &lt;code&gt;/blog/how-to-cross-multiply/&lt;/code&gt; linked to &lt;code&gt;/how-to-cross-multiply/&lt;/code&gt; — which returns 404. Forty real pages, forty URLs that do not exist, forty phantom orphans. I rebuilt the parser to key off sitemap order, re-ran it, and orphan count went to zero. What survived: fifteen posts with a single link, median two.&lt;/p&gt;

&lt;p&gt;Disclosure: this is my site, &lt;a href="https://proportioncalculatoronline.com/" rel="noopener noreferrer"&gt;proportioncalculatoronline.com&lt;/a&gt;, so I have the sitemap and the templates. The fix is boring — replace the date block with three hand-picked related posts per article, and add two links from the relevant guides into the fifteen single-link posts. About thirty manual links. Not done yet.&lt;/p&gt;

&lt;p&gt;The three pages holding 59% of the links are dated 13 and 14 November. Next week they will be three different pages. The fifteen sitting on one link will still be sitting on one link.&lt;/p&gt;

</description>
      <category>data</category>
      <category>python</category>
      <category>seo</category>
      <category>webdev</category>
    </item>
    <item>
      <title>I Read My Own Voting Site's API and Found 46 Votes, No Verdicts</title>
      <dc:creator>fei mai</dc:creator>
      <pubDate>Sat, 14 Nov 2026 08:13:35 +0000</pubDate>
      <link>https://dev.to/mediadownloaderguides/i-read-my-own-voting-sites-api-and-found-46-votes-no-verdicts-3cki</link>
      <guid>https://dev.to/mediadownloaderguides/i-read-my-own-voting-sites-api-and-found-46-votes-no-verdicts-3cki</guid>
      <description>&lt;p&gt;A voting site where nothing ever gets decided is a bug, not a feature. I found that out by reading my own product's JSON endpoint instead of looking at it.&lt;/p&gt;

&lt;p&gt;The first case on WalletJury went up on 15 October. A $24 oversized wool-blend coat. Then nothing for sixteen hours, and at 08:16 the next morning ten cases arrived inside a single minute.&lt;/p&gt;

&lt;p&gt;I did not spot that by watching the feed. I spotted it by dumping the table and sorting on &lt;code&gt;createdAt&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Here is the setup. WalletJury is mine. Users post a purchase they are about to make, everyone else votes Guilty or Not Guilty. The rendered page is 98 KB of JavaScript around 169 characters of visible text, so scraping the HTML gets you a currency selector and nothing else. The real data lives at &lt;code&gt;/api/list?sort=new&amp;amp;limit=200&lt;/code&gt;, which returns all 39 cases in one GET with &lt;code&gt;next&lt;/code&gt; set to &lt;code&gt;null&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;So on 14 November I fetched that endpoint through a local proxy, saved the payload, and did the counting in Python. No sampling. The sample is the database.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;What I counted&lt;/th&gt;
&lt;th&gt;Result&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Cases in the database&lt;/td&gt;
&lt;td&gt;39&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;First / last case&lt;/td&gt;
&lt;td&gt;15 Oct 16:42 / 23 Oct 21:08&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Days since a new case&lt;/td&gt;
&lt;td&gt;21&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Votes, guilty / not guilty&lt;/td&gt;
&lt;td&gt;33 / 13&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Cases with no votes at all&lt;/td&gt;
&lt;td&gt;22 of 39&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Most votes on any single case&lt;/td&gt;
&lt;td&gt;3&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Cases with a recorded outcome&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Cases posted anonymously&lt;/td&gt;
&lt;td&gt;39 of 39&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Cases with an image&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Model pre-verdict: guilty&lt;/td&gt;
&lt;td&gt;36 of 39&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Jury result across the 17 voted cases&lt;/td&gt;
&lt;td&gt;16 guilty, 1 tie, 0 not guilty&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Three findings, in the order they annoyed me.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;One. &lt;code&gt;outcome&lt;/code&gt; is null on every single row.&lt;/strong&gt; Thirty-nine cases, seventeen of them carrying a completed 3-0 or 2-1 tally, and not one recorded verdict. Either a closing threshold exists and nobody reached it, or the closing step shipped as a TODO. I can see the column is empty. I cannot see the server, so I will not pretend to know which.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Two. The crowd has never acquitted.&lt;/strong&gt; Sixteen guilty, one tie, zero not-guilty. Strong-looking signal until you check the model's pre-verdict: guilty on 36 of 39, and guilty on all 17 cases that later drew votes. A classifier that answers "guilty" nine times out of ten will appear to agree with humans at almost any sample size. The agreement is not evidence of judgement. It is evidence of a prior.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Three. None of the voters have accounts here.&lt;/strong&gt; The &lt;code&gt;replies&lt;/code&gt; field totals 51, and all 51 belong to the 17 voted cases, exactly three apiece. Every one of those cases carries a &lt;code&gt;last&lt;/code&gt; reply object tagged with a &lt;code&gt;source&lt;/code&gt;: facebook 7, reddit 6, instagram 3, x 1. Thirteen distinct handles across seventeen replies. Zero attributed to a WalletJury user.&lt;/p&gt;

&lt;p&gt;And a detail I keep thinking about. In 12 of those 17 cases, the most recent reply argues not-guilty while the counter reads guilty. The visitor's last impression is the losing argument.&lt;/p&gt;

&lt;p&gt;What I deliberately did not do: guess whether three is a hard cap. The API returns two integers, &lt;code&gt;g&lt;/code&gt; and &lt;code&gt;a&lt;/code&gt;, with no vote timestamps and no voter IDs. A cap of three and three coincidental visitors are indistinguishable from the outside. Writing that up as a finding would have meant inventing the part I could not see.&lt;/p&gt;

&lt;p&gt;I also skipped the question of why posting stopped. Three cases a night at 21:0x from 17 to 23 October, then silence. I did not read the cron before publishing, so I am not going to narrate a cause.&lt;/p&gt;

&lt;p&gt;A correction while I am at it. On 10 November I described this site as having "39 cases and 46 votes, and it is growing." It has not grown. Same 39, same 46, four days later. I looked at the total and skipped the timestamps, which is the same shortcut that made me wrong about a crawler last week.&lt;/p&gt;

&lt;p&gt;Disclosure: WalletJury is one of my sites, which is the only reason I can hit the raw endpoint. Front end is at &lt;a href="https://walletjury.com/" rel="noopener noreferrer"&gt;walletjury.com&lt;/a&gt;. I did not vote, post, or reply on anything during measurement — stuffing the ballot would have made the only number that matters meaningless.&lt;/p&gt;

&lt;p&gt;The most recent case is a $99.99 Minecraft creeper mini fridge. Zero votes, zero comments, no verdict, twenty-one days old.&lt;/p&gt;

</description>
      <category>api</category>
      <category>buildinpublic</category>
      <category>product</category>
      <category>startup</category>
    </item>
    <item>
      <title>The Page That Takes Payments Is the Only Page Without a Navigation</title>
      <dc:creator>fei mai</dc:creator>
      <pubDate>Fri, 13 Nov 2026 08:09:05 +0000</pubDate>
      <link>https://dev.to/mediadownloaderguides/the-page-that-takes-payments-is-the-only-page-without-a-navigation-1pj3</link>
      <guid>https://dev.to/mediadownloaderguides/the-page-that-takes-payments-is-the-only-page-without-a-navigation-1pj3</guid>
      <description>&lt;p&gt;Fourteen URLs on my job site. Thirteen of them have 9 to 18 links to other pages in the served HTML.&lt;/p&gt;

&lt;p&gt;The fourteenth has zero. It is /pricing.&lt;/p&gt;

&lt;p&gt;Not a broken template — the browser shows the nav, because the nav is React and it hydrates. But the document that comes off the wire contains no route out of the page at all, and six words of body text.&lt;/p&gt;

&lt;h2&gt;
  
  
  The measurement
&lt;/h2&gt;

&lt;p&gt;I pulled all 14 sitemap URLs with curl, then counted three things per response: words of visible text after stripping script and style content, hrefs pointing to other pages on the same host, and &lt;code&gt;Question&lt;/code&gt; nodes in JSON-LD versus the same text appearing in the delivered HTML.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Page&lt;/th&gt;
&lt;th&gt;Words&lt;/th&gt;
&lt;th&gt;Links&lt;/th&gt;
&lt;th&gt;FAQ Qs&lt;/th&gt;
&lt;th&gt;Qs in HTML&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;/guide/limits/&lt;/td&gt;
&lt;td&gt;2,879&lt;/td&gt;
&lt;td&gt;15&lt;/td&gt;
&lt;td&gt;12&lt;/td&gt;
&lt;td&gt;12&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;/guide/how-long-do-job-postings-stay-up/&lt;/td&gt;
&lt;td&gt;1,806&lt;/td&gt;
&lt;td&gt;15&lt;/td&gt;
&lt;td&gt;5&lt;/td&gt;
&lt;td&gt;5&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;/guide/recently-posted-jobs/&lt;/td&gt;
&lt;td&gt;1,544&lt;/td&gt;
&lt;td&gt;15&lt;/td&gt;
&lt;td&gt;5&lt;/td&gt;
&lt;td&gt;5&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;/guide/new-grad-jobs-2027/&lt;/td&gt;
&lt;td&gt;1,504&lt;/td&gt;
&lt;td&gt;15&lt;/td&gt;
&lt;td&gt;5&lt;/td&gt;
&lt;td&gt;5&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;/terms&lt;/td&gt;
&lt;td&gt;1,494&lt;/td&gt;
&lt;td&gt;4&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;/privacy&lt;/td&gt;
&lt;td&gt;1,456&lt;/td&gt;
&lt;td&gt;4&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;/guide/how-many-jobs-should-i-apply-to/&lt;/td&gt;
&lt;td&gt;1,455&lt;/td&gt;
&lt;td&gt;14&lt;/td&gt;
&lt;td&gt;5&lt;/td&gt;
&lt;td&gt;5&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;/guide/h1b-sponsorship-jobs/&lt;/td&gt;
&lt;td&gt;1,359&lt;/td&gt;
&lt;td&gt;14&lt;/td&gt;
&lt;td&gt;6&lt;/td&gt;
&lt;td&gt;6&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;/guide/best-job-aggregator-sites/&lt;/td&gt;
&lt;td&gt;1,057&lt;/td&gt;
&lt;td&gt;15&lt;/td&gt;
&lt;td&gt;5&lt;/td&gt;
&lt;td&gt;5&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;/about&lt;/td&gt;
&lt;td&gt;457&lt;/td&gt;
&lt;td&gt;9&lt;/td&gt;
&lt;td&gt;3&lt;/td&gt;
&lt;td&gt;3&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;/ (home)&lt;/td&gt;
&lt;td&gt;433&lt;/td&gt;
&lt;td&gt;10&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;/contact&lt;/td&gt;
&lt;td&gt;230&lt;/td&gt;
&lt;td&gt;5&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;/guide/&lt;/td&gt;
&lt;td&gt;174&lt;/td&gt;
&lt;td&gt;18&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;/pricing&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;7&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;0&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;8&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;0&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;All 14 answered 200. No retries needed.&lt;/p&gt;

&lt;h2&gt;
  
  
  What is on the page instead
&lt;/h2&gt;

&lt;p&gt;Three price tiers, and they are written better than anything else on the site:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;30 AI fit analyses, $4.99&lt;/li&gt;
&lt;li&gt;80, $9.99&lt;/li&gt;
&lt;li&gt;200, $19.99&lt;/li&gt;
&lt;li&gt;browsing, filtering, saving jobs and opening the employer's own posting stay free&lt;/li&gt;
&lt;li&gt;points never expire&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;That copy lives in the meta description, in &lt;code&gt;og:description&lt;/code&gt;, and in a React Server Components payload — one &lt;code&gt;self.__next_f.push&lt;/code&gt; call — that paints after hydration.&lt;/p&gt;

&lt;p&gt;It does not live in the body. And there is no &lt;code&gt;Product&lt;/code&gt; schema, no &lt;code&gt;Offer&lt;/code&gt;, no &lt;code&gt;price&lt;/code&gt; property. The only JSON-LD block on the page is an &lt;code&gt;FAQPage&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Eight FAQ entries: where the jobs come from, whether you need an account to buy points, what one point buys, whether refunds exist. Average answer 36 words. None of the eight question strings appears anywhere in the HTML a client receives without running JS.&lt;/p&gt;

&lt;p&gt;Across all 14 pages there are 54 &lt;code&gt;Question&lt;/code&gt; nodes and 46 of them are backed by real visible text. All 8 orphans sit on this one page.&lt;/p&gt;

&lt;h2&gt;
  
  
  Three conclusions, one of them uncomfortable
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;The page was never broken for humans.&lt;/strong&gt; It renders correctly. If I had only ever looked at it in a browser I would have found nothing — and for eight days I didn't.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;A page with zero outbound internal links is a sink.&lt;/strong&gt; Whatever authority reaches /pricing stops there. It cannot pass anything to the guides, and a visitor who lands from a search result for "job site pricing" has no path into the product except the back button.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Structured data is claiming content the document does not contain.&lt;/strong&gt; Not maliciously, and not cloaking — the text really is on the page once JS runs. But FAQ markup on an FAQ that isn't in the response is markup a validator will flag and a rendering crawler may or may not reconcile.&lt;/p&gt;

&lt;h2&gt;
  
  
  Limits of this audit
&lt;/h2&gt;

&lt;p&gt;I measured the HTTP response, not Googlebot's rendered DOM. Google executes JavaScript on a deferred budget; whether it ends up seeing those eight answers is something I cannot prove without Search Console, which I did not open for this post.&lt;/p&gt;

&lt;p&gt;Same for Bing and every other engine. No webmaster account, no claim.&lt;/p&gt;

&lt;p&gt;And no conversion data. Whether a dead-end pricing page costs revenue is a question for analytics, not for curl.&lt;/p&gt;

&lt;h2&gt;
  
  
  Disclosure
&lt;/h2&gt;

&lt;p&gt;JobRadarly is one of mine — &lt;a href="https://jobradarly.com/" rel="noopener noreferrer"&gt;jobradarly.com&lt;/a&gt;. It filters job postings against your role, city and salary floor and shows you the handful that clear the bar instead of a feed. The numbers here come from my own server.&lt;/p&gt;

&lt;p&gt;Correction to an earlier internal note: it said this page returns four words server-side. It returns six, plus an em dash. The number moved. The finding did not.&lt;/p&gt;

</description>
      <category>frontend</category>
      <category>react</category>
      <category>softwaredevelopment</category>
      <category>webdev</category>
    </item>
    <item>
      <title>I Counted 34,500 Flashcard Answers. 22% of Them Start With the Same Two Words.</title>
      <dc:creator>fei mai</dc:creator>
      <pubDate>Fri, 13 Nov 2026 08:08:51 +0000</pubDate>
      <link>https://dev.to/mediadownloaderguides/i-counted-34500-flashcard-answers-22-of-them-start-with-the-same-two-words-2akf</link>
      <guid>https://dev.to/mediadownloaderguides/i-counted-34500-flashcard-answers-22-of-them-start-with-the-same-two-words-2akf</guid>
      <description>&lt;p&gt;Two words: "The author".&lt;/p&gt;

&lt;p&gt;They open 5,642 of the answers on my flashcard site. Add "The chapter" and you are at 22.5% of 34,500.&lt;/p&gt;

&lt;p&gt;I found that by parsing my own database, which sounds more impressive than it was. The hard part was admitting I had shipped 156 decks without ever reading them end to end.&lt;/p&gt;

&lt;h2&gt;
  
  
  The parse
&lt;/h2&gt;

&lt;p&gt;Each deck page ships its whole card array inline as &lt;code&gt;window.BC_CARDS&lt;/code&gt; — the viewer needs to flip through cards without hitting the server. So the data was already public. I pulled 157 sitemap URLs with curl, matched one regex per file, and loaded each array with &lt;code&gt;json.loads&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;156 decks, 34,500 cards, no malformed payloads.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Measure&lt;/th&gt;
&lt;th&gt;Value&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Decks parsed&lt;/td&gt;
&lt;td&gt;156&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Cards&lt;/td&gt;
&lt;td&gt;34,500&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Unique normalized questions&lt;/td&gt;
&lt;td&gt;34,484&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Questions in 2+ decks&lt;/td&gt;
&lt;td&gt;9&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Exact duplicate answers&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Mean answer length&lt;/td&gt;
&lt;td&gt;60.58 words (sd 8.29)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Answers 50–75 words&lt;/td&gt;
&lt;td&gt;88.0%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Answers over 90 words&lt;/td&gt;
&lt;td&gt;50&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Chapters&lt;/td&gt;
&lt;td&gt;9,286&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Chapters with exactly 4 cards&lt;/td&gt;
&lt;td&gt;7,297 (78.6%)&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h2&gt;
  
  
  The duplicate hunt was a bust
&lt;/h2&gt;

&lt;p&gt;I went in expecting mass duplication. Same generator, 156 books, one prompt — surely the same cards came out under different titles.&lt;/p&gt;

&lt;p&gt;Nine questions out of 34,484 appear in more than one deck. Zero answers repeat verbatim.&lt;/p&gt;

&lt;p&gt;An earlier note of mine called duplication "the main structural problem". That note was a guess dressed up as a finding. Consider it withdrawn.&lt;/p&gt;

&lt;h2&gt;
  
  
  What the uniformity actually is
&lt;/h2&gt;

&lt;p&gt;Three things, none of them duplication.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Openers.&lt;/strong&gt; 5,642 answers begin "The author". 2,109 begin "The chapter". A quarter of the database opens with words that carry no information — the question already told the reader whose argument it is. Worse, 132 of 156 decks mix styles: "Mill argues..." on one card, "The author argues..." three cards down, same book, no rule.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Length.&lt;/strong&gt; sd 8.29 around a mean of 60.58. 88% land in a 26-word window between 50 and 75. Fifty cards out of 34,500 get past 90 words; 381 come in under 40. A one-line fact and a four-step argument both arrive at sixty words, which means one is padded and the other is cut.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Chapter caps.&lt;/strong&gt; 7,297 of 9,286 chapters hold exactly four cards. Only 154 hold one. Four is the ceiling and also the target, so most chapters carry a fourth card that exists to fill a slot.&lt;/p&gt;

&lt;p&gt;And the &lt;code&gt;ref&lt;/code&gt; field — the pointer back to the book — is just "Chapter 7" in 99.9% of cards. No page, no paragraph. Useless for finding the passage.&lt;/p&gt;

&lt;h2&gt;
  
  
  Limits
&lt;/h2&gt;

&lt;p&gt;I have not read the 156 source books, so I cannot tell you the answers are correct. I can tell you they are unique and consistently formatted. That is not the same claim.&lt;/p&gt;

&lt;p&gt;Whether any of this costs a student recall is a user question. I did not open user data for this post.&lt;/p&gt;

&lt;h2&gt;
  
  
  Disclosure
&lt;/h2&gt;

&lt;p&gt;WakeCards is mine — &lt;a href="https://wakecards.com/" rel="noopener noreferrer"&gt;wakecards.com&lt;/a&gt;, public-domain titles, five cards free, no account. The numbers above come from my own database and I am publishing them because nobody else was going to.&lt;/p&gt;

&lt;p&gt;The regex took ten minutes. Not looking for three months is the part I would rather not put a number on.&lt;/p&gt;

</description>
      <category>data</category>
      <category>database</category>
      <category>python</category>
      <category>sideprojects</category>
    </item>
    <item>
      <title>67 of 78 Brands on My Appliance Site Have No Manufacturer Link on Their Manual Pages</title>
      <dc:creator>fei mai</dc:creator>
      <pubDate>Thu, 12 Nov 2026 08:20:22 +0000</pubDate>
      <link>https://dev.to/mediadownloaderguides/67-of-78-brands-on-my-appliance-site-have-no-manufacturer-link-on-their-manual-pages-3a96</link>
      <guid>https://dev.to/mediadownloaderguides/67-of-78-brands-on-my-appliance-site-have-no-manufacturer-link-on-their-manual-pages-3a96</guid>
      <description>&lt;p&gt;Nine brands get a real manufacturer link on every manual page. Sixty-seven get none.&lt;/p&gt;

&lt;p&gt;I know the number because I fetched all 1,346 of those pages this morning, and the pattern turned out to be less about research and more about a dictionary key.&lt;/p&gt;

&lt;h2&gt;
  
  
  The page that started it
&lt;/h2&gt;

&lt;p&gt;A Speed Queen dryer. Title: "Speed Queen ADE6H178T Manual — Official." First line of body: &lt;em&gt;We do not have a verified manual link for the Speed Queen ADE6H178T.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;Below that, one outbound link. A U.S. Department of Energy JSON file.&lt;/p&gt;

&lt;h2&gt;
  
  
  Method
&lt;/h2&gt;

&lt;p&gt;&lt;code&gt;sitemap-models.xml&lt;/code&gt; lists 6,800 URLs; each model has manual, problems, parts and specs sub-pages, so 1,346 URLs end in &lt;code&gt;/manual/&lt;/code&gt;. I fetched every one — five curl workers, 30-second timeout, one retry on failure, regex over each page's &lt;code&gt;href&lt;/code&gt; attributes, host recorded.&lt;/p&gt;

&lt;p&gt;1,346 requests, 1,346 200s. Coverage: 78 brands, 586 dryers, 404 washing machines, 356 dishwashers.&lt;/p&gt;

&lt;h2&gt;
  
  
  Results
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Only outbound link points to&lt;/th&gt;
&lt;th&gt;Pages&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Manufacturer support page&lt;/td&gt;
&lt;td&gt;792&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;data.energystar.gov&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;547&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;A PDF on &lt;code&gt;s3.amazonaws.com&lt;/code&gt;
&lt;/td&gt;
&lt;td&gt;7&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;No page links two different external hosts. One target, one decision, per model.&lt;/p&gt;

&lt;p&gt;By brand:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Group&lt;/th&gt;
&lt;th&gt;Brands&lt;/th&gt;
&lt;th&gt;Pages&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Manufacturer linked on all models&lt;/td&gt;
&lt;td&gt;9&lt;/td&gt;
&lt;td&gt;792&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Partly linked (Whirlpool, Maytag)&lt;/td&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;td&gt;159&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Never linked&lt;/td&gt;
&lt;td&gt;67&lt;/td&gt;
&lt;td&gt;546&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h2&gt;
  
  
  What the numbers hide
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;A sub-brand is enough to break it.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;GE: 109 manual pages, 109 manufacturer links. GE Profile: 30 manual pages, 0 links. Same corporation, same support portal, different string in the brand column. There is no rule for &lt;code&gt;ge-profile&lt;/code&gt;, so those pages fell through to the default branch — and the default branch is a paragraph explaining that we could not verify a link.&lt;/p&gt;

&lt;p&gt;Midea has 80 pages with nothing. Summit 47. Beko 25. Asko 23. Miele, Haier, IKEA, Dacor, Viking, ZLINE, Bertazzoni, Fisher &amp;amp; Paykel — zero across the board.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The 7 PDFs explain themselves once you subtract.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Whirlpool: 121 pages, 114 manufacturer links. Maytag: 46 pages, 45 links. Those gaps are exactly 7, and there are exactly 7 PDF links on the whole site, six Whirlpool and one Maytag, all on one S3 bucket. Someone pasted those seven files in by hand. Everything else was generated.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The fallback copy is good, which is the problem.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The Speed Queen page explains that the brand serves manuals behind a lookup rather than at fixed URLs, says where the model number is printed, and describes the manual's contents — 24 to 40 pages, a single cycle chart for that drum size, symptoms instead of error codes.&lt;/p&gt;

&lt;p&gt;It reads like a person wrote it. That is precisely why 547 of them shipped without anyone noticing.&lt;/p&gt;

&lt;p&gt;Adjacent number: 1,344 of 1,346 pages contain "This model has no published error-code list." The Bosch error-code page, for the brand with the most dishwashers here, says "0 codes confirmed" and renders two empty header rows.&lt;/p&gt;

&lt;h2&gt;
  
  
  Where this stops
&lt;/h2&gt;

&lt;p&gt;I did not test whether the 792 manufacturer links land on the right document. They are lookup pages driven by JavaScript, occasionally captcha'd, occasionally geo-blocked. That test is doable and I did not do it.&lt;/p&gt;

&lt;p&gt;I did not open the 7 PDFs to check the model inside the file.&lt;/p&gt;

&lt;p&gt;I did not follow the 547 DOE links past confirming the host. Whatever a fuel-and-water dataset does for someone holding a broken dryer, a link audit cannot tell me.&lt;/p&gt;

&lt;p&gt;And the catalogue itself is only three appliance types. There are no manual pages for refrigerators or ovens yet, so 1,346 is a ceiling on the current build, not on the category.&lt;/p&gt;

&lt;h2&gt;
  
  
  Disclosure
&lt;/h2&gt;

&lt;p&gt;ModelNumberFinder is my site, so this is a self-audit rather than a review: &lt;a href="https://modelnumberfinder.com/" rel="noopener noreferrer"&gt;modelnumberfinder.com&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;The fix is a per-brand default instead of an allowlist. That is a day of work for the nine brands that already function and a different problem entirely for the 67 that do not — Miele and Viking do not publish manuals at stable URLs either. Those pages need to say something other than "we could not verify", and I do not know yet what that should be.&lt;/p&gt;

</description>
      <category>automation</category>
      <category>programming</category>
      <category>webscraping</category>
    </item>
    <item>
      <title>I Crawled My Own Sitemap Index. 34 of 94 Brand Files List Zero Vehicles.</title>
      <dc:creator>fei mai</dc:creator>
      <pubDate>Thu, 12 Nov 2026 08:17:39 +0000</pubDate>
      <link>https://dev.to/mediadownloaderguides/i-crawled-my-own-sitemap-index-34-of-94-brand-files-list-zero-vehicles-1eim</link>
      <guid>https://dev.to/mediadownloaderguides/i-crawled-my-own-sitemap-index-34-of-94-brand-files-list-zero-vehicles-1eim</guid>
      <description>&lt;p&gt;Saab, according to my own site, has 45 model years across 7 models.&lt;/p&gt;

&lt;p&gt;I know that because I read the Saab brand page. What I could not find was a single Saab URL in &lt;code&gt;sitemap-saab.xml&lt;/code&gt; — the file contains the brand page and seven model pages, and then it stops.&lt;/p&gt;

&lt;p&gt;So I did the boring thing and counted every file.&lt;/p&gt;

&lt;h2&gt;
  
  
  Method
&lt;/h2&gt;

&lt;p&gt;96 sitemap files: 94 brand files, a core file, an OBD file. &lt;code&gt;curl&lt;/code&gt; through a local proxy, one Python regex pulling &lt;code&gt;&amp;lt;loc&amp;gt;&lt;/code&gt; out of each XML. No page requests in this pass. Sitemaps are small, so the entire crawl finished in under a minute.&lt;/p&gt;

&lt;p&gt;A sitemap is a claim, not a fact, so there was a second pass. For each brand whose file held zero model-year URLs, I loaded the brand page, read its table (model, year count, year range), built a model-year URL from the widest span, and requested it.&lt;/p&gt;

&lt;p&gt;Numbers: 96 files, 3,198 indexed model-year URLs, 34 brand pages parsed, 34 year pages requested, 24 URLs successfully constructed.&lt;/p&gt;

&lt;h2&gt;
  
  
  Counts
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Measure&lt;/th&gt;
&lt;th&gt;Count&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Brand sitemaps in the index&lt;/td&gt;
&lt;td&gt;94&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Files containing at least one model-year URL&lt;/td&gt;
&lt;td&gt;60&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Files containing brand and model pages, zero year pages&lt;/td&gt;
&lt;td&gt;34&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Indexed model-year URLs&lt;/td&gt;
&lt;td&gt;3,198&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Nameplates (brand + model)&lt;/td&gt;
&lt;td&gt;550&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Model years claimed by the 34 silent brands&lt;/td&gt;
&lt;td&gt;642&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Years inside the 3,198: one URL per year for 1985 through 2008, two per year for 2009 through 2016, 226 for 2017, then 284 to 324 for each year from 2018 to 2026, and 152 for 2027.&lt;/p&gt;

&lt;p&gt;Second pass: 24 of the 34 brand tables were machine-readable. All 24 URLs returned 200. All 24 carried EPA fuel-economy rows. None of the 24 was in that brand's sitemap.&lt;/p&gt;

&lt;h2&gt;
  
  
  What surprised me
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;The pages exist. Only the list is broken.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;code&gt;/saab/9-3/1999&lt;/code&gt; answers 200 in 17 KB, with the EPA city, highway and combined figures for that year. &lt;code&gt;/pontiac/grand-prix/1995&lt;/code&gt;, &lt;code&gt;/mercury/grand-marquis/1995&lt;/code&gt;, &lt;code&gt;/plymouth/neon/1995&lt;/code&gt; — same. They are ordinary pages on an ordinary template, and none of them appears in the sitemap that is supposed to announce them.&lt;/p&gt;

&lt;p&gt;36% of the brand index, and if the brand pages are right about their own counts, 642 pages unannounced.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Before 2017 there is exactly one car.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Forty pages in the entire site sit before model year 2017. Thirty-two are Honda Accord, 1985 to 2016, consecutive. Eight are Honda Civic, 2009 to 2016. Everything else starts in 2017. 548 nameplates have no page older than that, and 72 have a single year page in total.&lt;/p&gt;

&lt;p&gt;So the claim "we go back to 1985" survives, and it rests on one nameplate.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The skipped brands are the ones nobody else is writing about.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Pontiac 109 model years. Mercury 92. Suzuki 83. Saturn 69. Oldsmobile 58. Isuzu 52. Saab 45.&lt;/p&gt;

&lt;p&gt;Every one of those stopped selling new cars in the US more than a decade ago. There is no competition for a 1995 Grand Prix spec page, because nobody is producing 1995 Grand Prixes. The generator left out precisely the pages with the least resistance.&lt;/p&gt;

&lt;p&gt;That is luck, not strategy. I would rather have found a strategy.&lt;/p&gt;

&lt;h2&gt;
  
  
  Where the measurement stops
&lt;/h2&gt;

&lt;p&gt;Eight of the 3,198 indexed pages were checked by hand — Accord 1988, 1996, 2004, 2010, 2016, 2018, 2022, 2026 — and all eight had EPA rows. Eight is a sample. Not a proof for the other 3,190.&lt;/p&gt;

&lt;p&gt;Ten of the 34 brands print a single year with no date range, so the script had nothing to build. Those ten are unverified and excluded from the 24.&lt;/p&gt;

&lt;p&gt;Whether Google already knows about the unlisted URLs, I cannot say. A sitemap omission is not a deindexing; internal links may have done the work. I have no Search Console access to this property, so I am treating that as unknown rather than guessing from a &lt;code&gt;site:&lt;/code&gt; count.&lt;/p&gt;

&lt;h2&gt;
  
  
  Disclosure
&lt;/h2&gt;

&lt;p&gt;I built CarSpecData, so this is an audit of my own crawl instructions rather than of somebody else's. Site: &lt;a href="https://carspecdata.com/" rel="noopener noreferrer"&gt;carspecdata.com&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;The generator fix is small. Whether the 642 pages deserve to be crawled is the harder question, and I do not have the traffic numbers to answer it.&lt;/p&gt;

</description>
      <category>data</category>
      <category>python</category>
      <category>webscraping</category>
    </item>
  </channel>
</rss>
