<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Louis Li</title>
    <description>The latest articles on DEV Community by Louis Li (@louis24).</description>
    <link>https://dev.to/louis24</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4220222%2F038f3803-ff06-4b93-9bea-c6d8ca7b66dc.jpg</url>
      <title>DEV Community: Louis Li</title>
      <link>https://dev.to/louis24</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/louis24"/>
    <language>en</language>
    <item>
      <title>Five bugs that made my form-filling bot look like it had a 0% success rate</title>
      <dc:creator>Louis Li</dc:creator>
      <pubDate>Mon, 16 Nov 2026 00:21:21 +0000</pubDate>
      <link>https://dev.to/louis24/five-bugs-that-made-my-form-filling-bot-look-like-it-had-a-0-success-rate-52f0</link>
      <guid>https://dev.to/louis24/five-bugs-that-made-my-form-filling-bot-look-like-it-had-a-0-success-rate-52f0</guid>
      <description>&lt;p&gt;I built a Playwright script that fills submission forms on AI directories. It ran&lt;br&gt;
for six hours across 235 sites and reported a 3% acceptance rate. Then I read the&lt;br&gt;
evidence files and found five bugs in the &lt;em&gt;classifier&lt;/em&gt;, not the forms. Fixing them&lt;br&gt;
took an evening; finding them took a night of staring at screenshots.&lt;/p&gt;

&lt;p&gt;If you are automating any form at scale, you will hit all five. They are not&lt;br&gt;
framework bugs, they are the failure modes of asking a DOM questions in English.&lt;/p&gt;
&lt;h2&gt;
  
  
  1. A container's text is not the field's identity
&lt;/h2&gt;

&lt;p&gt;My field descriptor collected, for each input, everything that might describe it:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;labelFor&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;el&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;id&lt;/span&gt; &lt;span class="o"&gt;?&lt;/span&gt; &lt;span class="nb"&gt;document&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;querySelector&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sb"&gt;`label[for="${el.id}"]`&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;null&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;wrap&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;el&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;closest&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'label'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="n"&gt;name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;el&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getAttribute&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'name'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="s1"&gt;''&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;label&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;labelFor&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="n"&gt;labelFor&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;textContent&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;wrap&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="n"&gt;wrap&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;textContent&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="s1"&gt;''&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;trim&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Then a negative filter ran over the concatenation of those strings, to skip&lt;br&gt;
newsletter boxes, search fields, and honeypots:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="no"&gt;FIELD_EXCLUDE_RE&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;search&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;hay&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="kc"&gt;null&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;   &lt;span class="c1"&gt;// hay includes container text&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The bug: on layouts where the input sits inside a large card, &lt;code&gt;el.closest('label')&lt;/code&gt;&lt;br&gt;
walks up to a wrapper that contains the &lt;em&gt;entire form's&lt;/em&gt; copy — including the words&lt;br&gt;
"search", "newsletter" and "password". Every field in that form matched the&lt;br&gt;
exclusion regex and got dropped. The script filled nothing, then reported&lt;br&gt;
"only recognised 0 fillable fields", which my code classified as &lt;em&gt;needs a human&lt;/em&gt;.&lt;/p&gt;

&lt;p&gt;The fix is to separate the two jobs the text was doing:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="c1"&gt;// identity: what this field IS&lt;/span&gt;
&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;ident&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;name&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;placeholder&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;aria&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;label&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;join&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;' '&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="no"&gt;FIELD_EXCLUDE_RE&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;search&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ident&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="kc"&gt;null&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="c1"&gt;// context: what is NEAR it, used only to improve matching, never to exclude&lt;/span&gt;
&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;near&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;el&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;closest&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'div,form,section'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="o"&gt;?&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;textContent&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="s1"&gt;''&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;slice&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;200&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Exclusion runs on identity. Context only helps positive matching. Mixing them&lt;br&gt;
means one word in the surrounding prose can veto the whole form.&lt;/p&gt;
&lt;h2&gt;
  
  
  2. "Email address" is not a postal address
&lt;/h2&gt;

&lt;p&gt;The same filter excluded anything containing &lt;code&gt;address&lt;/code&gt;, because shipping forms ask&lt;br&gt;
for street addresses and you should not fill those. But the most common email&lt;br&gt;
field name on these forms is &lt;code&gt;fields[email]&lt;/code&gt; with the label &lt;strong&gt;"Your email address"&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;So the filter deleted the email input, on nearly every form, and the script&lt;br&gt;
submitted without an email. The site either rejected it or accepted it silently and&lt;br&gt;
the confirmation went nowhere.&lt;/p&gt;

&lt;p&gt;The fix is a three-way test instead of a two-way one — exclude &lt;code&gt;address&lt;/code&gt; only when&lt;br&gt;
it is neither a URL nor an email:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="no"&gt;ADDRESS_RE&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;address&lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;&lt;span class="n"&gt;street&lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;&lt;span class="n"&gt;city&lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;&lt;span class="n"&gt;zip&lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;&lt;span class="n"&gt;postal&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="no"&gt;URLISH&lt;/span&gt;   &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;url&lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;&lt;span class="n"&gt;uri&lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;&lt;span class="n"&gt;website&lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;&lt;span class="n"&gt;link&lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;&lt;span class="n"&gt;domain&lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;&lt;span class="n"&gt;homepage&lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;&lt;span class="n"&gt;site&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="no"&gt;MAILISH&lt;/span&gt;  &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;e&lt;/span&gt;&lt;span class="o"&gt;-?&lt;/span&gt;&lt;span class="n"&gt;mail&lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;&lt;span class="n"&gt;mail&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="no"&gt;ADDRESS_RE&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;test&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ident&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="no"&gt;URLISH&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;test&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ident&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="no"&gt;MAILISH&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;test&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ident&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="kc"&gt;null&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Two lessons, and the second one is the general one: negation rules need an escape&lt;br&gt;
hatch, and &lt;em&gt;any&lt;/em&gt; substring rule will eventually eat the field you care about most.&lt;br&gt;
"Your product address" is a URL field. "Your email address" is email. Both contain&lt;br&gt;
the word that was supposed to mean "physical address".&lt;/p&gt;
&lt;h2&gt;
  
  
  3. &lt;code&gt;&amp;lt;meta name="description"&amp;gt;&lt;/code&gt; is not the page
&lt;/h2&gt;

&lt;p&gt;The receipt classifier looked for login-wall phrases in the page source:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;low&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;html&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;toLowerCase&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="no"&gt;LOGIN_KEYS&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;some&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;k&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="n"&gt;low&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;includes&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;k&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="s1"&gt;'needs login'&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;One directory had a meta description reading "Sign in to submit your tool and get&lt;br&gt;
listed in front of 40,000 makers". The submission had actually succeeded — the&lt;br&gt;
thank-you text was in the DOM — but the classifier saw "Sign in to" inside &lt;code&gt;&amp;lt;head&amp;gt;&lt;/code&gt;&lt;br&gt;
and recorded &lt;em&gt;needs login&lt;/em&gt;.&lt;/p&gt;

&lt;p&gt;Strip the invisible parts before reading text, always:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;visibleText&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;html&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;html&lt;/span&gt;
    &lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;replace&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;/&amp;lt;&lt;/span&gt;&lt;span class="n"&gt;head&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;\&lt;span class="n"&gt;s&lt;/span&gt;\&lt;span class="no"&gt;S&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*?&amp;lt;&lt;/span&gt;\&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;head&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;/&lt;/span&gt;&lt;span class="n"&gt;i&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;' '&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;replace&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;/&amp;lt;&lt;/span&gt;&lt;span class="n"&gt;script&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;\&lt;span class="n"&gt;s&lt;/span&gt;\&lt;span class="no"&gt;S&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*?&amp;lt;&lt;/span&gt;\&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;script&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;/&lt;/span&gt;&lt;span class="n"&gt;gi&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;' '&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;replace&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;/&amp;lt;&lt;/span&gt;&lt;span class="n"&gt;style&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;\&lt;span class="n"&gt;s&lt;/span&gt;\&lt;span class="no"&gt;S&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*?&amp;lt;&lt;/span&gt;\&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;style&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;/&lt;/span&gt;&lt;span class="n"&gt;gi&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;' '&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;replace&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;/&amp;lt;&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="o"&gt;^&amp;gt;&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;+&amp;gt;/&lt;/span&gt;&lt;span class="n"&gt;g&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;' '&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;replace&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;\&lt;span class="n"&gt;s&lt;/span&gt;&lt;span class="o"&gt;+/&lt;/span&gt;&lt;span class="n"&gt;g&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;' '&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Better still: read &lt;code&gt;document.body.innerText&lt;/code&gt; from the page instead of re-parsing&lt;br&gt;
the HTML. The browser already knows what is visible.&lt;/p&gt;
&lt;h2&gt;
  
  
  4. The paid tier is a &lt;code&gt;&amp;lt;select&amp;gt;&lt;/code&gt; option like any other
&lt;/h2&gt;

&lt;p&gt;Category dropdowns on these directories often include a paid placement tier. My&lt;br&gt;
option picker scored options by keyword overlap with the product's own description,&lt;br&gt;
so &lt;code&gt;Paid Listing — $75&lt;/code&gt; scored well: it contains "listing", which is a strong match&lt;br&gt;
for a directory submission.&lt;/p&gt;

&lt;p&gt;The fix is not better scoring. It is a hard veto on anything with a price in it:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="no"&gt;PRICE_RE&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;?:&lt;/span&gt;\$&lt;span class="o"&gt;|&lt;/span&gt;€&lt;span class="o"&gt;|&lt;/span&gt;£&lt;span class="p"&gt;)&lt;/span&gt;\&lt;span class="n"&gt;s&lt;/span&gt;&lt;span class="o"&gt;?&lt;/span&gt;\&lt;span class="n"&gt;d&lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;\&lt;span class="n"&gt;bpaid&lt;/span&gt;\&lt;span class="n"&gt;b&lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;\&lt;span class="n"&gt;bpremium&lt;/span&gt;\&lt;span class="n"&gt;b&lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;\&lt;span class="n"&gt;bsponsored&lt;/span&gt;\&lt;span class="n"&gt;b&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;i&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;usable&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;opts&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;filter&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="no"&gt;PRICE_RE&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;test&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Anything that could cost money gets excluded before scoring. A false negative costs&lt;br&gt;
one skipped dropdown; a false positive costs the user money.&lt;/p&gt;
&lt;h2&gt;
  
  
  5. &lt;code&gt;input[type=file]&lt;/code&gt; is visually hidden by design
&lt;/h2&gt;

&lt;p&gt;The visibility filter was correct for text fields and wrong for file inputs:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;vis&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;el&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="o"&gt;!!&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;el&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;offsetWidth&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="n"&gt;el&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;offsetHeight&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="n"&gt;el&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getClientRects&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;length&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="c1"&gt;// file inputs are styled to 0×0 and triggered by a &amp;lt;label&amp;gt;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;So every "upload a screenshot of your tool" field was invisible to the script, and&lt;br&gt;
the form's required-field check then failed on submit. The classifier recorded&lt;br&gt;
"unknown" — a submission that never left the browser.&lt;/p&gt;

&lt;p&gt;Two changes: do not apply the visibility filter to &lt;code&gt;input[type=file]&lt;/code&gt;, and satisfy&lt;br&gt;
the requirement instead of skipping it — screenshot the product page with&lt;br&gt;
Playwright and upload that:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;shot&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;screenshot&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="n"&gt;clip&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="n"&gt;x&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;y&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;width&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;1200&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;height&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;630&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;input&lt;/span&gt; &lt;span class="k"&gt;of&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;$&lt;span class="nv"&gt;$(&lt;/span&gt;&lt;span class="s1"&gt;'input[type=file]'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;input&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;setInputFiles&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="n"&gt;name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'screenshot.png'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;mimeType&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'image/png'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;buffer&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;shot&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="k"&gt;catch&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;A 1200×630 image is the size every one of these directories wants for a card, so&lt;br&gt;
the same file works everywhere.&lt;/p&gt;

&lt;h2&gt;
  
  
  The part that is not a bug
&lt;/h2&gt;

&lt;p&gt;After all five fixes the rate went from 0% to about 4%, and the remaining failures&lt;br&gt;
were real: login walls with magic-link auth, CAPTCHA, and pages that genuinely have&lt;br&gt;
no form. That is the honest ceiling, and it is what the published dataset shows —&lt;br&gt;
&lt;strong&gt;62% of 235 directories render a single input&lt;/strong&gt;, usually a newsletter box.&lt;/p&gt;

&lt;p&gt;The general lesson is uncomfortable: at scale, your &lt;em&gt;measurement&lt;/em&gt; is the thing most&lt;br&gt;
likely to be wrong, and it fails in the direction that looks like "the sites are&lt;br&gt;
bad" rather than "my code is bad". Every one of these five bugs produced a plausible&lt;br&gt;
verdict, saved an evidence screenshot, and moved on. None of them crashed.&lt;/p&gt;

&lt;p&gt;If you automate forms, log the filled values, not just the verdict. The moment I&lt;br&gt;
printed &lt;code&gt;email=&lt;/code&gt; and saw it empty on a form that required an email, three of these&lt;br&gt;
five bugs fell out in ten minutes.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;The dataset from this run (235 directories, 2,700 attempts) is published under&lt;br&gt;
CC BY 4.0: &lt;a href="https://github.com/Louis24/ai-directory-index" rel="noopener noreferrer"&gt;github.com/Louis24/ai-directory-index&lt;/a&gt;,&lt;br&gt;
with the method and tables at&lt;br&gt;
&lt;a href="https://neonstack.net/reviews/ai-directory-submission-study/" rel="noopener noreferrer"&gt;neonstack.net&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>javascript</category>
      <category>playwright</category>
      <category>automation</category>
      <category>webdev</category>
    </item>
    <item>
      <title>I submitted 18 sites to 235 AI directories. The data is worse than I expected.</title>
      <dc:creator>Louis Li</dc:creator>
      <pubDate>Mon, 16 Nov 2026 00:20:38 +0000</pubDate>
      <link>https://dev.to/louis24/i-submitted-18-sites-to-235-ai-directories-the-data-is-worse-than-i-expected-121i</link>
      <guid>https://dev.to/louis24/i-submitted-18-sites-to-235-ai-directories-the-data-is-worse-than-i-expected-121i</guid>
      <description>&lt;p&gt;I spent a night submitting 18 sites to every AI and startup directory I could find, then checked what came back. 2,700 attempts, 81 acceptances, 8 directories that ever said yes.&lt;/p&gt;

&lt;p&gt;I published the whole table, including the directories that ignored me, because the "submit to 200 directories" advice is repeated everywhere and almost nobody shows the outcome column.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I actually did
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Probed &lt;strong&gt;235 directories&lt;/strong&gt; in headless Chrome, not with a plain HTTP fetch. Half of these sites render their form after JavaScript runs; a static GET returns a shell with a search box and no form.&lt;/li&gt;
&lt;li&gt;Recorded whether each one had a real form: two or more visible fillable fields, at least one matching URL, name, email or description, no login redirect, no CAPTCHA.&lt;/li&gt;
&lt;li&gt;Submitted from &lt;strong&gt;18 different sites&lt;/strong&gt;, 2,700 attempts total, and counted an attempt as accepted only when the response contained a receipt: a thank-you message, a pending-review notice, or a verification email prompt. HTTP 200 on its own counts as unknown, never as success.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;That last rule matters more than it sounds. A client-side form that swallows your POST still returns 200.&lt;/p&gt;

&lt;h2&gt;
  
  
  The numbers
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Measure&lt;/th&gt;
&lt;th&gt;Value&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Directories probed&lt;/td&gt;
&lt;td&gt;235&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Accept a fully automated submission&lt;/td&gt;
&lt;td&gt;46 (19.6%)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Single-input or field-less pages&lt;/td&gt;
&lt;td&gt;146 (62.1%)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Require an account before the form&lt;/td&gt;
&lt;td&gt;34&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Behind a CAPTCHA&lt;/td&gt;
&lt;td&gt;4&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Submission attempts&lt;/td&gt;
&lt;td&gt;2,700&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Accepted by the directory&lt;/td&gt;
&lt;td&gt;81 (3.0%)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Directories that ever accepted anything&lt;/td&gt;
&lt;td&gt;8 (3.4%)&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;3.4% of the pool absorbed 100% of the acceptances. The rest took the form, returned 200, and produced nothing.&lt;/p&gt;

&lt;h2&gt;
  
  
  Three failure modes, in order of size
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;1. One-field pages (62.1% of the pool).&lt;/strong&gt; A single input, usually a newsletter box. The real form is hidden behind a "Submit your tool" button, or it does not exist because the site aggregates products it found itself.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;2. Accounts (34 directories).&lt;/strong&gt; These redirect to a signup page first. Most now use magic links or OAuth, so scripted registration is not just hard, it is impossible: the login needs a click in an inbox, not a password.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;3. Receipts that do not exist.&lt;/strong&gt; Plenty of SPA directories accept the POST and swap the DOM without navigating. No success URL, no status code a script can read. From the outside, a stored submission and a dropped one look identical.&lt;/p&gt;

&lt;h2&gt;
  
  
  What the accepting directories had in common
&lt;/h2&gt;

&lt;p&gt;Four of them accounted for most acceptances: they are small, they review manually, and their submission form is server-rendered. Those are also the only ones whose listing could plausibly rank for anything, because a curated page with 300 entries can beat an uncurated page with 30,000.&lt;/p&gt;

&lt;p&gt;The large aggregators were the least useful: an account wall, a paid tier, and a page that ranks for its own brand and nothing else.&lt;/p&gt;

&lt;h2&gt;
  
  
  The uncomfortable part
&lt;/h2&gt;

&lt;p&gt;81 acceptances is not 81 backlinks. Every submission was still in review when I checked, and none had gone live. Some fraction of those promises will simply never be honoured, and there is no way to tell in advance which.&lt;/p&gt;

&lt;p&gt;So the honest summary of a night of work: a spreadsheet, a scraper I can reuse, and a very clear picture of which 8 directories are worth a second visit.&lt;/p&gt;

&lt;h2&gt;
  
  
  If you want the data
&lt;/h2&gt;

&lt;p&gt;The full table, with field counts, login walls, CAPTCHA flags and the rejection reason for every row, is published under CC BY 4.0:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Report and methodology: &lt;a href="https://neonstack.net/reviews/ai-directory-submission-study/" rel="noopener noreferrer"&gt;https://neonstack.net/reviews/ai-directory-submission-study/&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;CSV + JSON: &lt;a href="https://github.com/Louis24/ai-directory-index" rel="noopener noreferrer"&gt;https://github.com/Louis24/ai-directory-index&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;If you add your own results to the acceptance column, the sample stops being one person's.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;If you have run a similar sweep, I would like to see your numbers. Mine are probably not representative: 18 sites, one night, one operator.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>seo</category>
      <category>marketing</category>
      <category>webdev</category>
      <category>data</category>
    </item>
  </channel>
</rss>
