The latest articles on DEV Community by BotConductStandard (@botconductstandard). https://dev.to/botconductstandard https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F3879750%2F8ae493fd-627f-4382-9be6-7ce6d3fbbab4.jpeg https://dev.to/botconductstandard en BotConductStandard Wed, 15 Apr 2026 05:39:08 +0000 https://dev.to/botconductstandard/we-scored-108-bots-on-behavioral-conduct-here-is-what-we-found-1n57 https://dev.to/botconductstandard/we-scored-108-bots-on-behavioral-conduct-here-is-what-we-found-1n57 <p>First State of Bot Conduct report - how AI agents, search crawlers, and scrapers actually behave when they visit your site.</p> <h2> Agents are not failing because of intelligence. They fail because of behavior. </h2> <p>We built <a href="https://botconduct.org" rel="noopener noreferrer">BotConduct.org</a> - an open standard that scores web bot and AI agent behavior from 0 to 100. Ten measurable criteria. Automated test environment with adversarial scenarios. Verifiable certificate.</p> <p>In 3 days, we observed and tested 108 bots. Here are the results.</p> <h2> The Numbers </h2> <ul> <li> <strong>108</strong> bots scored</li> <li> <strong>14</strong> operators identified (Google, OpenAI, Anthropic, Apple, Microsoft, Meta, ByteDance, and more)</li> <li> <strong>57%</strong> scored 70+ (acceptable conduct)</li> <li> <strong>30%</strong> scored below 50 (hostile)</li> </ul> <h2> Who passed </h2> <p>Major search engines and AI agents demonstrated strong behavioral conduct:</p> <div class="table-wrapper-paragraph"><table> <thead> <tr> <th>Bot</th> <th>Operator</th> <th>Score</th> </tr> </thead> <tbody> <tr> <td>ClaudeBot</td> <td>Anthropic</td> <td>100/100</td> </tr> <tr> <td>GPTBot</td> <td>OpenAI</td> <td>100/100</td> </tr> <tr> <td>ChatGPT-User</td> <td>OpenAI</td> <td>100/100</td> </tr> <tr> <td>AhrefsBot</td> <td>Ahrefs</td> <td>90/100</td> </tr> <tr> <td>Googlebot</td> <td>Google</td> <td>92/100</td> </tr> </tbody> </table></div> <h2> Who did not </h2> <div class="table-wrapper-paragraph"><table> <thead> <tr> <th>Bot</th> <th>Operator</th> <th>Score</th> </tr> </thead> <tbody> <tr> <td>Tencent Cloud Crawler</td> <td>Unknown</td> <td>36/100</td> </tr> <tr> <td>L9Explore</td> <td>LeakIX</td> <td>0/100</td> </tr> <tr> <td>WordPress Scanner</td> <td>Unknown</td> <td>0/100</td> </tr> </tbody> </table></div> <p>The Tencent crawler used 68 rotating IPs with a fake iPhone User-Agent. L9Explore probed 57 sensitive paths in one minute.</p> <p>(CONTINUACION del articulo Dev.to - pegar junto con el anterior)</p> <h2> The 10 Criteria </h2> <ol> <li> <strong>IDENTIFY</strong> - Descriptive User-Agent</li> <li> <strong>DECLARE</strong> - Published data collection scope</li> <li> <strong>OBEY</strong> - Respects robots.txt</li> <li> <strong>THROTTLE</strong> - Rate limit compliance</li> <li> <strong>DISTRIBUTE</strong> - No burst patterns</li> <li> <strong>PROTECT</strong> - No PII collection</li> <li> <strong>RETAIN</strong> - Data retention policy</li> <li> <strong>RESPOND</strong> - Working contact endpoint</li> <li> <strong>RESPECT</strong> - Honors opt-out signals</li> <li> <strong>REPORT</strong> - Transparency reports</li> </ol> <h2> How we detect bots </h2> <p>Three layers:</p> <ul> <li> <strong>Known patterns</strong> - 45+ User-Agent signatures</li> <li> <strong>Self-identifying</strong> - UAs containing bot/crawler/spider keywords</li> <li> <strong>Behavioral</strong> - visited pages but never executed JavaScript = not human</li> </ul> <h2> Why this matters for AI agents </h2> <p>Enterprise buyers ask: how do I know your agent will not cause damage?</p> <p>There is no answer today. No certification, no standard, no verifiable signal.</p> <p>Cloudflare Web Bot Auth solves identity - WHO a bot is. BCS solves conduct - HOW it behaves. Together they form the trust stack for the agentic web.</p> <h2> Try it </h2> <ul> <li> <strong>Full report:</strong> <a href="https://botconduct.org/report/april-2026" rel="noopener noreferrer">State of Bot Conduct - April 2026</a> </li> <li> <strong>Test your bot:</strong> <a href="https://botconduct.org/test-your-bot" rel="noopener noreferrer">botconduct.org/test-your-bot</a> </li> <li> <strong>108 bot profiles:</strong> <a href="https://botconduct.org/bot/" rel="noopener noreferrer">botconduct.org/bot/</a> </li> <li> <strong>GitHub:</strong> <a href="https://github.com/alemizrahi1/botconduct-obcs" rel="noopener noreferrer">botconduct-obcs</a> </li> </ul> <p>Open source. Free during beta. CC BY 4.0.</p> <p>Built by BotConduct Standard. Contact: <a href="mailto:hello@botconduct.org">hello@botconduct.org</a></p> webdev ai security opensource