When shipping a new side-project, most developers remember meta tags and sitemaps, but completely forget two silent killers:
AI Crawler Directives (
robots.txt): Search engines are evolving into AI-first answers (SearchGPT, Claude, Perplexity). If your robots.txt does not explicitly handle bots likeGPTBotorOAI-SearchBot, your site might not get indexed for AI citations.Email Security (SPF & DMARC): If you send welcome emails or magic links, missing SPF/DMARC records guarantee high spam rates.
I built a free, no-signup lightweight checker to scan your domain's AI visibility and email configuration in 10 seconds:
👉 Check your site on Zen-O-Raen
Would love your feedback on which other AI crawlers I should add next!
Top comments (0)