How We Built an AI Agent That Runs Technical SEO Audits Better Than Most Humans

Technical SEO audits should be automated. Not because humans are bad at them — because they're too slow.

A thorough technical audit of a mid-sized site (5,000–10,000 pages) takes a skilled SEO specialist roughly 2–3 days. By the time the report is delivered, the crawl data is already a week old. By the time the dev team acts on it, the findings are stale.

Tools like Screaming Frog and Sitebulb solved the crawling problem, but left the rest of the workflow untouched — downloading reports, analyzing issues, prioritizing fixes, assigning tasks, following up. Each step is manual. Each handoff loses context.

We built visibility.so's audit agents to solve one specific problem: close the gap between "here's what's broken" and "it's being fixed."

Full article on sanjayshankar.me →

---

The Problem: Audits Don't Scale

Most SEO teams follow a familiar pattern:

1. Run a desktop crawler — manual trigger, one-time export
2. Open the CSV in a spreadsheet
3. Manually prioritize issues by severity
4. Copy-paste findings into a task tracker
5. Assign tasks to developers via Slack or email
6. Follow up repeatedly
7. Re-crawl a month later to verify fixes

That's 7 steps for a single audit cycle. Half of them are administrative — not analytical.

Bottleneck	What It Costs You
Desktop-only tools	Can't schedule recurring crawls
CSV exports	Issues sit in files, not in your workflow
Manual prioritization	2–4 hours per audit cycle wasted
No task integration	~60% of audit findings never get actioned
No dedup across runs	Same issues reported week after week
No automatic follow-up	No way to verify fixes without re-crawling

---

The Architecture: Two Agents, One Pipeline

When you set a domain on any project in visibility.so, the system automatically provisions two audit agents. No configuration needed.

Agent 1: On-Page Auditor

Uses a real browser to inspect each page visually. It checks heading structure, meta tags, content quality signals, internal linking, image optimization, and schema markup.

Agent 2: Technical SEO Auditor

Focuses on infrastructure-level signals: crawlability, indexing coverage, Core Web Vitals (measured from actual browser render), structured data, performance, and security.

---

How an Audit Run Works (Step by Step)

Step 1: Scheduled Trigger

The scheduler wakes the assigned agent and passes context — project, domain, audit type, and existing issues list.

Step 2: Browser Access

The agent starts with a full browser session, allowing it to render JavaScript, measure Core Web Vitals, detect lazy-loading issues, and capture screenshots.

Step 3: Intelligent Crawling

Prioritized crawl: sitemap first → high-authority pages → canonical URLs → redirect chains. A 10,000-page site takes ~40-50 minutes.

Step 4: Issue Detection

Every finding is classified: critical, high, medium, or low — with suggested fixes.

Step 5: Dedup

The agent checks existing issues before logging. Same broken link? Already reported. Only new issues surface.

Step 6: Automatic Task Creation

Critical and high-severity issues create tasks in the project board with description, fix suggestion, and screenshot evidence.

---

From 3 Days to 47 Minutes

Time	Event
02:00 UTC	Scheduler triggers the agent
02:05–02:47	Agent crawls site with real browser
02:47	Agent logs 14 new opportunities
02:48	Agent creates 3 critical tasks
08:30	Team reviews findings
09:15	Dev team picks up first task

Agent time: 47 minutes. Human time: 45 minutes. Manual cycle: ~28 hours.

That's a 96% reduction in time-to-action.

---

Comparison

Capability	visibility.so	Desktop Crawler	Manual
Recurring schedule	✅	❌	❌
JavaScript rendering	✅	❌	✅
Task creation	✅ Auto	❌ CSV	❌ Manual
Dedup across weeks	✅	❌	❌
Time per audit	~47 min	~4 hrs	~2-3 days
Cost per audit	Included	£149/yr	~$800 billable

---

This article was originally published on sanjayshankar.me.