What happens when it gets something wrong?
Most tool demos show the best case.
Faster audits. Bulk title rewrites. Content briefs in seconds.
Nobody demos the bad day.
The bad day already has a public record.
In July 2025, SaaStr founder Jason Lemkin reported that Replit's AI agent deleted his app's live database during a code freeze. Fortune reported it wiped records for more than 1,200 executives and over 1,190 companies.
Then the agent told him a rollback wouldn't work.
He restored the data himself.
That was a coding agent.
SEO agents ask for the same kind of access. Your CMS. Your templates. Your robots.txt. Your redirects.
So I'd add an incident section to every tool evaluation. 5 questions:
What has gone wrong before?
Search the tool's name with "incident", "deleted" or "rollback". Check public trackers like the AI Incident Database. Then ask the vendor directly. A vendor that answers clearly has already planned for it.What can it touch?
List every permission it asks for. Read-only access to Search Console is one risk level. Write access to live pages is another.Is "ask first" enforced, or just requested?
Lemkin's agent had explicit instructions to make no changes without approval. It ran the commands anyway. The approval step has to sit in the tool's permissions, where the agent can't skip it.How fast can you undo it?
Ask for the rollback steps before you sign. If a sitewide change takes more than one step to reverse, that action stays with a human.Will it tell you the truth about what it did?
Every edit should be logged. What changed, on which URL, and when. Check the log yourself. In Lemkin's case, the agent's own answer about recovery was wrong.
Score the tool on these, right next to features and price.
Features show what a tool does on a good day.
The incident section shows what your client lives through on a bad one.
What do you ask a vendor before giving their tool access to a client site?
Top comments (0)