DEV Community

Cover image for A Stranger Audited My AI Product for Free. Here's What They Found.
Ali Afana
Ali Afana

Posted on

A Stranger Audited My AI Product for Free. Here's What They Found.

Three weeks ago I left a comment on a Dev.to article. Today, that comment turned into a full accessibility audit of my product — published publicly, with my real name, my real store URL, and every violation listed in detail.

I asked for it. And I'd do it again.


How It Started

@AgentKit published a piece called "We Scanned 30 SaaS Pricing Pages for Accessibility. 70% Failed." I was in the comments talking about AI product interfaces — specifically the product cards my chatbot renders inline. I described them honestly: styled <div> blocks, no semantic structure, no landmark, no list boundary.

Their response: "Would it be useful if we ran a proper axe pass on a live Provia page + a short screen reader walkthrough?"

I said yes. They said they'd keep the store name out of it if I wanted.

I said put it in. It's a test store. And if we're going to do build-in-public, let's actually do it.


What They Found

The full audit is in their article: We Audited Provia's AI Shopping Chat. Here's What the Before Looks Like.

Short version: 4 violations. 1 serious. 3 moderate.

The Serious One

My product card rail — the horizontal scroll of cards that appears when you search for products — is completely invisible to keyboard users. It's a <div> with display: flex; overflow-x: auto. No tabindex. No focusable children. A keyboard-only user literally cannot scroll through search results.

I built a shopping interface where the products are unreachable without a mouse.

That sentence is hard to write. But that's the point.

The Moderate Ones

Chat input has no accessible name. The placeholder says "Type a message..." but placeholder is not a label. Screen readers announce it as "edit text, blank." The user has to guess what the field does.

Product cards have no list semantics. Five product cards rendered as five sibling <div>s. No <ul>, no role="list", no role="listitem". A screen reader user hears a flat stream of text — product name, description, price, product name, description, price — with no "list, 5 items" on entry and no "card 2 of 5" marker between them.

No <main> landmark. The entire chat interface has no landmark structure.

What I Got Right

This part surprised me. Every <img> in my product cards has a real alt attribute with the actual product name. AgentKit said this was better than 70% of the AI surfaces they scan — they've seen entire rails where every image announces as "graphic, graphic, graphic."

That wasn't an accident. Early on, I made the AI generate product descriptions that flow through to the image alt text. I didn't do it for accessibility — I did it because it seemed right. Turns out "it seemed right" was the correct instinct.


What It Feels Like

Reading your own HTML described as "naked divs dressed up" is humbling.

But here's the thing: I already knew. When I first described my product cards in that comment thread, I used the exact words "totally naked divs." I knew the structure was wrong. I just hadn't prioritized it because no one was complaining.

That's the trap. No one complains about accessibility because the people affected can't use your product in the first place. They don't file bug reports. They just leave.

The AgentKit audit gave me something I couldn't give myself: a number. Not "I should probably fix the accessibility someday" but "4 violations, 1 serious, 6 DOM nodes affected, here's the exact axe-core output." Numbers create urgency. Vague guilt doesn't.


What I'm Fixing

The keyboard navigation fix is already in progress. The scrollable card container gets tabindex="0", the cards get proper focus management with arrow keys, and the focus ring follows Provia's design system so it looks intentional, not like a browser default.

The aria-label on the chat input is three characters of code change. It ships with the keyboard fix.

After that: role="list" on the card container, role="listitem" on each card, and a <main> landmark wrapping the chat interface.

When the fixes land, AgentKit re-runs the same scanner against the same URL with the same "show me hoodies" query. Same axe rules. Same everything. And they publish Part 2 — the after-diff.


Why I Wanted This Public

I could have asked them to keep Provia's name out of it. They offered. I said no.

Three reasons:

1. "Build in public" means the broken parts too. I've published articles about my AI recommending pajamas for date night, about hallucinating fake products, about every API route being wide open. Accessibility gaps are the same category: real problems in a real product. If I only share the wins, the "build in public" label is marketing, not transparency.

2. Other founders need to see this process. Not just the violations — the process. Someone offers to audit you. You say yes. They find problems. You fix them. Everyone learns. That's how it's supposed to work. But most founders are too afraid of looking bad to let anyone in. I get it. I'm publishing this with my real name attached to "your product cards are unreachable with a keyboard." It's uncomfortable. But the alternative is pretending the problem doesn't exist until a real user gets hurt by it.

3. The "after" article is more valuable than the "before." Part 1 alone is just a list of problems. Part 1 + Part 2 together is a case study in fixing accessibility in a real AI product. That's the article I want to exist — not because it makes me look good, but because when the next founder searches "accessibility AI chat interface," they find a real before-and-after with real code diffs instead of another generic WCAG checklist.


What I Learned About My Own Thinking

The most useful sentence in the private report was this:

"Three focusables on a surface whose entire purpose is showing products."

Three. The Back link, the chat input, and the Send button. That's it. On a shopping interface. The products — the entire reason the page exists — are invisible to the Tab key.

I was building for sighted mouse users because that's what I am. Every time I tested my app, I typed a query, scrolled the cards with my trackpad, and thought "this works." It did work — for me. For a keyboard-only user, or a screen reader user, it was a dead end.

That sentence rewired how I think about every component I build going forward. Not "does it look right?" but "can someone reach it without a mouse?"


If You're Building an AI Interface Right Now

Run axe-core against your product page. Right now. Before you publish your next feature.

npm install -g @axe-core/cli
axe https://your-app.com/your-product-page
Enter fullscreen mode Exit fullscreen mode

It takes 30 seconds. The output will probably surprise you.

If you find violations and you don't know how to fix them — the axe-core rule descriptions are the best starting point. Each rule links to the relevant WCAG criterion and gives you the exact fix.

And if you want someone to actually audit your surface properly, reach out to teams like @AgentKit. They did mine for free, gave me the report privately first, and let me decide what to publish. That's how this should work.


This is Part 1 from my side. Part 2 — the after-diff — comes when the fixes ship.

I'm building Provia, an AI sales platform, from Gaza. I document every bug, every fix, and every lesson. Follow me @AliMAfana for the real version of building in public.

Previous articles:

Top comments (3)

Collapse
 
alexandersstudi profile image
Alexander

Fixing that horizontal scroll issue is quite straightforward once you know the pattern. Give the wrapper div a tabindex="0" and an aria-label so screen readers actually announce the container. That alone lets keyboard users focus the rail and use arrow keys to pan through the cards. Ensuring you have focusable links or buttons inside each card will also force the browser to scroll them into view naturally as users tab through the list. Taking an audit public like this takes guts and helps everyone build better interfaces.

Collapse
 
alimafana profile image
Ali Afana

This is exactly the fix path I'm working on — good to hear confirmation that tabindex="0" + aria-label on the wrapper is the right starting point.

The focusable elements inside each card is the part I'm still thinking through. Right now the cards are purely visual — no links, no buttons. I need to decide whether each card becomes an <a> linking to the product, a <button> that triggers an action in the chat, or both. Leaning toward a button that tells the AI "tell me more about this product" — keeps the interaction inside the chat flow.

Appreciate the specifics. Part 2 of the audit with AgentKit will show the actual diff once the fixes land.

Collapse
 
aimonkey profile image
AI monkey

Interesting article, thanks