Someone asks ChatGPT for "the best fragrance-free sunscreen for oily skin". The answer names three brands, links two review sites and gets one price wrong. If one of those brands is yours, or should have been, you'll never hear about it. Nothing shows up in your analytics.
AI brand tracking is how you find out. You ask the assistants the questions your buyers ask, you record what they say, and you check it on a schedule. It sounds like a marketing job, but it's mostly a data-collection problem, so it's a good one for a developer to own.
Here's a version you can set up in an afternoon and keep up in about an hour a month.
Why AI brand tracking matters for developers
Three reasons it ends up on your desk:
- The fix is usually on your site. When an assistant gets a fact wrong, the cause is often an outdated page, a missing spec, or structured data that says something different from the visible text. Those are code changes.
- There's no dashboard for it. Assistants don't publish what people ask them or how often you appear. You have to generate the data yourself.
- Answers vary from run to run. Ask the same question twice and you can get a different list. One screenshot proves nothing; you need repeated runs logged the same way each time.
Step 1: Pick the brand questions to test
Write 15 to 20 questions in your buyers' words, not your marketing copy. Split them into four groups, because each one tests something different:
| Group | What it tests | Example (skincare) |
|---|---|---|
| Category | Are you recommended at all? | "Best fragrance-free sunscreen for oily skin in India" |
| Comparison | How are you described next to a rival? | "[Your brand] vs [competitor] sunscreen" |
| Brand | Are the facts about you right? | "Is [your brand] sunscreen safe for sensitive skin?" |
| Problem | Do you appear before they know the product? | "Why does my sunscreen leave a white cast?" |
Good sources for the wording: your support inbox, product reviews, Reddit threads in your category, and Google's "People also ask" boxes.
Keep the list fixed. If you change the questions every month, you can't compare the results.
Step 2: Run the same prompts across LLMs and record the answers
You have two ways to collect answers, and you want both.
In the apps (what shoppers actually see). Open ChatGPT, Gemini, Perplexity and Google, use a fresh chat each time, and sign out or use a private window where you can, so your history doesn't shape the answer. Paste each question and log the result. This is the version shoppers see, with web search switched on.
Through the APIs (repeatable, scriptable). API answers often differ from the apps, because the app may search the web and the API call usually doesn't. Even so, they're useful for spotting what a model "knows" about you without a search step, and you can run them as often as you like.
Here's a small script for the API side. It uses OpenRouter so one key covers several model providers, and it only needs the Python standard library. Copy the exact model IDs you want from openrouter.ai/models.
import csv, json, os, re, time, urllib.request
from datetime import date
BRAND = "YourBrand"
COMPETITORS = ["Competitor A", "Competitor B"]
MODELS = ["provider/model-one", "provider/model-two"] # exact IDs from openrouter.ai/models
QUESTIONS = [
"Best fragrance-free sunscreen for oily skin in India",
"Is YourBrand sunscreen safe for sensitive skin?",
]
RUNS = 2 # answers vary, so ask each question more than once
def ask(model, question):
request = urllib.request.Request(
"https://openrouter.ai/api/v1/chat/completions",
data=json.dumps({"model": model, "messages": [{"role": "user", "content": question}]}).encode(),
headers={"Authorization": f"Bearer {os.environ['OPENROUTER_API_KEY']}",
"Content-Type": "application/json"},
)
with urllib.request.urlopen(request, timeout=120) as response:
return json.load(response)["choices"][0]["message"]["content"]
def named(text, name):
return re.search(rf"\b{re.escape(name)}\b", text, re.IGNORECASE) is not None
rows = []
for question in QUESTIONS:
for model in MODELS:
for run in range(1, RUNS + 1):
answer = ask(model, question)
rows.append({
"date": date.today().isoformat(),
"question": question,
"model": model,
"run": run,
"brand_named": named(answer, BRAND),
"competitors_named": "; ".join(c for c in COMPETITORS if named(answer, c)),
"urls_cited": "; ".join(sorted(set(re.findall(r"https?://[^\s)\]>\"']+", answer)))),
"accurate": "", # fill in by hand: yes / no / partly
"notes": "", # what was wrong, and where it probably came from
"answer": answer,
})
time.sleep(1)
with open(f"ai-brand-check-{date.today()}.csv", "w", newline="") as f:
writer = csv.DictWriter(f, fieldnames=rows[0].keys())
writer.writeheader()
writer.writerows(rows)
for model in MODELS:
answers = [r for r in rows if r["model"] == model]
print(f"{model}: {BRAND} named in {sum(r['brand_named'] for r in answers)} of {len(answers)} answers")
For the app checks, log into the same columns by hand so both sources land in one sheet:
date | question | assistant | run | brand_named | competitors_named | urls_cited | accurate | notes
A plain-text template helps keep the manual runs consistent. Paste the question exactly as written, then fill this in straight after:
Question: [paste exactly]
Assistant: ChatGPT / Gemini / Perplexity / Google AI
Signed out: yes / no
Brand named: yes / no Position in answer: 1st / 2nd / later / not named
What it said: [one line]
Correct: yes / no / partly What was wrong: [ ]
Sources cited: [domains]
Step 3: Score accuracy, mentions, and sources cited
Three numbers per assistant are enough to start:
- Mention rate: the share of answers that name you. Report it as "named in 5 of 8 runs", not a ranking. There's no position 1 in a generated answer.
- Accuracy: of the answers that name you, how many get the price, ingredients, sizes, shipping and returns right.
- Sources: which domains get cited, and how often. This is the most useful column. It tells you which review sites, forums and buying guides the assistant trusts in your category.
Sort the sources column by count. If one buying guide shows up in half the answers and you're not in it, you've found your next outreach target.
Step 4: Fix the gaps on your site and re-check
You can't edit an AI answer. You change the pages it reads, and the answer follows: sometimes within days, sometimes months later.
| What the sheet shows | Likely cause | Fix |
|---|---|---|
| A wrong fact about your product | An old page, a stale marketplace listing, or a detail missing from your site | Correct it on your product page and every listing, in the same words |
| Your structured data disagrees with the page | Product schema generated from an old field | Make the JSON-LD and the visible text come from one source |
| You're named but never linked | The model knows the brand but didn't read a page | Make sure the matching page states the answer in plain text, high up |
| A competitor wins with one cited guide | That guide is the assistant's main source | Pitch the guide's author with real product details |
| A discontinued product shows up | The old URL still resolves or is still listed elsewhere | Redirect it to the replacement and update your feeds |
Then run the same questions next month and compare the CSVs. Same questions, same columns, same number of runs. That's the whole trick.
A few honest limits: twenty questions are a sample, not the thousands of ways people ask. API results aren't what shoppers see in the apps. And a mention isn't a sale, so pair this with referral traffic from assistants and searches for your brand name.
Read the full walkthrough here: https://wolfvisibility.com/blog/ai-brand-monitoring-ecommerce
Top comments (0)