DEV Community: Cole · Tracking AI Search

How to Measure Organic AI Visibility Without Turning It Into Rank Tracking

Cole · Tracking AI Search — Sat, 15 Aug 2026 06:04:42 +0000

AI visibility is often discussed as if it were a traditional rank-tracking problem: choose a keyword, record a position, repeat every day.

That mental model is convenient, but it misses the most useful question for many founders and marketing teams:

When a buyer asks an AI assistant for solutions in my category without naming my brand, does the assistant recommend me, recommend competitors, or skip me entirely?

That is organic AI visibility. It is closer to a structured recommendation audit than a search-engine position check.

This article lays out a practical way to measure it without pretending that probabilistic AI answers behave like a stable list of blue links.

1. Start with the decision you want to understand

A useful prompt represents a real buying or evaluation decision. It should contain enough context for an assistant to make a recommendation, but it should not lead the model toward your brand.

Weak prompt:

Is Acme Analytics a good product?

This measures what the model says after you name the brand. It does not measure organic discovery.

Stronger prompt:

What are the best analytics tools for a small B2B SaaS team that needs simple activation and retention reporting?

This gives the assistant room to assemble its own shortlist. The resulting answer reveals which brands it associates with the need, how it describes them, and which criteria appear to drive the recommendation.

2. Build a small prompt set around buyer intent

You do not need hundreds of prompts for an initial snapshot. Start with 10 to 20 prompts covering a few distinct intents:

Category discovery: "What are the best tools for X?"
Use-case fit: "What should a team use to solve Y?"
Constraint-based evaluation: "Which options work for a small team, limited budget, regulated market, or technical buyer?"
Comparison intent: "What are the strongest alternatives for this type of workflow?"
Implementation intent: "How should a company set up or improve this process?"

Keep the market, language, and audience consistent. Record the exact prompts. If you later repeat the exercise, methodological consistency matters more than adding a large number of loosely related questions.

3. Classify the answer before interpreting it

For each answer, capture a small set of observable facts:

Was your brand recommended?
Which competitors were recommended?
Where did each brand appear in the answer?
What reasons, use cases, or differentiators were attached to each brand?
Were sources or citations provided?
Did the answer express uncertainty or qualify the recommendation?

A simple classification works well:

Recommended: your brand appears as a genuine solution to the unnamed need.
Mentioned: your brand appears, but not as a clear recommendation.
Competitors only: the model recommends alternatives and excludes your brand.
No usable recommendation: the answer is generic or avoids naming products.

This avoids a common measurement error: treating every textual mention as equivalent visibility.

4. Separate presence from positioning

A mention alone is not necessarily a win.

Suppose an assistant names your product but describes it as suitable only for enterprises, while your actual target customer is a five-person startup. That is visibility, but the positioning is inaccurate.

For every meaningful mention, record:

the category the model places the product in;
the audience it thinks the product serves;
the strengths and limitations it repeats;
the competitors it groups with the product;
the evidence or sources behind those claims.

This turns a vague "AI mentioned us" observation into a concrete content and positioning backlog.

5. Treat the result as a snapshot, not a permanent rank

Generative answers can vary because of model updates, sampling, prompt phrasing, geography, and changing source material. A single run should therefore be presented as a directional sample.

Avoid statements such as "we rank number three in ChatGPT." There may be no stable third position to own.

A more defensible statement is:

In this prompt set and sample, the brand was organically recommended in 4 of 12 buyer-intent questions, while two competitors appeared in 8 and 7 answers.

That description preserves the evidence and the limits of the method.

6. Convert gaps into specific improvements

The most valuable output is not a score. It is the explanation of what to improve next.

Common gaps include:

the category is unclear on the product site;
comparison pages do not answer real evaluation questions;
use-case pages lack concrete examples;
third-party reviews describe an outdated product;
the product has weak evidence for important claims;
documentation and pricing information are difficult to find;
competitors have stronger coverage on trusted, crawlable sources.

Map each gap to an asset or action. For example, if assistants repeatedly associate a competitor with agencies but never connect your product to that audience, publish a genuinely useful agency workflow page with examples, limits, and proof.

7. Repeat only when the comparison is meaningful

A later snapshot can be useful after you ship material changes: clearer positioning, new comparison content, better documentation, updated third-party coverage, or customer evidence.

Use the same prompt set and sampling method when comparing snapshots. Otherwise, changes in the test itself can look like changes in visibility.

This is also why AI visibility should not automatically be packaged as continuous daily rank tracking. Many teams first need a reliable diagnosis and a prioritized improvement plan, not another dashboard to watch.

A lightweight workflow

A practical first audit can fit in a spreadsheet:

prompt | intent | brand outcome | competitors | positioning | citations | next action

Run the prompts, save the exact answers, classify them consistently, and summarize:

recommendation coverage;
competitor frequency;
recurring reasons for inclusion or exclusion;
positioning errors;
the highest-impact next actions.

If you prefer a structured report instead of manual collection, TrackAIMentions provides a free AI visibility check focused on ChatGPT-style answers plus a Perplexity visibility sample. It is report-first: the goal is to show whether AI organically recommends your brand or competitors and what to improve next, rather than imitate a traditional SEO rank tracker.

The key principle is simple: measure the recommendation decision, preserve the evidence, and be honest about uncertainty. That produces a smaller number of metrics, but far more useful work.

How to Check If a Roof Replacement Quote Is Too High

Cole · Tracking AI Search — Thu, 04 Jun 2026 14:24:23 +0000

A roof replacement quote can be hard to judge because most homeowners only see this purchase once or twice in their lives. A $9,000 quote and a $19,000 quote can both look plausible until you compare the scope line by line.

Here is the checklist I use when reviewing a roofing estimate.

1. Start with roof size, not house size

Many quotes are built around the roof surface area, not the interior square footage of the home. Pitch, overhangs, garages, dormers, hips, valleys, and waste factor can all change the final number.

If the quote does not clearly state the measured roof area or number of squares, ask for it. Without that, it is almost impossible to compare two bids fairly.

2. Confirm the material system

A quote should not just say "shingles" or "metal roof." It should identify the material system and any major components included. For example:

3-tab asphalt shingles
Architectural shingles
Standing seam or exposed-fastener metal
Tile
Flat or low-slope membrane

The material choice changes cost a lot. It also changes warranty expectations, underlayment needs, flashing details, and installation complexity. A cheap quote may simply be pricing a different system.

3. Check tear-off and disposal

One of the biggest quote gaps is whether the contractor is removing the old roof. Tear-off, hauling, dump fees, and cleanup should be explicit.

If one quote includes full tear-off and another assumes an overlay, they are not equivalent. Overlay can sometimes lower upfront cost, but it may hide decking issues and can affect future replacement options.

4. Look for decking repair assumptions

Decking is often the unknown item. A good quote will explain whether it includes a small allowance, a per-sheet replacement price, or no decking repair at all.

This matters because the final invoice can move quickly if rotten plywood or plank decking is found after tear-off. The red flag is not necessarily a decking charge. The red flag is a quote that is vague about how decking will be handled.

5. Read the flashing, ventilation, and underlayment lines

These details are easy to miss, but they are part of what makes a roof perform. A complete estimate should mention items like:

Drip edge
Valley treatment
Step flashing and wall flashing
Pipe boots
Ice and water shield where relevant
Synthetic or felt underlayment
Ridge vents, box vents, or other ventilation approach

A quote that only lists shingles and labor may be too thin to evaluate.

6. Ask who handles permits

Permit rules vary by city and county. Some quotes include permit handling, some pass permit fees through to the homeowner, and some leave the topic out completely.

That omission can create confusion later. If permits are required in your area, the estimate should say who pulls them and whether fees are included.

7. Compare the price against a local range

National averages are useful, but local labor, disposal, access, code requirements, and weather exposure matter. A coastal Southern California roof will not always price like a similar roof in inland Florida.

I built Roof Cost Data as a free homeowner research tool for this exact comparison problem. You can estimate a rough planning range with the roof replacement cost calculator, then sanity-check a written bid with the roof quote checker.

For local examples, the site also has city pages such as Long Beach roofing cost estimates, where coastal exposure, permits, material choices, and roof size assumptions can change the range.

8. Do not treat the lowest quote as automatically best

A low quote can be legitimate. It can also be missing tear-off, permit handling, ventilation work, flashing replacement, decking allowances, cleanup, or warranty detail.

A high quote can be padded. It can also include better materials, more difficult access, full tear-off, damaged decking allowances, or stronger warranty support.

The better question is: what exactly is included, and can another contractor price the same scope?

Quick red flags

Be careful if a roof quote:

Does not state roof size or squares
Uses vague material names
Leaves tear-off unclear
Has no decking repair terms
Ignores flashing or ventilation
Says nothing about permits
Pushes for a same-day decision
Has a very large deposit requirement
Refuses to put scope details in writing

Roof Cost Data is not a roofing contractor and does not perform roofing work. The goal is to help homeowners understand the quote before they decide who to hire.