DEV Community: Alex Shcoder?

Deploying a Telegram Bot Should Not Require Becoming a DevOps Engineer

Alex Shcoder? — Wed, 22 Jul 2026 22:53:58 +0000

A Telegram bot can be surprisingly small.

The first version may contain:

one command handler;
a few API calls;
a database connection;
several environment variables;
a background loop or webhook;
a couple hundred lines of Python.

Building it can take an evening.

Keeping it online is where the project suddenly turns into infrastructure work.

You need a server, a process manager, logs, HTTPS, secrets, restarts, updates, and some way to understand why the bot stopped responding at 3 a.m.

For a production platform, that work is reasonable.

For a prototype, internal tool, weekend project, or first customer demo, it is often too much.

That gap is why I started building Deployka: a deployment service focused on Telegram bots, small MVPs, automation scripts, and AI projects.

This article is not a guide to building another Kubernetes abstraction.

It is about a simpler question:

What is the minimum deployment experience a small bot actually needs?

The code is rarely the hard part

Consider a simple Python bot:

import os

from telegram import Update
from telegram.ext import Application, CommandHandler, ContextTypes


async def start(
    update: Update,
    context: ContextTypes.DEFAULT_TYPE,
) -> None:
    await update.message.reply_text("The bot is running.")


def main() -> None:
    token = os.environ["TELEGRAM_BOT_TOKEN"]

    app = Application.builder().token(token).build()
    app.add_handler(CommandHandler("start", start))
    app.run_polling()


if __name__ == "__main__":
    main()

Locally, the workflow is straightforward:

export TELEGRAM_BOT_TOKEN="..."
python bot.py

The bot works.

But a local terminal is not a deployment environment.

The moment the bot must run continuously, a new list of questions appears:

Where should the process run?
How does it restart after a crash?
Where should the token be stored?
How can I view logs?
What happens after I update the code?
How do I know whether the process is healthy?
Do I need Docker?
Do I need a reverse proxy?
Do I need a domain and TLS certificate?
How do I avoid breaking the currently running version?

None of these questions are unusual.

The problem is that they arrive before the project has earned the complexity.

“Just rent a VPS” is only the beginning

A virtual private server is flexible and inexpensive.

It is also a blank machine.

A typical manual deployment can involve:

ssh root@server
apt update
apt install python3 python3-venv git
git clone ...
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Then the process needs to remain alive after the SSH session closes.

That usually means adding systemd, Supervisor, Docker Compose, or another process manager.

A basic systemd unit might look like this:

[Unit]
Description=Telegram bot
After=network.target

[Service]
WorkingDirectory=/opt/my-bot
ExecStart=/opt/my-bot/.venv/bin/python bot.py
Restart=always
EnvironmentFile=/opt/my-bot/.env

[Install]
WantedBy=multi-user.target

This is not particularly difficult for an experienced infrastructure engineer.

It is still operational surface area:

Linux package updates;
SSH access;
firewall configuration;
service files;
environment files;
log rotation;
disk usage;
backups;
runtime upgrades;
incident recovery.

A VPS gives you a machine.

It does not give you a product deployment workflow.

Docker helps, but it does not remove operations

Docker improves reproducibility.

A small bot can be packaged with a short Dockerfile:

FROM python:3.12-slim

WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY . .

CMD ["python", "bot.py"]

That solves an important problem: the application now has a predictable runtime.

But the container still needs somewhere to run.

Someone must still handle:

image builds;
container restarts;
environment variables;
logs;
networking;
persistent storage;
deployments;
rollbacks;
health checks.

Docker is a packaging format, not a complete hosting experience.

For many small projects, the ideal workflow is not “learn more infrastructure.”

It is:

code → configure secrets → deploy → view logs

What a small bot actually needs

A lightweight deployment platform does not need to solve every enterprise infrastructure problem.

It needs to solve the first operational problems well.

1. A predictable runtime

The platform should identify or accept the project runtime, install dependencies, and start the correct process.

For a Python bot, this may mean:

detecting requirements.txt;
installing packages;
starting the configured command;
keeping the process alive.

For a web API, the start command may be different:

uvicorn app:app --host 0.0.0.0 --port $PORT

The platform does not need to guess everything.

It does need to make the required configuration obvious.

2. Environment variables

Secrets should not live inside the repository.

A deployment flow should provide a clear place for values such as:

TELEGRAM_BOT_TOKEN
OPENAI_API_KEY
DATABASE_URL
WEBHOOK_SECRET

The important part is not only storing them.

The interface should also make it easy to see which variables are required without displaying secret values unnecessarily.

3. Logs that are easy to find

When a bot stops working, the first question is usually:

What happened?

Logs should not require another monitoring product or an SSH session.

For an early-stage project, a useful log view needs to answer:

Did the build succeed?
Did the process start?
Which error caused it to exit?
Is it restarting repeatedly?
Did the application receive a request or update?

Good logs reduce support work for both the developer and the hosting platform.

4. Restart without server access

A failed process should restart automatically when appropriate.

The developer should also be able to trigger a manual restart from the dashboard.

This sounds like a tiny feature.

It removes a large amount of friction for people who would otherwise need to connect to a server just to restart one process.

5. HTTPS when the project needs a web endpoint

Polling bots may not need a public HTTP endpoint.

Webhooks, FastAPI services, Flask apps, and small MVPs often do.

Automatic HTTPS removes an entire class of setup work:

certificates;
renewal;
reverse-proxy configuration;
domain routing.

The user should be able to move from code to a working HTTPS endpoint without becoming an Nginx administrator.

Polling and webhooks are different deployment shapes

Telegram bots commonly receive updates through long polling or webhooks.

Long polling

The process continuously asks Telegram for new updates.

This is operationally simple:

no public endpoint is required;
the process must stay alive;
only one active polling instance should normally consume updates for the bot.

The deployment platform mainly needs to run a reliable background process.

Webhooks

Telegram sends updates to an HTTPS endpoint.

This requires:

a public URL;
TLS;
request routing;
a web server;
webhook registration;
a process listening on the platform-provided port.

A webhook bot looks more like a small web service.

The important product decision is not to force every bot into the same deployment model.

The platform should support the shape of the application instead of requiring the application to be redesigned around the hosting platform.

Templates are useful when they remain transparent

A template can reduce the time from an idea to a running project.

Examples include:

a Python Telegram bot;
a FastAPI application;
a scheduled Python script;
a parser that runs on a schedule;
an AI agent connected to Telegram;
an automation service such as n8n.

Deployka publishes a growing set of deployment templates and documentation for these project types.

But templates become dangerous when they hide too much.

A useful template should make clear:

what code will run;
which environment variables are required;
which process starts;
whether storage is persistent;
whether the project exposes a web endpoint;
which external services it contacts.

“One click” should remove repetitive setup.

It should not remove the developer’s understanding of the application.

AI projects make deployment friction more visible

A basic AI bot may depend on several services:

Telegram
   ↓
Bot process
   ↓
Model API
   ↓
Database or vector store
   ↓
Logs and monitoring

The first prototype may only need two environment variables:

TELEGRAM_BOT_TOKEN
OPENAI_API_KEY

Later versions may add:

a database;
a queue;
scheduled jobs;
file processing;
embeddings;
browser automation;
multiple workers.

It is easy to overbuild the infrastructure before users have validated the product.

The first deployment system should make the simple version easy while leaving a path to a more advanced architecture later.

A prototype hosting platform does not need to replace every cloud provider.

It needs to help the developer reach the point where a more sophisticated setup is justified.

The goal is not “no DevOps forever”

“No DevOps” is useful product language, but technically every hosted application has operations somewhere.

Someone still manages:

compute;
networking;
builds;
runtime isolation;
restarts;
security updates;
capacity;
failures.

The real goal is different:

The application developer should not need to manage infrastructure that does not differentiate the product.

A founder building a Telegram assistant should spend time on:

the conversation flow;
reliability of answers;
onboarding;
user feedback;
pricing;
retention.

They should not spend the first weekend configuring log rotation.

As the product grows, its infrastructure requirements may grow too.

That is normal.

The deployment experience should match the current stage of the project.

When a simple platform is the wrong choice

A lightweight hosting service is not the best fit for every workload.

You may need a more advanced platform when the project requires:

multiple tightly coordinated services;
private networking;
custom load balancing;
dedicated GPUs;
strict regional deployment;
complex persistent storage;
high availability across regions;
advanced autoscaling;
enterprise access controls;
custom observability pipelines;
regulated infrastructure requirements.

In those cases, a larger PaaS, managed container platform, or cloud provider may be more appropriate.

The useful question is not:

Which platform is the most powerful?

It is:

Which platform removes the most work without hiding capabilities this project actually needs?

A practical deployment checklist

Before deploying a Telegram bot or small AI service, write down:

Runtime

language and version;
dependency file;
start command;
whether the process is a worker or web service.

Configuration

required environment variables;
secret rotation plan;
external API dependencies.

Networking

polling or webhook;
required public endpoint;
expected port;
custom domain requirements.

State

whether files must persist;
database requirements;
backup expectations.

Operations

where logs appear;
restart behaviour;
health checks;
deployment and rollback process.

Limits

expected traffic;
memory requirements;
CPU-intensive tasks;
long-running jobs;
API rate limits.

This checklist is intentionally small.

Its purpose is to prevent surprises without turning a prototype into an infrastructure project.

What I am building with Deployka

Deployka is focused on a narrow deployment experience:

Launch bots, MVPs, automation scripts, and AI projects without manually operating a server.

The current product is built around a straightforward workflow:

upload or connect code
        ↓
configure variables
        ↓
build and deploy
        ↓
receive HTTPS when needed
        ↓
view logs and restart

The dashboard is designed around the actions a small project needs most often:

deploy;
inspect logs;
restart;
update configuration;
understand whether the project is running.

The platform also includes starter templates for common bot, automation, and AI workloads.

The objective is not to become the most complex cloud platform.

It is to make the first reliable deployment much less intimidating.

Final thought

The first version of a bot should be allowed to remain small.

Its deployment process should not require a second project consisting of shell scripts, service files, reverse proxies, and server maintenance.

Developers should still understand how their application runs.

They should still know where data goes, how secrets are handled, and what happens when the process fails.

But they should be able to learn those things through a clear product workflow rather than by assembling an operations stack from scratch.

A Telegram bot may contain only a few hundred lines of code.

Getting it online should feel proportional to the thing being deployed.

You can explore the current platform, templates, and documentation at Deployka.

Disclosure: I used an AI writing assistant to help structure and edit this article. The product decisions, technical examples, conclusions, and final review are my own.

Before You Ask If Your AI Product Complies, Build an AI System Brief

Alex Shcoder? — Wed, 22 Jul 2026 22:38:39 +0000

Most AI teams begin compliance work with the wrong question:

Does our product comply with the EU AI Act?

It sounds reasonable. But a lawyer, compliance specialist, enterprise customer, or assessment tool cannot answer it responsibly without first understanding what the product actually does.

They need to know:

what the AI system is intended to do;
who uses it;
who is affected by its outputs;
what decisions it influences;
which models and vendors are involved;
what data it processes;
where human review exists;
what evidence the company can provide.

Without those facts, “Are we compliant?” is not yet a legal question.

It is an incomplete product-discovery task.

Before building a large compliance program, create a concise AI system brief.

This article is for general informational purposes and is not legal advice.

Compliance starts with product facts

Companies often describe themselves as having “an AI product.”

In practice, one SaaS application may contain several separate AI-enabled systems or features:

a customer-facing chatbot;
a summarisation tool;
a recommendation engine;
a fraud-risk score;
a document classifier;
a candidate-ranking feature;
an internal support assistant;
an agent that can update records or send messages.

These features may have different purposes, users, risks, and technical dependencies.

A chatbot that answers product questions is not necessarily analysed in the same way as a system that ranks applicants for employment.

Even when both features use the same underlying model, their legal and operational context can be very different.

The first useful step is therefore not assigning one compliance label to the entire company.

It is identifying the individual AI systems inside the product.

Start with intended purpose

The intended purpose is one of the most important fields in an AI system brief.

Avoid vague descriptions such as:

We use AI to improve productivity.

That does not explain what the system actually does.

A better description would be:

The system analyses incoming customer-support tickets, assigns a suggested category, and drafts a proposed response for review by a support agent.

That statement reveals several important facts:

the system analyses customer communications;
it classifies tickets;
it generates text;
a human is expected to review the output;
the result supports a business process rather than independently completing it.

Compare it with:

The system analyses job applications, assigns each candidate a suitability score, and determines which candidates are shown to a recruiter first.

Both products may be described as “AI assistants.”

But the second system influences an employment-related process involving natural persons.

The label “AI assistant” tells us very little.

The intended purpose tells us much more.

The AI Act’s official Article 6 classification rules focus on the system’s intended purpose, the use case, and whether the output materially influences decision-making.

Identify the people involved

An AI system brief should distinguish between at least three groups.

Users

The people who directly operate or interact with the system.

Examples:

support agents;
recruiters;
administrators;
consumers;
developers.

Affected people

The people whose opportunities, access, treatment, or experience may be influenced by the output.

Examples:

job applicants;
borrowers;
students;
patients;
employees;
customers.

Responsible people

The people expected to review outputs, handle incidents, approve changes, or respond to complaints.

These groups are not always the same.

A recruiter may use a ranking tool, job applicants may be affected by it, and an HR manager may be responsible for oversight.

Documenting these relationships makes later risk and governance discussions much more concrete.

Record what the output influences

Do not stop at describing what the model produces.

Document what happens next.

For example:

The system produces a score from 0 to 100.

That is technically correct but incomplete.

A more useful explanation is:

The score determines the order in which applications are reviewed. Candidates below a threshold are not automatically rejected, but recruiters rarely review them under the current workflow.

The second version explains the practical influence of the system.

Ask:

Is the output informational?
Is it a recommendation?
Does it rank or score people?
Does it determine eligibility?
Does it automatically trigger an action?
Can a person override it?
Does the reviewer realistically have enough time and context to evaluate it?
What happens when the model is uncertain?

A system may be described as “advisory” while still having a major practical effect on decisions.

The workflow matters as much as the interface label.

Write down your role hypothesis

The AI Act assigns responsibilities to different operators, including providers, deployers, importers, and distributors.

The official definitions are available in Article 3 of the AI Act.

A company that uses a third-party model is not automatically “only a deployer.” The analysis may depend on how the resulting system is developed, branded, placed on the market, and used.

Your brief should therefore contain a role hypothesis rather than a confident label with no explanation.

For example:

Potential role:
Provider of the customer-facing AI system

Confidence:
Medium

Reason:
We define the intended purpose, integrate the model into our
application, and offer the resulting system under our own name.

Missing facts:
Whether the customer materially changes the system's intended purpose.

Legal confirmation required:
Yes

Documented uncertainty is more useful than an unexplained green checkmark.

Map the technical dependencies

Most AI SaaS products depend on several external components.

A system brief should identify:

model providers;
hosting providers;
vector databases;
analytics tools;
observability platforms;
data-labelling services;
document-processing services;
speech or image APIs;
open-source models and libraries.

For each dependency, record:

what it provides;
what data is sent to it;
where processing occurs;
how long data is retained;
whether customer data may be used for training;
how model updates are communicated;
what technical documentation is available;
what happens if the vendor changes or retires a model.

Using a well-known model provider does not answer questions about your own product.

Your application still has its own:

intended purpose;
interface;
integrations;
users;
data flows;
safeguards;
failure modes.

The complete system is what customers and affected people experience.

Capture the data flow

A useful data-flow description does not need to begin as a complex architecture diagram.

A simple table may be enough:

Stage	Data	Destination	Purpose	Retention
User input	Support message	Application server	Create ticket	12 months
Model request	Ticket text	Model API	Draft response	Provider-specific
Logging	Prompt and output	Observability platform	Debugging	30 days
Review	Draft and edits	Internal dashboard	Human approval	12 months

The goal is to make hidden processing visible.

This often reveals important questions:

Are personal details included in model requests?
Are prompts stored by external providers?
Can sensitive fields be removed?
Are logs accessible to too many employees?
Does deletion cover every vendor?
Can the company reproduce which model produced an output?
Can the company identify which system version was active during an incident?

These questions matter for privacy, security, procurement, debugging, and incident response even before a specific AI Act obligation is confirmed.

Document human oversight honestly

Many product descriptions contain the phrase:

A human is always in the loop.

That statement is often too vague to be useful.

A system brief should answer:

Who is the responsible human?
At which point do they review the output?
What information do they see?
Can they reject or modify the result?
Are they trained to identify failures?
Are overrides recorded?
What happens under high workload?
Can an affected person request human review?
Who handles escalations?

A confirmation button does not automatically create meaningful oversight.

If employees approve hundreds of outputs per hour without enough context to evaluate them, the formal presence of a human may not reflect the real workflow.

Describe what happens in practice, not only what the interface allows in theory.

For high-risk AI systems, human-oversight requirements are addressed specifically in Article 14.

Build a lightweight evidence pack

Compliance is not only about saying that controls exist.

It is about being able to show them.

A lightweight evidence pack may include:

the AI-system inventory;
a one-page brief for each system;
model and vendor inventory;
intended-purpose description;
data-flow description;
user disclosures;
human-oversight procedure;
risk and limitation notes;
testing records;
logging approach;
incident and escalation process;
unresolved questions for legal counsel.

Not every item is a universal legal requirement for every AI system.

That distinction matters.

The AI Act sets specific requirements for high-risk systems, including:

A lower-risk product may still maintain similar materials because they help with enterprise procurement, security, customer trust, debugging, or future readiness.

Label each item in the evidence pack as one of the following:

Confirmed legal requirement
Recommended readiness practice
Customer or procurement requirement
Requires legal confirmation

This prevents a readiness document from pretending to be a legal opinion.

Prepare for transparency at the product level

Article 50 contains transparency obligations for certain AI systems, including some systems that interact directly with people and some systems that generate or manipulate content.

According to the Commission’s official AI Act implementation timeline, these transparency obligations apply from 2 August 2026.

For a product team, preparation may involve questions such as:

Does the user know they are interacting with AI?
Is the disclosure shown before or during the first interaction?
Is it clear on mobile?
Is generated content labelled where required?
Does the interface overstate human involvement?
Does the disclosure remain accurate when the workflow changes?
Is the information accessible and understandable?

This is not only a policy-document task.

It is a product-design task.

A disclosure hidden in a long terms-of-service page may not reflect what the user actually experiences during the interaction.

Include known uncertainty

An AI system brief should not pretend that every classification question has already been resolved.

A useful section might look like this:

Potential classification:
Possible Annex III high-risk use case

Confidence:
Low

Reason:
The system is used in an employment workflow, but it may perform
only a preparatory task and does not automatically reject candidates.

Missing facts:
How recruiters use the ranking in practice and whether the output
materially influences who advances.

Legal confirmation required:
Yes

This gives a lawyer or specialist something concrete to examine.

It also prevents internal assumptions from slowly turning into “facts” merely because they appeared in an old spreadsheet.

A one-page AI system brief template

A lightweight brief can contain the following sections.

1. System identity

system or feature name;
owner;
version;
current status;
last review date.

2. Intended purpose

what the system does;
what it is not intended to do;
prohibited or unsupported uses.

3. Users and affected people

direct users;
affected groups;
responsible reviewers.

4. Output and influence

type of output;
downstream decisions;
automation level;
override process.

5. Models and vendors

external providers;
model versions;
infrastructure dependencies;
known update process.

6. Data

input data;
personal or sensitive data;
storage and retention;
processing locations.

7. Oversight

responsible human;
review points;
escalation process;
appeal or correction process.

8. Evidence

testing records;
logs;
policies;
system documentation;
incident history.

9. Preliminary hypotheses

likely operator role;
possible risk category;
possible transparency obligations;
unresolved questions.

10. Review history

reviewer;
date;
product changes since the previous review.

This document does not prove compliance.

It creates the factual foundation needed for a serious assessment.

Automated tools can help without inventing certainty

An automated checker can help a team:

organise product facts;
identify missing information;
surface potentially relevant provisions;
prepare questions for counsel;
create a preliminary readiness checklist.

It should not claim to provide a definitive legal classification from a few form fields.

The European Commission provides an official beta EU AI Act Compliance Checker. Its disclaimer states that the results are informational, are not legal advice, and do not represent the Commission’s assessment of a company’s situation or obligations.

That is the right model for responsible tooling.

A useful system should expose assumptions, uncertainty, and missing facts rather than hide them behind a confident score.

The best legal handoff begins before the legal call

Compare these two messages.

Message A

We use AI. Can you tell us whether we comply with the AI Act?

Message B

We provide a candidate-ranking feature used by EU employers. The ranking determines which applications recruiters review first, but it does not automatically reject candidates. We use a third-party model, store outputs for 30 days, and require recruiter confirmation. Here is our intended purpose, data flow, oversight process, vendor list, and the five classification questions we have not resolved.

The second message does not replace legal advice.

It makes legal advice more efficient.

The lawyer can focus on questions that require legal interpretation instead of spending the first part of the engagement reconstructing the product from screenshots, Slack messages, sales copy, and incomplete technical explanations.

Final thought

AI compliance does not begin with a checkbox.

It begins with a clear description of the system.

Before asking whether an AI product complies, document:

what it does;
who it affects;
what decisions it influences;
which vendors and models it relies on;
how people review its outputs;
what evidence exists;
what remains uncertain.

That brief remains useful even when the final legal conclusion changes.

It can support:

legal review;
enterprise procurement;
investor diligence;
security assessments;
product governance;
incident response.

The goal is not to replace a lawyer.

The goal is to give the lawyer — and your own team — something concrete to review.

About the project

I am building ActBrief, a project that helps AI teams organise product facts, documentation gaps, effective dates, and questions for qualified counsel.

ActBrief produces preliminary readiness material. It does not provide legal advice or a definitive compliance determination.

Disclosure: I used an AI writing assistant to help structure and edit this article. The product observations, source review, conclusions, and final editorial decisions are my own.

Why One AI Answer Is Not a Ranking

Alex Shcoder? — Wed, 22 Jul 2026 22:06:02 +0000

When I ran the first AI visibility test for an established software brand, the result looked almost perfect.

The brand appeared in every unbranded buyer prompt.

At first glance, the conclusion seemed obvious:

The brand had 100% AI visibility.

But the result looked too clean.

So I repeated the same prompts.

The numbers changed.

Across repeated runs, the average mention rate fell to 65%, the recommendation rate fell to 59%, and the brand was consistently recommended in only 11 of 18 unbranded buyer prompts.

The more honest result was not 100%.

It was:

61% stable recommendation coverage.

That experiment changed how I think AI visibility should be measured.

A single AI answer can be useful.

It is not a ranking.

Why AI answers are unstable

Traditional search gives marketers a relatively familiar object to measure.

A page has a position. That position may change, but there is still a search-results page that can be checked repeatedly.

AI answers are different.

The response may depend on:

the wording of the question;
the model or assistant being used;
whether web search is available;
the sources retrieved during that request;
the date and location;
previous conversation context;
normal model variation.

Ask the same question twice and you may get:

a different order of products;
a different top recommendation;
new competitors;
fewer products;
different sources;
or no citations at all.

This does not make AI visibility impossible to measure.

It means the unit of measurement cannot be a single screenshot.

Buyer prompts are not keywords

Another mistake is treating every question about a category as equivalent.

A buyer looking for code-review software might ask:

What are the best AI code review tools?

But they might also ask:

How can we catch bugs before merging a pull request?

Or:

What is an affordable PR review tool for a small engineering team?

Or:

What are the alternatives to CodeRabbit?

These prompts represent different stages of awareness.

Category prompts

The buyer already understands the product category.

Problem prompts

The buyer has a problem but may not know which type of product solves it.

Segment prompts

The buyer adds constraints such as company size, budget, workflow, or technical stack.

Comparison prompts

The buyer is already evaluating known products.

A brand can perform well in category prompts while remaining invisible in problem-oriented discovery.

That distinction matters more than a single overall score.

Branded and unbranded visibility are different

Consider these two prompts:

Best AI code review tools

and:

Is CodeRabbit worth it?

The second question already contains the brand.

A mention is expected.

That result tells us how an AI system describes a company when the buyer already knows it.

It does not tell us whether the company will be discovered by a buyer who has never heard of it.

This is why branded and unbranded prompts should be reported separately.

Branded visibility helps reveal:

reputation;
objections;
comparisons;
perceived strengths;
perceived weaknesses.

Unbranded visibility helps reveal:

category discovery;
problem discovery;
new customer acquisition;
whether a brand enters the consideration set.

For most growth teams, unbranded visibility is the more valuable test.

Mentioned does not mean recommended

AI visibility tools often combine several outcomes into one number.

But there is a major difference between these statements:

CodeRabbit is one of several available tools.

and:

CodeRabbit is the best option for a small team that wants fast pull-request feedback.

The first is a mention.

The second is a recommendation.

There is another distinction:

Greptile may be a better option for teams that need deeper repository context.

In this answer, CodeRabbit may still be discussed, but another brand wins the recommendation.

A useful AI visibility analysis should therefore separate:

missing;
mentioned;
recommended;
top recommendation;
branded discovery;
unbranded discovery;
stable and unstable results.

Without these distinctions, a brand may appear highly visible while consistently losing the actual buyer recommendation.

What repeated runs revealed

For the pilot, I tested 24 buyer prompts related to AI code-review products.

The prompts covered four groups:

category discovery;
segment and constraint;
problem-oriented discovery;
comparison and alternatives.

Each prompt was repeated three times under the same documented conditions.

The first single-run result suggested near-perfect visibility.

The repeated result was more nuanced:

average unbranded mention share: 65%;
average unbranded recommendation share: 59%;
consistently recommended in at least two of three runs: 11 of 18 unbranded prompts;
stable recommendation coverage: 61%.

The brand was clearly visible.

But its visibility was not universal or perfectly stable.

That is a much more useful conclusion than “100% visibility.”

Stable coverage is more useful than a screenshot

Imagine two brands.

Brand A

It appears in 15 of 20 prompts during one run.

When the prompts are repeated, only six recommendations remain consistent.

Brand B

It appears in 11 of 20 prompts during one run.

Ten of those recommendations remain consistent across repeated runs.

A single-run report would favour Brand A.

A consistency-first report would favour Brand B.

For a founder or marketer, Brand B probably has the stronger position.

This is why the metric I now care about most is:

Stable recommendation coverage: the percentage of buyer prompts where a brand is recommended consistently across repeated runs.

It does not remove uncertainty.

It makes the uncertainty visible.

Different AI engines represent different discovery environments

There is no single universal AI search result.

Different AI assistants can return different answers to the same buyer prompt.

They may rely on different:

models;
search systems;
sources;
ranking signals;
answer formats;
citation behaviour.

A company might be strongly recommended by one engine and rarely mentioned by another.

This gives AI visibility at least three dimensions.

Prompt coverage

In how many buyer situations does the brand appear?

Repeatability

Does the same engine produce a similar result across repeated runs?

Cross-engine agreement

Do different AI systems recommend the same company?

These dimensions should not be hidden inside one unexplained score.

A better report shows where the systems agree, where they disagree, and where a result is volatile.

Citations reveal why competitors may be winning

Brand mentions show what appeared in an answer.

Citations can reveal which sources influenced the answer.

A competitor may be repeatedly supported by:

documentation;
independent comparison pages;
review platforms;
GitHub repositories;
industry publications;
Reddit discussions;
customer stories;
category roundups.

This can expose a source gap.

The problem may not be that an AI model dislikes a product.

The product may simply be missing from the sources repeatedly used to explain the category.

That leads to a more practical question:

Where does AI find convincing information about competitors, and where is our brand absent?

The answer can suggest concrete work:

create a clearer use-case page;
publish a comparison page;
improve documentation;
add customer evidence;
answer problem-oriented buyer questions;
earn inclusion in relevant third-party sources;
repeat the test after the changes.

AI visibility should be treated as an experiment

The wrong workflow is:

Run one prompt → get a score → celebrate or panic.

A better workflow is:

Define a representative set of buyer prompts.
Separate branded and unbranded discovery.
Record a baseline.
Repeat prompts to identify unstable results.
Compare multiple AI engines.
Find consistently lost buyer intents.
Review which sources support competitor recommendations.
Make one specific change.
Repeat the same test.
Measure directional movement.

This is closer to experimentation than traditional rank tracking.

There is no guarantee that a brand will hold a fixed position inside an AI-generated answer.

But it is still possible to observe whether visibility becomes:

broader;
more consistent;
stronger across engines;
better supported by relevant sources.

The most valuable result is often a lost prompt

A company may already appear for:

Best AI code review tools

But remain absent for:

How can a growing engineering team reduce manual review workload?

The second prompt may represent a larger opportunity.

The buyer has a problem but has not yet chosen a category or vendor.

When a brand repeatedly disappears from these prompts, the result can lead to a useful action:

publish content around the problem;
explain the workflow before introducing the product;
create material for a specific team size;
clarify how the product differs from alternatives;
improve external evidence.

This is more actionable than being told that an “AI visibility score” increased from 42 to 47.

What teams should measure

A useful AI visibility report should answer:

Where are we mentioned without the buyer naming us?
Where are we genuinely recommended?
Where are we the first recommendation?
Which results remain stable across repeated runs?
Which engines favour our competitors?
Which buyer intents do we consistently lose?
Which sources are repeatedly cited?
What change should we test next?

The goal is not to manufacture a permanent AI ranking.

The goal is to replace random screenshots with repeatable evidence.

What this approach still cannot prove

Repeated testing makes the result more defensible, but it does not eliminate uncertainty.

A visibility snapshot cannot prove that:

every user will receive the same answer;
an AI assistant will keep recommending the same brand;
citations directly caused a recommendation;
higher visibility will automatically produce revenue;
a content change will improve every engine equally.

It is still a sampled observation.

The purpose of repetition is not to create false certainty.

It is to identify which results appear stable enough to investigate and which ones may have been one-off variations.

Final thought

For years, marketers asked:

Where do we rank in Google?

The equivalent AI-search question should not be:

What is our ChatGPT rank?

A more useful question is:

Across the buyer questions that matter, how consistently do AI systems recommend us—and where do competitors win instead?

One AI answer cannot answer that.

A representative prompt set, repeated observations, multiple engines, and prompt-level evidence can get much closer.

The public research snapshot and future updates are available at BuyerPrompt.

Disclosure: I used an AI writing assistant to help structure and edit this article. The experiment, results, interpretations, and final review are my own.