Most startups don't need an AI feature yet.
You don't need "AI-powered" in the headline. You need a faster onboarding, a clearer offer, and a product that does one job well on a Tuesday afternoon.
This is easy to say and hard to follow, because the pressure is real. Customers ask about it. Investors ask about it. Competitors put a sparkle icon on a button. A model can draft, summarize, classify, and extract in a way that would have looked like science fiction a few years ago. That doesn't mean your product's first problem is the absence of a model.
We're not anti-AI. We use it when it earns a place. The test is simple: if you removed the model, would the product still be worth opening? If the answer is no, the AI isn't a feature. It's a disguise.
When a model actually earns a place
AI is a good tool when three things are true at once.
The task is frequent, and similar each time. Drafting a reply in a known tone. Turning a long document into a short summary for a person who has to read many of them. Classifying inbound messages into a small set of buckets you already understand. The work repeats. A person already knows what "good enough" looks like.
A pretty-good answer is actually useful. This is the part teams skip. If the output is a nice-to-have draft that a human will edit, pretty-good is a gift. If the output is a medical decision, a legal conclusion, a payment, or a deletion, pretty-good can be a problem. "Mostly right" is not a universal product quality.
You can catch the times it's wrong. A person reviews the draft. A source is cited and can be opened. A low-confidence result becomes a regular form. A dangerous action needs confirmation. If a wrong answer disappears into an automatic workflow, you don't have a feature. You have a future incident.
When those conditions aren't there, you still have options: a template, a checklist, a search box, a rule, a human on the other side of an inbox. Those are allowed. They are often better.
The five-question test
Run this before you add "AI-powered" to anything.
1. Is the task frequent, and similar each time?
If the work happens once a quarter, or it's different every time in a way you can't describe, a model will be an expensive novelty. Write down the task in a sentence, including how often a real user would do it. "Summarize this week's support threads" is a task. "Be a smart assistant for the whole product" is not.
2. Is a "pretty good" answer actually useful?
Imagine the output is slightly off — a missed detail, a confident tone around a shaky fact, a category that's close but not right. Does the user still save time? Or does the mistake create more work than it removes? If the product needs to be exact, you either need a tighter scope, a mandatory review, or a tool that isn't a model.
3. Do you have the data — or the documents — it would need?
Models don't know your product because you added a chat widget. They know what you give them: help articles, policies, the customer's own files, structured records you already store. If that material is thin, outdated, or locked in someone's head, the feature will hallucinate around the gap. Start with the corpus. If you wouldn't be happy for a new teammate to learn the product from those documents, you won't be happy with the answers either.
4. What happens when it's wrong? Can a person catch it?
Write the failure case before you write the prompt. Who sees the output? Can they tell what it was based on? Can they undo it? Is there a path to a human? "We'll add guardrails" is not an answer until you can describe them. If you can't answer this question, don't ship it yet.
5. Would a simpler rule, template, or search box do the job?
A lot of "AI" requests are search requests in a costume. People want to find the right policy, the right record, the right example. A good search bar, a short FAQ, or a template library will sometimes get them there with less risk and less ceremony. If the simple version would work, ship the simple version. You can always add a model later. You can't always undo a bad one.
If you can't answer 3 and 4, stop. Those two are the ones that turn a demo into a liability.
If you can't answer 3 and 4
The usual pattern is a demo that looks wonderful on stage. A founder types a clean question. The model answers in the brand voice. Everyone nods. Then a real customer asks something slightly sideways, or the docs don't cover the edge, and the product invents a policy you don't have.
That's not a reason to never use models. It's a reason to treat the demo as a demo. Before you put it in the path of real work, you need a source of truth, and you need a way to be wrong safely.
Sometimes the honest next step is not "build the assistant." It's "write the docs," "clean the FAQ," or "make the ordinary flow faster." A confusing first run, a slow signup, or a page that doesn't say who it's for will not be rescued by a chat panel in the corner. The model will just become another place to get lost.
A useful question for the roadmap: what would you fix first — the model, or the path to the first useful moment?
A simpler first version: a docs-based FAQ bot
If you do have a good reason to add a model, start smaller than a general agent.
A docs-based FAQ bot is not magic. It's four boxes:
- Your docs — the pages, policies, and help articles you are willing to stand behind.
- Chunks — those documents, split so they can be found.
- Retrieval — a way to pull the few passages that are actually relevant to the question.
- An answer with sources — a response that points back to the passages, so a person can check.
Start with the docs, not the model. If it isn't in the docs, the bot should say so. That single behavior — declining to invent — is more valuable than a warmer tone.
This shape has limits. It won't take actions in your app unless you carefully add them. It won't know a customer's private data unless you deliberately connect that data, with permissions and a plan for mistakes. It won't replace a clear help center. In many products, that's a feature. You want a tool that answers from a known set of pages, not a personality that improvises.
The same five questions still apply. Frequent, similar questions? Pretty-good plus a source link is useful? The documents exist and are current? A person can open the source and catch a miss? Would a search box have been enough for now? If search would have been enough, you may still want search. The bot can come after the docs are worth reading.
Takeaways
- Don't put "AI-powered" in the headline if the product isn't worth opening without the model.
- Use a model when the task is frequent and similar, a pretty-good answer is useful, and a person can catch mistakes.
- Run the five questions: frequency, "pretty good," data or documents, failure behavior, and whether a simpler tool would do.
- If you can't answer the data question and the failure question, don't ship it yet.
- A confusing signup, a vague offer, or a weak first run will not be fixed by a chat widget.
- If you start, start with a docs-based bot: docs, chunks, retrieval, and answers with sources. If it isn't in the docs, say so.
Which question is hardest on your product?
We're nothink, a small team that designs and builds products for startups — more at nothink.site.
Top comments (0)