Hiring someone to build an AI feature is hard to judge from a portfolio, because demos look good by construction. The questions below separate people who have shipped measurable work from people who sell the word "AI". For each one I list what a good answer sounds like and what should worry you. I write this as someone who sells this kind of work, so treat it as a checklist you can apply to me as well.
1. "How will we know it works?"
Good: a number, a test set and a method, agreed before building: "X% handled automatically at Y% accuracy on a labelled sample from your data".
Worry: "it will be very accurate", or accuracy quoted from a generic benchmark with no link to your data.
2. "What happens when it is wrong or unsure?"
Good: a described human path: confidence threshold, review queue, fallback, who handles what and how fast.
Worry: "the model rarely makes mistakes", or no plan for exceptions.
3. "What do you need from us, and when?"
Good: a concrete list: sample data, a process owner, someone to label examples, access. With dates.
Worry: nothing needed, because "AI will figure it out".
4. "Where will our data go?"
Good: a clear answer on which services see it, retention terms, whether personal data is redacted, and what is stored in logs. Willingness to do this in writing.
Worry: vague reassurance, or no idea which provider terms apply.
5. "What exactly will you deliver, and what is out of scope?"
Good: a short, written scope with explicit exclusions and acceptance criteria tied to question 1.
Worry: an open-ended "AI solution", or a scope that grows without a change process.
6. "What will it cost to run after launch?"
Good: an estimate of per-item or monthly cost (API, hosting, monitoring) and who pays for tuning as data changes.
Worry: only the build cost is discussed, or running cost is "negligible" with no numbers.
7. "Can I see something you built, and what did you measure?"
Good: a real example with the method and numbers, including the limits, or an open demo with its caveats stated. An honest "this was a public benchmark, not a client result" is a good sign.
Worry: unnamed "Fortune 500 clients", screenshots without metrics, results that are suspiciously perfect.
8. "What if we stop after the pilot?"
Good: a pilot with a decision rule agreed upfront (scale, change, stop), and you keep the code, the data and the documentation.
Worry: lock-in, an unclear ownership of the result, or pressure to commit to a long contract before any measured pilot.
Small signals that count
- They ask about your process before talking about models.
- They say what AI is not good for in your case.
- They propose the simplest thing that could work.
- They are comfortable writing the acceptance number down.
If you get a few of these right, you already know more than most buyers do.
Takeaways
- Insist on a measurable target and a method before work starts.
- Ask about errors, data, running cost and exit, not only features.
- Prefer evidence with limits over perfect-looking claims.
- Start with a short, measured pilot.
Portfolio and contacts: @Kotenev01_bot.
I write about practical, measurable AI adoption for business: demos on test data, no invented case studies. Telegram channel: @kotenev01_en
Top comments (0)