An AI engineer builds products and systems on top of AI models: they connect models to data (retrieval, RAG), give them tools (agents), make their output reliable (validation, evaluation, guardrails), and run them in production at a sensible cost and latency. An ML engineer focuses more on training, fine-tuning and serving models; a data scientist on analysis, experiments and modelling for decisions. The lines blur, and good AI engineers know enough of both, but the defining skill is shipping AI features that keep working when real users and real data arrive. Here's what the job involves, and what I'd look for when hiring one.
What does the work involve?
From my own work across identity verification, financial intelligence and developer tools:
- Computer vision and OCR in production: detecting, reading and verifying identity documents at ShuftiPro. See how KYC verification works.
- Retrieval and RAG: chunking, embeddings, search and evaluation. See evaluating RAG.
- Agents and tools: tool design, errors, permissions and protocols such as MCP. See tool calling explained and an OAuth 2.1 MCP server.
- Multi-agent systems: at Kryptomind, pipelines for financial intelligence. See single agent or multi-agent.
- Reliability: structured output, validation and fallbacks. See reliable JSON from LLMs.
- Security: prompt injection, authentication and audit trails. See prompt injection from scraped data and localhost is not authentication.
- Cost and operations: gateways, routing, caching and monitoring. See what an LLM gateway is and cutting LLM costs.
- Upstream fixes: when a dependency breaks, fixing it at the source, as in my contributions to LiteLLM and Node.js.
How is it different from ML engineering and data science?
| Role | Main focus | Typical output |
|---|---|---|
| AI engineer | Building products on models; reliability, evaluation, integration | Features, agents, RAG systems, APIs |
| ML engineer | Training, fine-tuning, serving and scaling models | Models, training and serving pipelines |
| Data scientist | Analysis, experiments, statistical modelling | Insights, forecasts, decisions |
In a small startup, one person often covers all three. The question is which one they're strongest at, and whether that matches your next six months.
What skills matter most?
- Software engineering first. AI features are mostly ordinary code: APIs, databases, queues, auth and tests. Weak engineering shows up as unreliable AI.
- Evaluation habits. They measure before and after changes and distrust demos. Ask how they knew a change helped.
- Skepticism about model output. They validate, check numbers against sources and design fallbacks.
- Security awareness: prompt injection, least-privilege tools, secrets and data handling.
- Cost and latency sense: they know where tokens go and how to cut them.
- Ability to read and fix other people's code, including open-source dependencies. Merged upstream PRs are good public evidence.
Interview questions for founders
- "Tell me about an AI feature that worked in the demo and failed with real users. What broke, and how did you find out?"
- "How would you know whether a prompt change made our assistant better or worse?" (Look for a test set and metrics.)
- "Our agent sometimes makes up a number. How do you fix that?" (Look for verification in code, not "a better prompt".)
- "What could go wrong if our agent reads customer emails and can send replies?" (Look for prompt injection, confirmation and permissions.)
- "Walk me through code you wrote, and change it live." Works for spotting understanding, whoever or whatever wrote it; see my AI code detector post-mortem.
Red flags
- Only notebooks and demos, nothing in production.
- No way to measure quality besides "it looks good".
- Every answer is a bigger model or a longer prompt.
- No interest in what happens when the model is wrong.
I'm an AI engineer in Lahore, Pakistan. My experience and open-source work are on the home page and the projects page, and all my writing is on the blog.
Written by Ali Ahmed ยท Original article
Top comments (0)