DEV Community

Ali Ahmed
Ali Ahmed

Posted on Originally published at aliahmed.si

What does an AI engineer do? A working definition, and what to look for when hiring one

An AI engineer builds products and systems on top of AI models: they connect models to data (retrieval, RAG), give them tools (agents), make their output reliable (validation, evaluation, guardrails), and run them in production at a sensible cost and latency. An ML engineer focuses more on training, fine-tuning and serving models; a data scientist on analysis, experiments and modelling for decisions. The lines blur, and good AI engineers know enough of both, but the defining skill is shipping AI features that keep working when real users and real data arrive. Here's what the job involves, and what I'd look for when hiring one.

What does the work involve?

From my own work across identity verification, financial intelligence and developer tools:

How is it different from ML engineering and data science?

Role Main focus Typical output
AI engineer Building products on models; reliability, evaluation, integration Features, agents, RAG systems, APIs
ML engineer Training, fine-tuning, serving and scaling models Models, training and serving pipelines
Data scientist Analysis, experiments, statistical modelling Insights, forecasts, decisions

In a small startup, one person often covers all three. The question is which one they're strongest at, and whether that matches your next six months.

What skills matter most?

  1. Software engineering first. AI features are mostly ordinary code: APIs, databases, queues, auth and tests. Weak engineering shows up as unreliable AI.
  2. Evaluation habits. They measure before and after changes and distrust demos. Ask how they knew a change helped.
  3. Skepticism about model output. They validate, check numbers against sources and design fallbacks.
  4. Security awareness: prompt injection, least-privilege tools, secrets and data handling.
  5. Cost and latency sense: they know where tokens go and how to cut them.
  6. Ability to read and fix other people's code, including open-source dependencies. Merged upstream PRs are good public evidence.

Interview questions for founders

  • "Tell me about an AI feature that worked in the demo and failed with real users. What broke, and how did you find out?"
  • "How would you know whether a prompt change made our assistant better or worse?" (Look for a test set and metrics.)
  • "Our agent sometimes makes up a number. How do you fix that?" (Look for verification in code, not "a better prompt".)
  • "What could go wrong if our agent reads customer emails and can send replies?" (Look for prompt injection, confirmation and permissions.)
  • "Walk me through code you wrote, and change it live." Works for spotting understanding, whoever or whatever wrote it; see my AI code detector post-mortem.

Red flags

  • Only notebooks and demos, nothing in production.
  • No way to measure quality besides "it looks good".
  • Every answer is a bigger model or a longer prompt.
  • No interest in what happens when the model is wrong.

I'm an AI engineer in Lahore, Pakistan. My experience and open-source work are on the home page and the projects page, and all my writing is on the blog.


Written by Ali Ahmed ยท Original article

Top comments (0)