A lot of engineering interview preparation focuses on solving a well-defined problem with clearly specified inputs and outputs.
But customer-facing engineering rarely starts that way.
Imagine receiving a client request like:
“We have thousands of support tickets. Can you use AI to identify the important ones and route them to the right teams?”
Before writing code, I’d want to understand:
- What actually makes a ticket important?
- What happens when the system makes a mistake?
- How do we evaluate success when labels are incomplete?
- When should an automated decision be escalated to a human?
- How would we detect problems after deployment?
The interesting engineering work is often in discovering the requirements, designing evaluations and deciding where automation should stop.
I’m exploring how to turn this kind of ambiguity into a practical engineering exercise.
For engineers who have worked in applied AI, solutions engineering or forward-deployed roles:
- Which skills are hardest to assess through conventional coding exercises?
- What real deployment or customer-integration challenges would make a good practical exercise?
- If you’ve prepared for these roles, what did existing resources fail to cover?
Please avoid sharing confidential interview questions or employer information.
I’m researching the problem and may eventually develop an educational product. Nothing is being sold or promoted here.
Top comments (1)
How well a person can actually do the job. A manual code review, or a thoroughly spec'd out, or completely ambiguous task, does not show a person's worth to the company. - Case in point, my interview process at Wasmer, I failed manual code review, I nailed the take home projects, but I failed the debrief... Yet 2 of the PRs I made during that task, got merged, because I found genuine bugs in their codebase (and they were pretty bad...). Just because I dont interview well, doesnt mean I cant work well and just because I work well, doesnt mean I can communicate well. A slack channel message is more likely to tell if you can work with someone, than a video call and review and Q&A through message, tells more about whether they can do the job, than asking them to explain their work, while trying to catch them out on ambiguous questioning.
Honestly, whatever is on the job-board. Dont bother with fake scenarios, give a real task, just liaison the feedback loop and you can see whether the person would genuinely be able to do the work. Given that it's a test, you dont rely on their work, an internal employee does the same task, directly with the customer and they grade the implementation vs theirs and the feedback loop vs theirs. That gives genuine insight into how they work and how they compare.
Honestly, they failed to cover that even for a senior position, where you have 15m LOC of open source backing your portfolio, they still think it's appropriate to ask you stupid junior task questions. We've moved past the point of manually writing code, but for some reason people on the 'cutting edge' think that AI cant do their job (they're wrong). They said 'you cant just ask AI to fix something in wasix', well I actually did, because I know how to identify the problem and I can be explicit with my prompts, so yes, 'cutting edge' still runs on ordinary code... Nothing special about it, it's standard patterns used everywhere. The delusion that their work is special, is a common trait and it's a fatal 1, because how you implement Tokio and how everyone else does, is exactly the same...