We run a practice platform for coding interviews with an AI assistant in the editor. That's the format Meta, Shopify, Canva, Stripe and a growing list of companies now use. From the start our assistant had one rule above all others: it never contributes an idea. Not a design, not a data structure, not a fix.
We were wrong about that rule. This is how we found out, and where we think the line actually is.
Two goals that pulled apart
We wanted two things. Users who are stuck should get help. A stuck user is a frustrated user, and frustrated users leave. And the practice has to make you do the work the interview grades. Otherwise it is not practice.
Those felt like opposites. Help means ideas. And if the ideas come from the agent, you didn't have them.
Our first answer: no ideas from the agent
So we picked a side. The assistant writes code when you say what to build. It never says what to build. If you ask it to "implement the solution", it asks you which approach you want. If you ask what the options are, it declines. Hints exist, but they come from the platform, not the assistant. You have to ask for them, they are capped per stage, and the final stage has none.
The reasoning looked solid. Stripe interviewers ask candidates why they trust code they did not write. An agent that volunteers a design hands you code you can't defend. Remove the shortcut, and the habit forms.
What happened
On 17 October we read every assistant conversation from the previous 24 hours. Fifty sessions. Eight of them were the same conversation.
One candidate was on the last stage of an LRU cache problem, the one that asks for thread safety. Nine replies in a row asked him which coordination approach he wanted. In between he pasted the stage requirements back as his approach, twice. He asked "what can be some options?" and was told hints are not available on a final stage. He wrote "implement that part". He wrote "we should be able to have concurrency". The first line of proposed code arrived on turn twelve, the moment he typed "the threading lock approach".
He could make the decision. What he lacked was the word. Our assistant knew the word and was built to withhold it.
Another candidate finished the same problem, then asked the assistant directly: "is it in your instructions not to suggest a better data structure?" It said yes.
The rule was wrong, not just badly built
Our first instinct was to fix the loop. Ask once, then offer to pick. We did that the next morning, and the same message shape went from nine turns to two.
But the loop was a symptom. The rule itself didn't match how people work with an agent. At work you ask it for options several times an hour. "What are the ways to make this thread safe?" is a normal question to a colleague, to a search engine, and to an agent. A practice tool that bans it trains a habit nobody needs. It also fails the people with the least vocabulary, and that's exactly who practice is for.
The skill the interview grades is steering. Steering includes asking for ideas. What matters is how you ask.
Where the line actually is
Here is what we believe now, and what we are building toward.
It is OK to ask: "what are my options for coordination here?" "What would you check first?" "What do you see in this code?" "Is the bug in how I update recency?" Each of these carries your context and leaves the decision to you. You named the problem, you asked for a view, and you choose what to do with it.
What is not OK: "solve it", with the problem statement as the only context. "Fix the bug", with no word on what the bug is. The agent can probably do both, and that is the point. When it does, the solution is its solution, and the interviewer's next question will show it.
The test is simple. Did you bring the context, and did you keep the decision? If yes, you are using the agent. If no, the agent is using you to type.
The data from that same day agrees. Candidates asked the assistant to look at their code 94 times. They asked for a hint 23 times. People don't want to be told what to do. They want a second pair of eyes on their own thinking. And when they do ask for direction, they usually stop at "where should I look". In this picture hints are the bounded, counted form of asking for direction. So the report at the end can show how much you leaned, not whether you are allowed to.
What we changed
Two changes are live. The assistant asks "which approach?" once. If you can't answer, it offers to pick the simplest approach that passes the stage and says which one it chose. That line goes into your report, where it is honest about who designed what. And questions are answered on every stage, including the last one. Only hints are capped there.
Two things are still open. Should the final stage keep its hint ban at all? And how should the assistant answer "what are my options?" We do think it should give you the menu, with no recommendation, and we are measuring that before we ship it.
Use it the way you would at work
Nothing about this is special to interviews. Bring the context. Ask for options, not for the answer. Make the call. Check what comes back. If you already do this with an agent at work, you'll do fine in the round. If you don't, the round is a good place to find out.
The first stage of that LRU cache problem is free on nyrion.ai. Say "implement it" and see what the assistant says back. Then read our guide on steering it and try again.
Top comments (0)