Many of my interview candidates do poorly as they don’t understand how I evaluate them. While other interviewers’ criteria vary, there are common themes. Knowing the basics helps you judge yourself and improve your technique.
At interviewing.io, I evaluate people on three categories. I think this fairly represents how various companies will rate you. Even if not explicitly considered, they’ll play a role in your overall evaluation.
- problem solving
- technical skills
- communication
I’m asked to give a rating to each one, but they are intimately linked together. A candidate who lacks one ability will also appear to lack the others. Let’s take a look at each point here, trying to isolate its specifics.
Problem Solving
Problem solving is the part abstract from writing code. I want to understand your process, and not just the result. This requires that you know what the problem is, and what I’ll accept as a valid solution. Problem solving is a lot about defining constraints and finding ambiguities, or uncertainties, and resolving them.
The challenge varies depending on the question. My card game question requires you to write a small game simulation. I want to see how you approach this problem. I want you to ask questions. What exactly is a “card”, and how will you represent it? What does “deal” concretely mean?
Problem solving is about taking high-level requirements and translating them into concrete steps. If you’re working on a project, this would be the phase where you write user stories and user journeys. You don’t have to be this formal in an interview, but I want to see that you’ve understood the requirements. Tell me what you’re thinking and write the key points.
Show the inputs and outputs to the problem you’re solving. State whether you’re transforming data, or implementing a process. Describe how to break down the large problem into smaller ones. For an algorithm, mention related algorithms, and tell me how you’d adapt them to this new situation.
Identify what parts you recognize and how it’s analogous to something you’ve done before. If there’s a whiteboard, sketch out flow, data sets, and anything else in your head. Especially if you’re stuck, I want to know how you’re solving the problem. Never fall into a blank silent stare. I’m there to help you, but I can’t do that if I can’t follow your process.
And definitely don’t treat problem solving as a phase. I don’t expect you to come up with the perfect solution at the start of an interview. I want to see you take an idea and run with it. It’ll likely have problems and require corrections. I’ll even change the requirements on you, or criticize your approach. I want you to accept change and adapt.
💭 Your ability to solve problems depends on your technical skills. While problem solving is not a tangible skill, your designs are based on the real-world limitations of computers. It’s not possible to completely separate problem solving and technical ability.
Technical Skills
There are two major aspects of technical skills that I consider. The first is your experience in coding idioms and wisdom. Do you know what is possible and how to go about coding it? The second is specific language ability. Once you’ve said what you want to do, how well can you implement that in the language you’re using.
The first point is about your general coding knowledge. Are you aware of data structures, program flow, and all the things programming languages are capable of doing? This knowledge influences your problem solving ability, as this is your toolbox. Whether your design will work depends on whether the tools are available.
I evaluate the candidate here on how much they hesitate, or how quickly they work. I’m not judging the speed itself, but as an indicator of one’s proficiency with the tools. I’m also listening to how you speak, whether your voice carries confidence or is it an inquisitive tone seeking affirmation.
The second aspect, knowing the syntax and semantics of a particular language, is vitally linked to the first. How you express yourself in code is my primary window into understanding your general coding knowledge. If you’re struggling to write a loop, I have to decide whether it’s because you don’t know the language, or you’re uncertain of how a loop applies to the problem.
I’m looking at how smoothly you translate your ideas into code. For example, from my card game question, there is a phase of dealing the cards. You must split the deck into two stacks, one for each player. Whether you’ve chosen a loop, or a high-level splitting function, you should be able to write the code. Writing the wrong code, forgetting how a loop works, using the wrong splitting syntax, all reflect negatively on the candidate.
I don’t punish people for making mistakes. They happen. However, a pattern of similar mistakes, or outright repetition of previous errors, shows a lack of knowledge. If I spot a mistake I may ignore it, or I may ask you about the code. Being able to recognize mistakes reflects positively on your abilities.
Languages have a lot of features and I’m looking for how well you use them. Are you using enumerations and constants? Can you create a structure to encapsulate values? Are you correctly passing variables by value or reference? Do you use the standard error mechanisms?
I don’t expect you to know everything, but I expect you to know all the basics. For individuals claiming to be experienced in the language I’ll be looking for common idioms. For example, in Python I look at the use of list comprehensions and dictionaries. In C++ I’ll look for smart pointers and possibly lambdas. If the code is absent all the common language idioms, then it looks like the candidate is inexperienced in that language.
💭 When I conduct general interviews I don’t put a strong emphasis on language knowledge. Your code is one way that helps me evaluate you. However, if you’re interviewing for a specific position, that requires specific language knowledge, your language skills will be weighted more heavily.
Communication
Voicing your thoughts correctly is the best way to ensure I properly evaluate you. Candidates that cannot do this perform worse than others. Not only does speaking well improve my evaluation of your communication skills, it also ensures the interview goes smoother and you finish the code on time.
Talking is your buffer to mistakes in your code. If you just write code, then my only view on your abilities is through that code. If you tell me why you are writing the code, then I can understand your approach even if the code turns out to be wrong.
Which also means it’s pointless to read the code aloud. I can see the code and read it myself. Your words should express thoughts that aren’t explicit in the code. Many candidates recognize the need to talk, but I feel are unaware of what they should talk about. Thus they end up reading the code. Sharing your thoughts is something you can learn, but it takes practice.
Pay attention to programming vocabulary. You should know how to talk about software in industry terms. This reveals a lot about your experience and your attention to detail. I want to hear about concrete things, like instance variables, constant values, and calling functions with arguments. I want to hear you talk about popping items off a stack and enumerating the elements of a map. I don’t want to hear about “doing something with this thing here”.
I also don’t want to hear buzzwords. Don’t use terms you don’t know and don’t claim to know words that you don’t. Dishonesty is a major red flag in an interview. Don’t think you can get away with it. I’m good at spotting bullshitters, as are most people working in HR and many programmers who have done a lot of hiring.
Don’t worry about not knowing everything. I tailor my expectations based on your level of experience. Junior programmers have not been exposed to many concepts. Be open, honest, and inquisitive. However, I expect a lot from somebody with ten or more years of experience.
Interviews are interactive and communication ensures we’re both on the same track. A failure to communicate generally results in a misrepresentation of your problem solving abilities, and general programming knowledge. And the more I know what you are thinking the more I can help you towards solving the problem.
Conclusion
Generally it’s a lack of communication skills or problem solving that sinks a candidate. I can’t imagine a company that wants to hire somebody with either skill lacking. Okay, sure there are some cases where specific, and uncommon, technical knowledge is required. But even then, you’ll be competing with other candidates.
There are also individuals who can write basic code, but not apply their knowledge to new situations. They appear to have learned one particular approach without understanding any of the concepts behind it. I can’t always say how correct my impression is, as I go on what I see in the interview. This makes it imperative to show all these abilities.
You may be wondering how to demonstrate these skills. Obviously, to improve any one of them requires practice and study. But to show them all in an interview requires balance. Don’t keep your head stuck in the code. Be vocal about what your are thinking, this is the key way to reveal your knowledge and show how you’re solving problems. Don’t be afraid to show limitations in your knowledge, instead be open and show how you’re working around it. Talk to the interviewer. Sketch things on the whiteboard.
Be aware you are being evaluated, and on three primary things: problem solving, technical skills, and communication.
Top comments (30)
So you prefer more chatting people, people who find it easy to talk out of their heads even is not finished?
What about people (mostly introverts) who have the ideas in their heads but find it hard just to chat about them, those maybe they would be more comfortable just to write it down in silence.
I got reasonable marks in school, not because I was better than others, but because I was more chatty. Teachers preferred chatty people that "participate" in class. Today I feel that is unfair towards more quite people.
I prefer people that can voice their thoughts. This is an essential part of being a programmer, it's not limited to an interview situation. I need to know that you can communicate with the rest of the team. I need to know that when you're having problems, you'll be able to voice them and get the assistance you need.
This is definitely a hurdle for some individuals, but getting over it will make one much more successful in interviews, and on the job. Don't worry that it's "chatting" though. I don't want smalltalk, I want your thoughts, however terse they may be. Trust yourself, and understand issues are normal. There's no reason to be afraid of voicing what's in your head.
This is absolutely not an essential part of being a programmer. There's an enormous difference between explaining something you've had time to think about on your own with your co-workers and talking out loud while you're trying to solve a difficult problem with someone staring at you and judging you.
This fails to assess every single thing you're supposedly assessing. It's not a good indicator of problem solving ability, technical skills, or communication. It's a good indicator of whether or not that person doesn't have performance anxiety issues and/or has seen that same problem before and already given it thought.
I can not wait until more companies realize how ridiculous and biased this form of interviewing is.
I totally agree with you. What do you think would be a good approach to an interview, if you had the opportunity to lay it out?
Usually when we are interviewing, I try to keep the candidate as relaxed as possible: voicing that explicitly, at the maximum two people at a given interview (with some exceptions). Doing that is easy enough, but how to actually make the candidate comfortable?
One good way (IMO) is to give enough time for problem solving through a tool like Hacker Rank or similar, and then after you have that out of the way, a friendly conversation about systems design/cultural fit/etc in person.
Coding with a stranger over your shoulder who is judging you is a lot different and more intimidating than talking to a team member in the same foxhole as you. I'm watching one of these interviews, and I already feel stressed and my neurons are misfiring trying to come up with a solution. Whereas in a team environment I feel quite comfortable communicating and designing solutions in a group.
I won't disagree, there's added stress in the interview. But the purpose of an interview is to assess your skills. What other ways would I have of judging the person's communication abilities?
I have different goals on interviews. My situation biases me -- we tend to hire devs with no experience. Coding interviews are not very useful in that case. And we need to help develop their skills anyway. I believe every person has a unique combination of inherent strengths and weaknesses and that most skills are learnable to a passable level. Including technical and soft skills.
Instead of looking for skill expertise, I look for a person who is teachable (implying intellect as well as outlook). If I was hiring for an experienced dev position, I would probably look for a person who is still teachable. This kind of person tends to be a supportive team member. The people I don't want to work with (regardless of experience level or the skills they possess) are the ones who already have everything figured out. They tend to look pretty good on traditional interviews because their confidence level makes them convincing.
As far as coding tests, I have given small take-home coding challenges, but these did not play a major factor in hiring decisions. As in, we hired devs who could not yet complete them. They were more to answer the question "how close to zero are they starting?"
Even with a more experienced position an individual coding interview seems like it would have limited value for us. It could be I want to assess their problem solving ability. But we tend to solve the hard problems as a group. The stress and arbitrary-ness of individual code interviews can play a large factor in creating poor on-the-fly solutions. (Code interviews often have you solve contrived puzzles. But my instinct in a real work environment is to view contrived problems with suspicion and to try to find the Why rather than rattling off solutions.) So such an interview doesn't really indicate how they will do in position we have for them. (Another factor: it turns out our tech stack is pretty amenable to being maintained, so it takes the pressure off of making mistakes and learning from them while still delivering value.) Also, we can't really use the code interview to test for tech stack knowledge, because probably very few people use our stack. We would have to assume even experienced devs would need a couple of weeks to spin up.
For the rest of it -- verifying resume claims and evaluating various criteria -- it can be done through conversation. And it still boils down to a judgement call.
Fair disclosure of known biases: Our software department is small, so sample size is limited. I'm also saying the above as the person that chose and has been iterating on our tech.
I hope you are not saying "voicing thoughts" is the same as "communicating thoughts". The former is just blurting everything you are thinking. This can be hard for folks who can think very quickly but cannot express them quickly with words, e.g., non-native English speakers, visual thinkers (remember they have to explain the pics they draw). The latter is more controlled as it is about distilling and arranging a thought in a way that another person can understand.
IMO a better approach would be to let the candidates to solve the problem while applying their technical skills as they would do naturally. If the candidate solves the problem, then ask away about the solution and how he/she arrived at the solution. [Or do we think asking well-framed questions is hard? ;)] This approach provides a more accurate picture of how the candidates operates.
That said, if you will consider the candidate's operational style in the hiring decision, then you should be up front about it. Even so, if you are looking for candidates with specific operational style, I'd say you are narrowing your picks and losing out on diversity.
That is roughly what we are doing at JankariTech.
The candidate has to work on a programming exercise (btw. we don't care about syntax, the IDE will help you to place the brackets at the right place, nor do we expect the candidate to have a complete solution at the end of the time)
After the programming the candidate comes to the interviewer and explains what she/he had done, what other ideas she had and what would have been the next steps.
We hope that gives a good chance to people who can talk easy, but also to those who first have to think in isolation.
I personally develop ideas while talking, so I find it very easy to 'discuss' with my self, but I know a lot if people who are different. I feel a lot of Western cultures give an unfair advantage to rather chatty people.
Nice practice.
I am fine with folks developing ideas while talking but I doubt what one talks about while thinking would make sense or seem coherent to many others. Trying to make sure that the talking is sensible+coherent takes away the focus from the task of thinking.
30 years in industry and I've gotten rejected because "I didn't pass the programmer test"... what a joke! The question they wanted me to code up was never once necessary in 30 years of programming. duh... And they put me into an unfamiliar environment; IDE and System, with three guys watching (no collaboration) to do it.
Besides, the way I code today is to first go to StackOverflow whenever I have any questions and start from there. To create something from scratch is 1980's technology.
The best thing to come out of that was not to be invited to work with them. I think "show me your code chops" interviews are ridiculous.
I wholly agree with you. I have 30+ years of software development experience. Throughout I would classify the best and highly skilled developers I've met to be non-chatty. Brilliant people! Some prefer to locked themselves in a darkened room with their headsets on with a soothing, bubbling fish tank in the background. They will fail this soft of interview. Do a Google search for the article "Developers rise against whiteboard interviews" and what David Heinemeier Hansson (DHH), the creator of Ruby on Rails said.
I failed an interview a month ago. I learned a tremendously important lesson though: interviewers tend to ask meaningful questions. There were 4 interviewers asking me Angularistic questions. They all had papers in front of them. I had a mind block. I know my stuff. The "how" wasn't satisfactory to them. They insisted on "so..what you call it? What do you call tat term? or What do you call that life-cycle?" Who cares!!!x10 A switch went off in my mind: "All interview I do from now on I AM the interviewer. I'm the one who knows what I can do. I'm evaluating you. Just because you're seeking to hire me doesn't mean you're the best interviewer. You have a requirement and I am here to fulfill it."
I'd like to see interviewers put their stupid papers and notes away and talk. Interviewees can't bring in notes or search Google/SO. When I started programming in the 80s we never had these linting tools and intellisense to guide us. In 1990 when I was a freelance developer I once spent 3 days hunting down a compilation error in a desktop application. How could your problem-solving interview-technique tackle that one?
All this talk about how you interview is simple Applied opinion. It's not even a golden rule because there are lots of holes in all interview processes. It's not even a science. I've literally seen, before my VERY eyes, interviewers who laugh at candidates after interviewing them and how they fail miserably. They would laugh about the content of some resumes: "What? This is not even a skill, etc". None could deter them to treat them as themselves. We all want a better world. The questions are: what does that world look like? How are we going to get there? What are you willing to give up, ergo, your way (opinion) to achieve it?
There are plenty of bad interviews and bad interviewers in this industry. I strive not to follow bad practices.
So what's the solution? Are you going to brush up on today's tech interview questions etc...?
The solution is to not interview with those requiring coding tests. Why? They are saying we don't believe your resume.
There's a website where you can watch other people's code interviews?! Oh! My! God! Guess I need no Netflix anymore for the rest of the week. :-)
Sounds like a bad episode of Black Mirror 😬
Lol
Under typical work conditions, in a decent team work environment, there is the planning phase during which one should be able express why and how to solve something in a particular way. This is needed so your peers can review and poke holes at it. This can be done via email, Slack, IRC or in a conf. room. Once the planning phase is done, of course nobody sane wants endless chatter about it. This is the implementation phase, ideally carried out without distractions until done or something big is discovered which requires another round of planning.
Interviews, unlike ordinary day in a coder's life, are more stressful because they combine the two phases into a short time-frame: design (coming up with a solution) and implementation. The interviewer thus has a critical role to ease the pain of the candidate and make her/him feel welcomed, not rushed. Nevertheless the candidate must be able to explain the design/solution because it's part of the phase-1 objective, i.e. to avoid coding mistakes before any code is even written. Establishing if a candidate can describe the design or solution is reasonable expectation and edA-qa's article nailed it.
A person can be bad in communication sometimes. I have experienced this in my early career. But that does not always mean that he is not hire able or bad at programming. I think we must find better approach for people with lack of communication skills.
Are there any mockup, or recorded, or just role-playing interviews we can watch of this concept?
As a freelancer, I've done almost no interviews, so I actually have very little idea what "level" I'm at, or how I would fair in such an interview. So I'm curious to see a mock or real or roleplay interview so I can judge my own self how I might do in one.
Yes! Exactly what I was looking for, I got rejected in an interview recently I wonder how many of the things mentioned here, I did not do.
TBH only the third tells me anything useful about a dev in the "heat" of the moment of an interview (very heavy emphasis on the quotes, I'll get to that in a moment). I do not do live code or whiteboarding exercises, nor terminology quizzing, period.
Why? Because here's what I've learned from them: That devs fresh out of CS degree programs and boot camps - and job hoppers - tend to do much better than those who actually solve real world problems every day.
Perhaps even worse is what I'm conveying to the candidate about myself and the company, no matter how unintentionally or inaccurately, by putting them in such situations: That they are expected to work under undue, on-the-spot pressure, caused by me or someone else above them in the company, and that they're "free" to solve the problem in their own way... But we're watching at all times.
That's bunk, and it stifles creative ingenuity from the very start.
So what works?
Why does this work for me, and why do I think companies should adopt it themselves and finally get rid of the live code interview? I already covered why pressure doesn't work (no matter how many times you say "no pressure"). The key to this is letting them relax. Not only do they feel more comfortable to create, to take chances, and more importantly to reason about solutions in their own way... It's also a lot easier to detect BS when a person is relaxed, while even the best candidates can crumble under pressure when they know it's their only shot.
If you'd have to suggest just one book to someone reading your lines, doubts himself and thinks "Would I do good with this interviewer? There's so much more I have to learn....", what would it be?
Can I suggest my book ...or my course :)
The book isn't targeting interviews, but I think it carries an important message for your concern. Programming is a vast field, and you should always be telling yourself "There's so much more I have to learn..." Until you have 15-20 years of experience behind you, there's no way you should feel as though you've seen everything.
Even then, you shouldn't assume you know everything. I walk into interviews with the same concerns as everybody else: how do I look to this interviewer. This doubt is normal, but you need to believe in yourself.
Remember, you're goal is to give an honest reflection of who you are, not to sell some artificial version of yourself. If you don't fit into the role, or the interviewer reads you wrong, oh well. On to the next interview.
Some comments may only be visible to logged-in visitors. Sign in to view all comments.