loading...

Engineering whiteboard interviews: yay or nay?

lynnetye profile image Lynne Tye Updated on ・9 min read

Whiteboard interviews

Photo by Mark Rabe on Unsplash.

Software engineers hate whiteboard interviews. How do I know? We continually tell complete strangers just how much we hate them. But it's not that simple.

The problem is that we often conflate several issues: using random CS trivia/riddles to assess candidates, interviewers deliberately creating a stressful environment for interviewees, competent humans who happen to interview poorly, and the job search sucking in general.

"Whiteboarding" has become too large of an umbrella term, one that groups together everything that's wrong with the interview process. The truth is more complex, which is why I talked to hiring managers at several tech companies to get their opinions.

Are whiteboard interviews the best way to evaluate engineering candidates?

Jamie Karraker

CTO at Alto, previously engineering at Facebook (Parse).

I don't believe in the traditional whiteboarding questions on data structures and algorithms, at least not for our business. An interview should be designed to test as closely as possible for the skills an engineer would be using day-to-day in the job. For us that means being extremely productive, given we are an early stage startup and need to build a lot of product and make a large impact with very few engineers. But for Google they have to be able to solve very few, but very complex, technical problems, and productivity is less of an issue given their limitless resources. So for them the whiteboarding question might be the right fit, as it is an efficient and scalable way to test for performance on solving complex technical problems.

Eugenia Dellapenna

Engineering manager at Medium, previously engineering at Pivotal Labs.

I don't think they're the best way to evaluate candidates, but I also don't think they're as useless as some blog posts make them out to be. I actually think that they're the best way to ask certain kinds of interview questions, such as high level system design questions, but that they aren't the best way of asking coding questions since it's pretty different from how people are used to writing code on a day-to-day basis.

Nowadays, I tend to favor using collaborative coding environments, like CoderPad, for coding interviews. CoderPad is nice because it allows you to run the code, so you can quickly test the code with different inputs to see if all edge cases are handled. This also allows you as the interviewer to see how the candidate debugs real code and how well they're able to interpret things like compiler errors and runtime exceptions. It usually takes a bit longer to complete interview questions in this coding environment though (because of the time spent debugging) so you tend to not have as much time to build on the initial question or discuss further improvements, which is one caveat.

I still usually give candidates the option to write their solution on the whiteboard if they really want to, since some people specifically practice that when they're preparing for interviews, and I want people to feel as comfortable as possible. I think in general, it's your job as an interviewer to set candidates up for success so you can get as much signal during the interview as possible. Doing that well is more important than if you're asking the question on a laptop or a whiteboard.

Brian Wheeler

Senior Director of Engineering at Braze, previously engineering at Microsoft.

I'd say "best" is an overstatement, but I do think they can be a valuable component of an interview process. When done properly they can be a reasonably consistent evaluation of problem solving ability, communication skills, and preparedness. Some questions are not suited for whiteboard interviews, but for all others the more important factors are what you ask, how you ask it, and how you dialogue.

Are whiteboard interviews the best way to evaluate engineering candidates?

Dan Erickson

VP of Engineering at Eaze, previously CTO at Getable and engineer at Yammer.

No, not at all. Whiteboard interviews test one thing well: How well does a candidate code on a whiteboard. Engineers on my team never have to code on a whiteboard (whiteboards are really bad at running code), why would I make candidates do something that I don't ask the engineers already on my team to do?

Some argue that whiteboard interviews can help you understand a person's "soft" skills (a term I hate because it downplay's the importance and difficulty of emotional intelligence skills). In my experience, there are better ways to evaluate these skills, and often times directly asking the candidate to give examples of these skills is more effective.

Desmond Brand

Director of Engineering at Flexport, previously engineering at Khan Academy and Microsoft (Bing).

Whiteboard interviews might not be popular with the median candidate, but I think Kevin Lacker nails it in the example at the bottom of this blog post.

There's nothing special about the medium of a whiteboard itself, however. At Flexport, we give people the option to write code on their own laptop if they prefer, and some take us up on that. This works better for those who use a TDD-style process or who just like to insert text before other text :)

But how and where code is written is incidental. The important part of these interviews is communicating (using words & diagrams) about a problem and approaches to solving it, and then also communicating (using words & code) about teaching a computer to solve that problem too.

There are more drastic changes we've experimented with, e.g. a take-home project, but those tend to impose an asymmetric time investment upon the candidate, which some people greatly dislike. So in practice, projects can only be an option at best. However, having different styles of interview can be problematic too because it increases noise in evaluations of candidates, which then increases the risk that unconscious bias creeps into hiring decisions.

Sandy Jen

Co-founder of Honor, previously founded Meebo and sold it to Google in 2012.

From a functional perspective, it's not very useful... no one will be producing code on a whiteboard as their primary job. However, it is a good way to learn how a candidate communicates, expresses ideas, and takes feedback. The expectation is not to ace the whiteboard question, but to learn more about you as a potential team member.

David Chang

Engineering Manager at NerdWallet, previously engineering at aboutLife.

There’s no one best way to evaluate engineering candidates. All interview processes have their own strengths and weaknesses, and it’s important to understand how much (or how little) any given question can tell you about a candidate.

At NerdWallet, we usually conduct architecture and design questions on a whiteboard. It helps us to better understand how candidates structure and communicate complex technological solutions and the broad strokes of the design are more important than the specifics of any particular line of code. We also normally ask a couple laptop coding questions, for obvious reasons. Recently, we incorporated a pull request question, because working with other people’s code and giving feedback is a key part of being a productive engineer at NerdWallet and we were missing that from our previous interview process.

None of the questions are perfect, though. Some engineers feel uncomfortable designing completely new systems on demand, coding questions may just happen to hit on a candidate’s blind spot, and understanding a PR for code you’ve never seen before can be challenging. We’re big believers in feedback and data-driven decision making, so we’re always working to better understand what our interview process can tell us about a candidate and determine ways in which we can improve it.

Are whiteboard interviews the best way to evaluate engineering candidates?

Zeesha Currimbhoy

Director of Product Engineering at Branch, previously VP of the Augmented Intelligence Team at Evernote.

I have spent my entire professional management career debating what the best way to interview candidates is. In my ideal world, a candidate should never have to "prepare" for an interview and an interview is just as simulation of a day in the life at work. In this vein, I have played around with different formats, the take home challenge, the white boarding interview, the algorithms and data structures sections, a live coding session, a debugging session and the list goes on. I have a lot of opinions on this topic (much like any other manager). But to sum it up, whiteboard interviews, in my opinion is a great "tool" to use while evaluating candidates.

Thats right, in my opinion a white boarding session should be thought of as "a tool" to enable you to problem solve with a candidate. What’s great about a white boarding session is that it is a blank piece of canvas, where the interviewee can effectively think out loud, collaborate, draw diagrams and also brainstorm with the interviewer. It doesn’t have friction between your thoughts and being able to communicate them. At the same time, it shouldn’t be the only way to interview candidates and is definitely not a one-size-fits-all. An effective interview is an assessment across several different dimensions with different ways of evaluation. Whether we like it or not, a whiteboard represents a large part of technical communication not just in an interview but in day to day operations as an engineer.

Larry Salibra

Engineering Partner at Blockstack, previously founded Pay4Bugs.

I don't think whiteboard interviews are particularly useful in that they don't reflect the reality of performing the job. Open-ended coding challenges that involve using our software to build something give us much more information about the skill level of a candidate and his or her interest in the project.

As an open source project, we're lucky in that candidates really interested in our project can prove themselves by sending pull requests or building apps on our platform - in some cases this means we're comfortable making an offer without asking the person to do an additional coding challenge.

Derrek Harrison

Engineering Manager at Samsara, previously the Director of Engineering at Kinetic Growth.

Great engineers have many tools to choose from when solving a problem — great companies also have many tools at their disposal when searching for and evaluating talented team members. A whiteboard interview is an excellent tool when the candidate needs a large canvas to diagram out a problem or design a system. This is particularly true when asking a system design/architecture question or asking the candidate to describe the interaction between multiple systems.

However, whiteboard interviews are not a fitting format when testing a candidate’s ability to write code. If you take away the ability to debug and test during an interview, you are left with a scenario that is not an accurate representation of day-to-day engineering. A collaborative coding environment is a better choice here — it gives the interviewer the ability to clearly define a problem in writing and gives the candidate the ability to write, debug, and test in a familiar setting.

We strive for an interview format that allows a candidate to shine and gives us the best possible signal on how the candidate would perform as a member of the team. This means that sometimes, but not always, a whiteboard interview is the right format.

Adam McKerlie

Director of Engineering at G Adventures, previously an Engineering Manager and Developer at G Adventures.

After years of iterating on our interview process, testing out different ways to evaluate a person's skill and personality, I've found that whiteboards add very little to the overall interview process. They can be useful at understanding how someone works, but more often than not, I've found them to weed out really good candidates who haven't had the opportunity to do whiteboarding interviews before. I much prefer to talk with a candidate over a problem and have a conversation with them, similar to how we'd discuss a problem in real life when scoping out a project. I find people are generally more at ease with this approach and don't get tied up on syntax, their messy writing, or talking aloud while "coding."

Looking for a new job?

Everyone quoted in this article is a hiring manager at a company on Key Values. You can learn more about their engineering culture by reading their team's profile, and if you're still interested, I encourage you to apply or even reach out to them directly.

Posted on by:

lynnetye profile

Lynne Tye

@lynnetye

Lynne is the creator of Key Values, a website that helps engineers find teams that share their values. She lives in San Francisco, is an Iron(wo)man, and loves meeting new peeps!

Discussion

pic
Editor guide
 

Some of these answers point to "new alternative live-coding environments" which solve a bit of the problem, but this still seems like a less-than-ideal way to evaluate someone's actual coding experience. It also optimizes for what they happen to know this exact moment.

I really feel like if you want them to come work with you for longer than a month or two, the gaps in their knowledge will close pretty quickly as long as they have a good approach and perhaps some relevant experience. I can't think of a lot of scenarios where I want to have my opinion be formed based on an abstract live-coding scenario.

 

I think part of the problem is that we are "testing" or "challenging" someone. This implies a level of distrust and hostility which only serves to heighten stress levels in what is already a very stressful situation.

You're 100% on-point. Regardless of medium (whiteboard, live-coding in a shared editor, timed coding challenges e.g., Hackerrank, etc.), these are still less-than-ideal ways of evaluating someone. Enough that there's a cottage industry of tech interview prep companies out there which I feel are exacerbating the problem.

 

I genuinely think coding challenges give no indication of actual programming skill. Programming is like 90% problem solving so I think intelligence, experience, and creativity are much more important. You can test for 2 of those!

 

Stefan, you really hit the motherload! I agree with you 100%.

But the problem is, how can you test intelligence or experience or creativity? Sure, there are some ways, but sadly not everyone is qualified to test such things. Developers surely cannot test them. Most of HR specialists are not psychologists/sociologists.

I love these talks, and I am gonna remember your answer. I really want us all work together, use our collective brainpower to solve the interview dilemma once and for all, so both interviewers and interviewees are OK with the terms of interviews.

I think we should look at how business consultant interviews are done. Some of their interview questions might sound stupid (how many ping-pong balls fit inside of an airplane?) but they do test problem-solving skills and general intelligence. What is a developer if not a problem solver, using a programming language as his tools?

 

Technical tests of all kinds are great for people just starting out in the industry but once you have a CV with 4 or 5 years with references you have already proved yourself. When I'm looking for a role now, if they ask me to do a technical test, unless it's a company I really like, I politely declined and explain my reasoning. 9 times out of 10 this is acceptable to the employers and they see my point.

 

once you have a CV with 4 or 5 years with references you have already proved yourself

Not all jobs are created equal though. Someone who spent 5 years as part of a big team working in an enterprise environment might have very different strengths and weaknesses from someone who only worked in early stage startups.

 

You'd be surprised. A lot of the time the real talent is actually snapped up by startups as they are the ones driving innovation. You'll find that those working in an enterprise environment typically tend to be behind the curve and stagnate.

Exactly my experience, I only worked in startups or agencies myself.

I think you misunderstood my comment: What I tried to say was that after 4-5 years you have not automatically proved yourself for a new position. If those 5 years were in an enterprise environment, you might not be the best fit for a startup since you may be used to a slower pace, different stack and definitely a different working culture. The other way around no matter how much you learned in startups, you may not be able to deal with a more rigorous process, bigger teams etc. Hence "different strengths and weaknesses".

 

Great collection of answers!, this goes directly to my bookmarks.

I'm a software engineer and I do not hate whiteboard interviews.

When I say "whiteboard" I mean any env that lets you draw and type freely, from a physical whiteboard to draw.io.

Like any tool, the whiteboard is the best fit for specific questions (about the candidate. how they think mostly), great way to express yourself, and it does not exclude the other types of interviews, I see them as complementary.

I think the statement "my devs do not code on whiteboard" is just wrong, if you know your language you should be able to code on toilet paper, the environment does not matter. And noone expects that the whiteboard code should be syntax error free, is about the main idea/algorithm.

I love whiteboards, we have them at work on every wall (kind of, glass walls), I have one at home, and it helps me solve any kind of problem, from data structures to life decisions. But depending on the company and role it may not be the best solution for the interview.

 

Not really. A lot of the roles I've come into, I've never done the particular language or technology that I'll be working with before. In this sense, whiteboard tests really do not apply.

 

«if you know your language you should be able to code on toilet paper» this one goes straight to the list of my favourite quotes 👍

 

I think that they are a good first step, but I don't think that they should be the final result. The are a good way to test if a person is programmer, but asking someone a difficult problem on a whiteboard is not a good way to evaluate them as a programmer.

I lean more and more to the idea that a portfolio is the best way to evaluate someone. You can see the way that they work. But more importantly you can see the quality of the code that they produce. That being said, portfolios have their own issues. You could be a phenomenal programmer, but if you don't code outside of work, you can't really show that. Also, it is a huge time commitment, and some people don't want to do that. Lastly you can just plagiarize code.

There is no perfect interview method, but I don't think that whiteboarding is that best method for final evaluation, but it may be good enough for initial evaluation.

 

A portfolio is often the rarest thing you will find with full time developers who are working on key industry projects. Often they don't have time to code outside of their job or the last thing they want to do after coding at work all day is do the same when they get home in order to make a portfolio. If I see a developer with a portfolio who is not a contractor I start to wonder; do they have an unhealthy obsession with development and no social life or are they so devoid of work in their role that they have the time to do a portfolio on the side.

 

I think that these are all good, valid points. :)

I just think that it is a good way to judge the development skills of junior devs. When I got my first job, it was definitely being able to talk about the side project that I had created and the processes that I used that landed me the job, not my ability to solve fizz buzz and do SQL queries.

Once you are a mid-level or senior dev, I think that people are able to evaluate your skills in a different ways.

 

I'm curious to hear how these hiring managers are dealing with the other problems in this list: "The problem is that we often conflate several issues: using random CS trivia/riddles to assess candidates, interviewers deliberately creating a stressful environment for interviewees, competent humans who happen to interview poorly, and the job search sucking in general."

 

Nay.

Is that the right answer?

 

🤔 I can't tell if you're kidding, but...

the point is that it's not binary, even though some people treat it that way. If you read what the hiring managers quoted in the article say about whiteboarding, you'll see that it's really nuanced.

 

I'm mostly kidding, but if I had to make a gun-to-the-head choice, it's be no. It's a reply because the question is binary.

Whiteboard interviews obviously is a term that has grown to not just include actual whiteboards and there's room in the world for a lot of different techniques - you can learn a lot from asking people how to solve something in a practical sense :)

 
 

Nay in most cases.
"competent humans who happen to interview poorly"
Exactly. In truth, whiteboard interviews select for interview skill, not for competency, or even for day-to-day soft skills. In an actual work scenario, you won't have a senior dev breathing down your back and asking you a battery of questions about what you're doing. They'll check out your PR and accept it or make comments as needed.

 

I love whiteboards for discussion within teams, but they seem too much of a crutch for use during interviews. Ask trivia question about an algorithm or something, make code monkey write it on the board, move them to the next step if they can do it.

If it's used in the way whiteboarding is used in real life, as a discussion, that's better for the interviewer and the interviewee.

"How would you approach this system?" They draw it out a bit. "What's that part over there do?" They explain, keep drawing, say they don't know how to proceed. Interviewer asks a relevant question to see what their thought process is. Interviewee answers and they talk it out.

It's hard to interview properly, but that's also how you can find the best fit on both sides.

 

Whiteboard interviews look at a small sliver of skills you'll need for a software engineering job namely problem solving and mathematical thinking. Other equally important skills are communication with colleagues and non-technical team members, design & modeling, familiarity with domain knowledge (especially in niche companies) & perseverance (a good developer is aware of the inherent frustrations of dealing with multiple layers of abstraction). A better way to test candidates would be to ask them to add a small feature/resolve an issue on your existing code base and pair them with an experienced developer (with due compensation for time). But this is not a scalable process as it calls for equal investment from the companies' side as well.

 

This reminds me of the old joke, how you shut up an engineer? Take away the whiteboard.

It's one of those "it depends" kind of tools. What do you want to use it for? An in depth, code me a function now kind of thing? No, I find its not the best tool. Honestly, these days, I barely even see them around any more. In our open office we use them more as movable walls than really to draw and communicate on, except for those times in Sprint planning we need to talk things out. For interviews that's the only good use I get out of them, as a way to make the candidate speak to something.

As a communication and lead tool on a question or problem we face all the time in my environment, I draw something start asking questions and use it more to lead into how the person thinks and problem solves. There are better and more efficient tools to use if you really need to have someone program.

 

If a whiteboard is used as support for discussion, I'm fine with it, from something like "here is a piece of code, what's wrong with it?" to "that's our infrastructure, how would you improve on it?".

However, coding "live" on a whiteboard, paper, PC or else stresses me a lot.

I heavily rely on code completion (and copy/paste) and added to that, I have a really bad memory for names, be it for function, methods, methodology etc ...
So when it is time to write, I spend more time trying to remember the proper name of a function than to think about the logic of the code itself.

Pseudo-code or flowcharts would be fine as long as we don't focus on the minute details of the drawing.

I'd rather be given an assignment to complete before the interview for them to check I can create a solution based on the requirements.

 

The question is what problem does whiteboard coding solve for whom?

I understand the need for some companies to establish some kind of gateway mechanism in order to reduce the amount of hiring false candidates. And live coding is one way to reduce the number of false positives after hiring.

I never did live coding so far. And to be honest, I think I would perhaps fail epically. So chances are low, that such companies would hire me.

It says nothing about me as a developer - besides, that I am perhaps bad at live coding. But that as I said isn't the purpose of it.

 

At our specific agency, 90% of new hires are graduates fresh out of university, so we look for candidates to prove that they can program (at all).

We give a pen-and-paper programming test that candidates complete unsupervised, testing fundamental programming skills (no special data structures or compsci) as well as a database design and diagramming task.

This is very successful at weeding out the bottom 50% of candidates, who, despite just finishing their Computing degrees, cannot write or describe a working computer program. But we can conduct the test without openly embarrassing the candidate.

 

Personally, I think having a portfolio of code you have worked on on github is the way to go. Use that and explain what the problem was, how you approached solving it and justifying why you took that approach. That's what you'll be expected to do in any job anyway, so it makes sense in an interview as well.

 

My opinion, white boarding is a skillset. Like the ability to speak to a crowd. Some can do it, some cant. I would always argue for white boarding to bring clarity to an issue.

It usually flushes out a problem between differing opinions. But for an interview, just throw glass on the floor and ask me to dance. I need to know you better before I can play charades with any team.
-rhodey

 

Here is a list of companies that do not use whiteboard interviews:

github.com/poteto/hiring-without-w...