One of the many major things the tech industry is bad at is job interviews.
No other profession has so little correlation between candidates who are good at interviews and good at the job they are interviewing for. Most tech interviews focus on testing skills that have nothing at all to do with the kind of work the candidate would be doing day-to-day.
Good tech interviews are equally accessible to self-taught, bootcamp, or college graduates, and don't prefer developers who have enough free time to dedicate to drilling code trivia.
Good tech interviews favour candidates who are:
- Comfortable solving problems autonomously
- Able to recognize gaps in their knowledge
- Learn things as they need to know them
- Able to maintain their code as requirements change
Instead of white-boarding, code challenges, or testing knowledge of algorithms, I prefer to test candidates by giving them a small take-home TODO list application, written any way they want, followed by a live-coding segment where they add three small features.
Writing code and then adding features to that code models exactly what developers are expected to do on the job. Developers who have the time to practice LeetCode questions aren't better at writing TODO applications than developers who don't. Stanford has a class teaching how to pass the google exam, but Stanford students don't get an advantage here.
TODO applications are the most common tutorial, so almost every developer has interacted with one at some point. They don't require complex algorithms which are only taught in universities, and there are so many unique ways to build them depending on the developer's preferences, skills, and choices.
If you're hiring Frontend engineers, Backend engineers, DevOps engineers, or QA engineers, a TODO application can be written to stress the important skills for each role.
This is the spec I've used for dozens of interviews, made generic enough to allow the candidate shape the application to best demonstrate their abilities.
The Take Home Project
Imagine the business has asked you to build a simple todo list application for the web.
Requirements
- The user should be able to view a list of tasks
- Each task should contain a title
- The user should be able to create a task
- The user should be able to edit a task
- The user should be able to mark a task completed
Try to spend no more than 3 hours of work on this assignment. If some areas lack polish, it will be reviewed with the understanding that there was a narrow time constraint.
Use whatever tools, frameworks, and languages that you are most comfortable with. There are no extra points for using the same tech stack that we use. The more confident you are with the tools you've chosen, the more impressive your application will be. If you excel in a particular area, choose an architecture that allows you to demonstrate that.
- The application can be server side, client side, or both
- Data can be persisted to a database, to local storage, or not at all
- Tests can be end-to-end, integration, unit, or none at all
Create a repository on your preferred source control host (GitHub, Bitbucket, GitLab, etc) and commit your code to it, along with a README.md file that explains how to install and run the app. These instructions could be as simple as "clone the repo, run npm install, run npm start" but some projects take longer to set up.
When you're finished, email the interviewer with a link (and invite if the repository is private).
The Follow-Up Interview
Most candidates are nervous in interviews regardless of their confidence level on the job. Try to understand that getting or not getting the job will change the course of their career in one direction or the other.
Start the interview with a few minutes of small-talk to establish a bit of rapport and get the candidate more comfortable speaking on camera.
If you haven't already confirmed whether the candidate is legally able to get the job, do that now. Even if the requirements were listed explicitly in the job posting, don't assume it's been read or understood. In general, hiring contractors, even from out-of-country, is far easier than hiring employees and you'll have fewer questions to ask.
For employees, you should determine if they're legally allowed to work in your country. For example, whether or not they're a citizen or a permanent resident or if they're on a work visa. If they're on a work visa, see how long it's eligible for and when it's going to expire. If you're looking to hire someone who will last a year or more at your company but their visa is due to expire in six months, that's something you want to find out sooner rather than later.
Ask where they're currently living. This may not be the city they have listed on their resumé or online profiles or any other documentation you've seen so far. It's important to make sure that the hours they intend to work are compatible with the rest of the team. Even if you're fully asynchronous, it's good to have an idea of when they can be expected to be online.
I'd also recommend you ask what timeframe they would be looking to start once hired. Some candidates will be able to start right away and others might need weeks to transition from their current job, especially if they're relocating.
The Live-Coding Segment
Being able to see how people maintain code they've written themselves is a great signal for a developer who knows what they're doing
To maintain code is to modify it to handle a changing set of requirements.
Have the candidate share their screen as you move into the live-coding portion of the interview. It's a good idea to remind them to turn on Do not Disturb mode so that no notifications appear while they're sharing. If they have a large screen, encourage them to zoom in or increase font size so their code is legible.
Be prepared to walk them through allowing screen sharing permissions if they haven't already allowed those.
I start by having them walk through the code, pointing out anything notable in their implementation
They may have chosen to use a simple non-scalable architecture because that's all that the requirements of this demo project demanded. They also could have chosen to over-engineer the code as a demonstration of how they would handle a more complex project. Either decision is rational
Be careful about prodding these questions on your own so you don't hint at preferring one decision over the other. If the candidate feels like they've already made a mistake, they're less likely to perform as confidently in the interview as they would on the job.
Time-boxing this portion of the interview is a good idea. 45 minutes for the whole segment gives 15 for each task, and you can warn them if they're spending too long on any particular one.
Exercise 1: Permanently delete all completed tasks
Add a button that deletes all the tasks that have been marked as complete.
The usual solution here is to replace the list of tasks with a new array that contains only the incomplete tasks.
An easy way to make the new array is using Javascript's native array filter
const incompleteTasks = tasks.filter((task) => !task.completed)
The candidate might prefer stepping through the list of tasks in a loop to build the new array manually.
const incompleteTasks = []
for (const task of tasks) {
if (!task.completed) {
incompleteTasks.push(task)
}
}
Another solution would be to remove the completed tasks directly from the list without making a new array. This can be tricky because they're stepping through the list one by one but also removing tasks from it, so it's easy to accidentally skip an item. If the candidate presses the button with two tasks in a row marked complete, and it fails to delete the second one, this is usually the reason why.
for (let i = 0; i < tasks.length; i++) {
if (task.completed) {
tasks.splice(i, 1) // Remove task number i
i-- // If we deleted task 4, task 5 will slide up into its spot, so we need to check task 4 again next
}
}
Exercise 2: Sort tasks in descending order
Sort tasks in descending order, so that new items are added to the top of the list instead of the bottom.
If the candidate is not currently storing dates on each task, that's the first step, but it's up to them to determine that. They'll have to add dates to any new tasks they're adding plus any they might have stored to show up by default (if any).
There are a few ways to cheat here that should be discouraged. At the moment, every new task appears at the bottom of the list. That makes it look like it's already sorted in ascending order. The candidate might be tempted to render tasks.reverse() or to add new tasks to the beginning of the array instead of the end.
This only works by coincidence, and as soon as it's possible to add tasks with past or future dates, this fake sorting will break.
The usual solution is using javascript's native sort method. After giving this question to dozens of candidates I've concluded that no one remembers how this method works. To me, this question is an exercise on whether the candidate is able to look up documentation to patch their knowledge on anything they're missing, which is an incredibly valuable skill to screen for.
Sort works by comparing two tasks (A and B) in the list and returning -1, 1, or 0, depending on whether task A should be sorted before, after, or equally with B.
tasks.sort((a, b) => {
if (a.dateCreated < b.dateCreated) return -1
if (a.dateCreated > b.dateCreated) return 1
return 0
})
Using ternary is common here. It's not a big deal if they don't handle the 0 case for identical dates.
tasks.sort((a, b) => {
return a.dateCreated < b.dateCreated ? -1 : 1
})
If the dates are stored as a number (for example, a timestamp rather than a date), they might just subtract them. I'm less fond of this but it's incredibly common.
tasks.sort((a, b) => {
return a.dateCreated - b.dateCreated
})
When candidates implement the sort method incorrectly, common mistakes are to compare a - b directly, instead of a.dateCreated - b.dateCreated, or to return true or false instead of 1 or -1. Nudge them toward the documentation if they're making these sorts of mistakes. Sometimes candidates try too hard not to look anything up during the interview even if they would be quick to do so on the job, so extra encouragement can help.
When candidates implement the sort method correctly, the most common mistake here is to accidentally sort the wrong way first. If their sort doesn't appear to work the first time, it might be sorting into ascending order (which looks like nothing has changed). Most candidates will test swapping the order on their own, but feel free to suggest that if they seem confused.
The second most common mistake is forgetting that the sort method mutates the original array. If they've built all their code from scratch, this probably won't be an issue, but frameworks like React and Vue will throw errors if they mutate state variables. There are a few ways to clone the list of tasks before running sort, including Array().concat(tasks).sort, tasks.slice().sort, [...tasks].sort, or by chaining sort after a map or filter operation. If they're having trouble with this one, explain the problem, but give them time to find their own solution.
Exercise 3: Split the tasks into two lists
Split the tasks into two lists, with incomplete tasks on top, completed tasks on the bottom, such that marking a task as complete moves it from one list to the other.
It's up to you as the interviewer whether you require the sorting to still be in effect for this exercise. It's simpler if you don't, but optional.
The ideal implementation is also the simplest: keep one main array of tasks, and render two lists filtered to either complete or incomplete.
That might look something like this
const completeTasks = tasks.filter((task) => task.complete)
const incompleteTasks = tasks.filter((task) => !task.complete)
The more challenging implementation, which I've seen several candidates attempt but never complete within the time allowed, is to maintain two separate lists of tasks and move items from one to the other when marking as complete or incomplete. If they start to run out of time, I would suggest the simpler solution, but give them time to come to that conclusion on their own. The ability to realize when they're going down the wrong path and re-evaluate their chosen solution is a good skill to have, and this is a good place to watch for it.
Top comments (10)
Spot on - and if companies would adopt this approach then it would immediately make a ton of dev.to articles about "how to pass your technical interview by memorizing these 100 JS questions" obsolete ;)
(the funny thing is that this approach is probably EASIER than what most companies are currently doing - and it's going to yield better results to boot ...)
I think a big problem is that interviews are usually designed by professional interviewers or professional developers, and those are two distinct skills that need to work together to design an effective interview
It comes down to asking what behaviours you want to select for and to select against, and then testing your interview to make sure it actually works as intended
It's a lot of work to design an interview like that, but just to run it? You're absolutely right that it's easier. It basically brings you down to one question: is this candidate making reasonable decisions?
Yeah I suppose you're right, there's something to that.
I think the problem is also that companies aren't always asking themselves the right questions, or setting the right goals - what kind of a person am I looking for? I believe some of these interviews are even aiming at weeding out the bottom 97%, you know, that nonsense about the "top 3 percent developer", or finding the elusive rockstar or ninja or whatever.
In most cases companies don't need that, they need someone with a pragmatic and productive attitude, with an eye for quality, who gets stuff done and who can effectively communicate and be part of a team - I don't see how endless lists of questions about algorithms or the ability to code a quick-sort from scratch is achieving that, your approach would come a lot closer to that goal.
Just to add a viewpoint, I would reject this from the get go.
For any recuitment process, I feel non-paid 'take home' assignements are unfair and I would reject doing it if I estimate it will take more than 30m. There's too many juniors using too many hours of their time to get rejected.
I understand that this comes from a place of privilege, but I want companies to lose opportunities to that, so there's pressure to change.
If you want to test me at lenght, then you need to use the time too. Show that you care instead of just wasting people's time.
Want to see what I can do in 3 hours? Then you'll need someone with me, pairing for those 3 hours.
If you don't want to invest that time, then it means the process is not important for you and you shouldn't be wasting other people's time.
This is a position that I've developed after practical experience with the industry and the recruiting practices. This might very well be a golden standard, but I feel the push-back to bad practices is more important to the industry as a whole, and I feel one initial step for it is to make the time cost of any test the same for both parties.
I can definitely see that argument. Too many companies just throw out take-home projects as a way of filtering candidates, and then choose which ones they want to interview from the result pool.
I don't believe in giving such assignments until we're at the yes-if stage of the interview process. Yes, we'll hire you, if it turns out you can perform at the level you've been claiming to perform. If there's any question about their experience, education, or culture-fit, we should be screening for that before asking them to do any work.
Interviews can be quite a stressful experience, especially for anyone whose livelihood depends on it, and being able to demonstrate what you know outside of interviewer scrutiny (which reflects performance on the job much more realistically) can be a major asset to the otherwise nervous. Not everyone feels this way, as you don't, but I don't think there's ever going to be a one-size-fits-all solution.
And it's different again when hiring seniors, whose reputation often precedes them and doesn't need to be verified with technical challenges. I haven't had to hire any, so I can't speak to best practices there.
I'm also not a fan of unpaid exercises in general, but I've never been empowered by the folks who run the budget to pay for them. I think paying for 3 hours of effort creates strong incentives on both sides: to the candidate, who knows their work is valued, and to the business, who knows that these are not free and should only be given to qualified candidates.
This is fantastic. I've had similar tests from companies before and on a personal note, I do better. The company also gets to know early on, what myself and other candidates are capable of, plus how we think.
Unfortunately, with the traditional leet coding tests, they're okay I'd say for graduates, as they have the time to practice on the leet code websites. But for more intermediate and senior developers, who now probably have full time lives. This isn't so easy. In my opinion, with more intermediate to senior developers, I'd rather see what they are capable of on a daily basis. As well any any areas, I feel that they need to improve on. You can then compare that to the other candidates and see which one truly fits your team for now.
Plus you have so many former "fang" employees setting up tutorial courses on these leet code tests. Are you really seeing the candidates thoughts or the course creators work?
They're selecting for access to prep time and training materials for those leetcode interviews, not for ability to do the job
I think this approach is fine for junior level developers. Normally tasks envolve coding a certain feature that already have been design by someone more senior.
Mid level developers have to have a bigger set of competences. Is not only about the how, but also about the whys. For example, they need to be aware of design patterns and when one should be used. Implement some features in some time is a different skillset than thinking about a solution. In my opinion this is why 3h exercice plus a 1h live code is not enough.
Mid level developers should know why things work in certain ways.
Senior level developers is not much about the code ( after 5 or 10 years of programming of course they know how to program ). Is about how code should be, how it can be improved, how can it be scalable, there's so much more to it than code itself.
Here I think it come to play what does the company want:
Someone to implement features
Or someone to think about how a feature should be implemented
I think it is easier to teach someone how to do a todo app than teaching CS fundamentals.
Just my 2 cents here
Hiring seniors is definitely a different ballgame, but if I'm at the point of needing a senior I'm usually reaching out specifically to a person whom I want to do the job, and I already know their competency, so it's a bit different.
Of course, that doesn't apply to all hiring or all companies, but it's outside of my range of experience and I can't speak much on that.
It's interesting that this is basically exactly what my current employer does for interviews (though a different problem more derived from our domain instead of a generic todo) and has done since I was hired.
Judging by the quality of engineers hired over the almost 4 years I've been there I'd say it's a good strategy:
Does an excellent job of testing their coding abilities as it most closely matches actual workload without a lot of the false signals from trivia style coding problems (you studied this particular problem, you had someone else write your submission, etc.) It's the follow up that's important; it exposes why it's bad they wrote it all in one function, or didn't test it, or used regex for everything. Have a small enough follow up change so even the most junior can make progress and show their current skill level, but have enough follow ups that the most senior won't run out/can flex their thought process a bit.
Great article!