How to Create a Sensible Take-Home Coding Test

Tech assessment is an essential part of developer hiring. Take-home challenges seem a convenient option to assess tech skills because they are asynchronous and remove candidate's stress.

However, overlooking some fundamental aspects might lead to never completed tests, lots of time wasted on both sides, and bias when assessing candidates' results.

In this post, we'll cover the basic rules for ensuring that your take-home tests achieve their primary goal, i.e., identifying if someone has the necessary tech skills required for the job.

Problem

The main challenge with take-home tests is that people have to do them outside working hours. So you have to deal with the following constraints:

• Candidates are short on time. So they might not complete your take-home test.
• Take-home tests require a longer feedback loop. So given your candidates are in several interview processes, you might lose them to someone else.

Is There a Solution?

I have some bad news for you: unfortunately, there is no silver bullet to ensure that all your candidates will complete your take-home test and that you'll have enough time to select the best ones.

However, if you follow the ground rules below, it'll significantly increase your chances of getting the most out of your take-home coding tests.

Define a Concise Challenge Scope

First, define the main assessment objective and make your exercise revolve around it. The following questions should help you:

• What experience levels do we target?
• What tech skills do we want to assess?
• Do we only want to screen candidates or do a full tech assessment?
• What competencies can we assess automatically using auto-tests, and when would we need a manual code review?

Second, create a clear task description. Here's an example of the structure I stick to that has been working well so far:

• Introduction - provides a general description of the task.
• Getting started instructions - helps the candidate get started fast (e.g., describes the setup, how to run tests, describes external dependencies, etc.).
• Expectations - describes what the candidate should focus on.
• Way to get help - describes how to proceed in case the candidate gets stuck.
• Time estimate - helps the candidate plan their time before starting working on the task.

Here is an example of how it could look:

Third, provide auto-tests that the candidate can run locally to help them ensure they built what you wanted them to.

Once your candidate volumes have grown, you can also use the auto-test results to vet your candidates with no developer effort on your side.

Reduce Friction

Stick to the process developers use every day:

• Instead of sharing your take-home challenge as an email/Word doc/ zip file/, use a Git repository (ideally, use GitHub as most developers already have an account there).
• Instead of asking them to send you an email when they are ready, tell them to create a Pull Request.
• To avoid lost emails, ask them to create a ticket (e.g., GitHub issue) on their repository instead whenever they experience a problem.
• Pre-upload a project template (or templates if you expect submissions in multiple tech stacks) so that your candidates don't waste time setting everything up from scratch. Feel free to use this upload script that has support for lots of tech stacks.

Define a Standardized Code Review Scorecard

Because we're all humans, it's easy to slide into the bias-land when code reviewing someone's work.

To ensure no bias creeps in, create a code review scorecard that includes the following:

• The scoring rules. Here's one way to define it:

• Define scoring criteria per evaluated area. Here's an example of how it could look:

Embrace Continuous Improvement

Likely your take-home challenges will have some sharp edges in the beginning. What's worse is that not all your candidates will be vocal about it.

To ensure that your take-home challenges are up to standard, use this 2-step rule:

• Add the following to the challenge criteria: "Describe one thing you'd improve about this coding task".
• Once there is a good suggestion, implement it right away. Following this rule will save your tech interviewing team hours of future process revamping.

In Closing

Developer hiring is hard. But it doesn't have to be terrible. Following the take-home interview principles above will already put you well beyond most of the hiring processes out there.

Is there a good principle not described in this post? Don't hesitate to share it below in the comments.