Why engineers won't do your coding test

The idea behind a coding test is very simple: to filter out candidates who do not have the technical chops for the role, early on in the process before the hiring manager and candidate both waste their time with an in-person interview.

But most engineers today frown at the idea of completing a coding test, and over 50% straight out refuse to do status quo assessments (based on our research with 100+ companies in SEA).

Here are 3 of the most common reasons why engineers hate status quo coding tests:

1. They test for algorithmic skill rather than the ability to write code.

Companies need the scores from assessments to be significant, and the easiest way to do this to use trick questions on the assessments. To do well on these tests, candidates need to spend weeks practicing writing code for a list of trick questions. Only a fraction of developers can do well on these tests.

As an interviewer, it is very easy to forget how stressful the interview setting is for the interviewee. Having to write executable code for a very niche algorithm you studied at school (that too only if you were a CS major), and never really used in your time as an engineer in the real world, with the timer ticking, can be very very intimidating!

While it's great if someone's good at algorithms (even though this skill can be improved with practice), this is not a strong indicator of how good of an engineer someone is/ how good they're going to be in the role. Only a small fraction of tech roles require strong algorithmic ability. Also, this way of measuring developer skills' has an inherent bias against more experienced developers.

As you can imagine no great developer is excited about the prospect of giving a test in the first place. Add it to the fact that questions are irrelevant and 50% of candidates straight out refuse to take these tests.

2. They're too big an ask

Asking the candidate to spend more than 60 minutes on a coding test before you've spent any time is unfair.

When you use a 3-hour coding test it defeats the purpose of automation, because while the hiring manager has nothing to lose, the candidate now needs to spend more time on it than they would have for a video/ in-person interview.

The longer your assessment, the lower the test-taking rate will be.

3. It's harder to code in an unfamiliar environment

Most developers have a preference of IDE (integrated development environment) that they've customized in a way that helps them write code seamlessly. A test environment is unfamiliar, and it's harder for a software engineer to function optimally. This is especially true when the test requires the use of not just a programming language in a simple code editor but instead is testing for front-end/ backend code framework capabilities.
Developers often challenge the validity of coding tests/ assessments because of these and other reasons and understandably so.

So, should we skip coding tests altogether?

That is not an option. Anyone who has been involved in tech hiring knows that there are enough developers in the world who aren't qualified enough for the role, making it necessary to have some kind of a litmus test that candidates must pass before being invited to an interview.

Can't we use a resume screen instead?

Software engineers tend to not be good at selling themselves, and great candidates often massively undersell themselves on paper. At best, a resume screen helps you eliminate some candidates who are very clearly not qualified for the role and sort resumes by priority. Beyond that, using a resume filter has an inherent bias towards candidates with good credentials (education and work history). Good programmers can come from anywhere, and using keyword matching means you're probably missing out on a lot of great candidates. But if companies start interviewing everyone who applies, it would take up all of the engineering team's time just to interview candidates.

Now that we've established the need for an assessment solution, how do we evaluate if the assessment solution we're using for our hiring is a good one?

Here is a list of top things you want to check for. You're in good hands if:

1. Your test-taking rate > 70%.
2. The average time to complete the assessment is between 45-75 minutes.
3. When you ask candidates for their feedback during in-person interviews, they have good things to say about their experience giving the assessment.
4. Hiring managers are happy with the quality of candidates that are being forwarded to in-person rounds.

If your current solution does not satisfy these criteria, you might be missing out on strong candidates for your team. As software engineers and hiring managers, my co-founder and I have previously used a majority of the status quo solutions and found the results unsatisfactory. So this is what we've been working towards for the past couple years on, and have seen early success in.

At Adaface, we're building a way for companies to automate the first round of tech interview with a conversational AI, Ada.

So, is this just a coding test but in chatbot format?

No. Here's what we're doing differently from the status quo:

• Shorter assessments (45-60 mins) to make sure engineers can do it ASAP, they are investing as less time as possible, while still enough to showcase their expertise.
• Custom assessments tailored to the requirements of the role (NO trick questions).
• Questions at the simpler end of the spectrum (it is a screening interview) with a generous time allowance (3x what it takes our team to write code for it).
• Extremely granular scoring that eliminates false positives and false negatives.
• Friendly candidate experience (hints for each question, friendly messaging and chatbot; average candidate NPS is 4.4/5). ☺️

How does Adaface fare on our criteria for an assessment solution?

1. We have an average test-taking rate of 86%, as compared to the industry standard of 50%.
2. The average time to complete the assessment is 62 minutes.
3. We're most proud of the feedback candidates share after the assessment. The average rating is 4.4/ 5.
4. We focus on testing for on-the-job skills for each role. Ada can screen candidates for 700+ skills like Java, JavaScript, SQL, PHP, React and other popular skills in Software Engineering, Data science, DevOps, Analytics, Aptitude, etc. This helps our customers find the most qualified candidates. Several of our customers have moved from using status quo solutions to Adaface, are able to save upwards of 75% time in the screening process.

You might also want to check out our post on the top 52 pre-employment assessment tools for 2020.

Here's a sample review from one of the candidates who completed a first-round interview with one of our customers:

"I gave more then 15+ interview for this year, they some time get connect in phone call or in skype which seems quite time consuming as one need to wait for the call and then some time network issue or hardware issue, this interview seems very good as one can give it any time and is a virtually fit with new era of interview. Ada you are perfect..!!"

If you're currently hiring, I'd love for you to check out Adaface.

This article was originally posted here.

Comments

[Rob Kendal {{☕}}]

I love this! Something I feel very strongly about and it always seems to fall into two camps:

1. The tech test isn't meaty enough to really hold any merit of someone's ability,
2. or it's a large enough exercise, but asks so much of a candidate (in terms of free time, like spec-work) that it's unfair

Plus, when you apply it to other industries, it becomes really really weird. Like, you wouldn't assess the right plumber for your bathroom refurb by asking them to quickly hook up a sink for free would you?

[Deepti Chopra]

Glad you agree with the sentiment of the post, Rob. Cheers!

[Nedim F]

If I were to test people I would give them problem that they can solve in 30 minutes.
And let them do what ever they want with it.. use their own equipment and what ever else. But end program must pass all tests.