Why I Stopped Interviewing with Companies That Require a Coding Test

You may be familiar with a famous quote thats has been attributed to Albert Einstein:

"Everyone is a genius. But if you judge a fish by its ability to climb a tree, it will live its whole life believing that it is stupid."

Now there is some debate if Einstein actually said or wrote this BUT the wisdom, nonetheless, rings true:

If you judge everyone by the same metric, there is a chance that you may not get an accurate measure of success.

Sadly, during my 7+ years working as a professional developer in the tech industry, I have seen a metric used to measure developer and programmer success that in MOST cases does not match the actual skills needed to be successful.

And if you have at any point attempted to apply for a job or interview for a position, there is a STRONG possibility you have encountered this metric; Coding Tests.

Coding tests come in several different forms but at their core, their goal is to test your proficiency with computer science principles, algorithms, and general CS knowledge. Coding tests can be delivered in-person on a whiteboard, virtually in a coding sandbox environment or in live-virtual environment where an interviewer asks you to share your screen and watches you work.

Fundamentally, coding tests are non-problematic. They are just a tool to measure a developer or programmer's competency and can be administered to most any job applicant fairly consistently.

But as with any tool, when used inappropriately or without the proper understanding of its usefulness, they can do more harm than good.

So let me briefly summarize why I no longer interview with companies that require a coding test.

NOTE: I am specifically using the word "REQUIRE" as I do not have an issue with a company that uses coding tests as one of their many metrics to evaluate candidates. I am particularly talking about companies that evaluate all technical developer/programming candidates via a coding test and DO NOT have an alternative option for a coding test in place.

1. It Shows a Lack of Care or Concern

So I can understand in the grand scheme of things why coding tests are useful. They allow a hiring team to quickly screen a large amount of candidates and gives a fairly easy metric to judge candidates by. It it by no coincidence that most very large tech companies use them in their initial stage of the interviewing process as a means to weed out "low competency" candidates.

But it is this EXACT reason that coding tests being a requirement in an interview process shows a lack of care or concern. It shows that the goal in this interview process is to weed out candidates, regardless of their wholistic strengths.

It doesn't matter if the candidate is a self-taught programmer, a high-performing developer who has worked in the industry for decades, or a college student pursuing their first opportunity, coding tests only evaluate you on a single metric. And if you have spoken with many recruiters or HR professionals at some of the bigger companies, it acts as the ultimate gate keeper to opportunities.

Being a tech professional who has been interviewee, interviewer, and sometimes just observed technical interviews, I have found that coding tests can be used to invalidate a developers experience and skill in an instance.

I have watched as EXTREMELY competent coder crumble when they were given a coding question that they struggled to answer and that I later found out, the interviewer didn't know how to solve. The interviewer simply went online found common coding questions and presented them to candidates. Luckily, I was able to participate in the interview and "intervened". This particular story ended up having a happy ending but not every interview will have an "observer" associated to it.

I have personally been in interviews where I have done stellar on all technical questions related to the actual role (as explicitly stated by the interviewers in the process) but was no longer considered a viable candidate because I had a hiccup trying to solve an esoteric algorithm coding question that would NEVER be used on the job.

Personally, a coding test requirement in the interview process indicates to me that an employer and an hiring team aren't really in touch with what makes a quality developer and how to measure actual success. But are looking for any easy way to comb through candidates and minimize interactions with candidates until they are further down in the process.

2. Some Candidates are at a Massive Disadvantage

This point personally hits home for me, but there are circumstances that can make a coding test extremely unfair to candidates looking for an opportunity at a company.

Imagine we threw away everything I said in my first point. Imagine that all coding tests were administered by hiring professionals and teams that genuinely cared about their prospective candidates. Even if the hearts of every interviewer and person in the hiring process was absolutely pure, coding tests would still put an undue burden on otherwise competent and valuable candidates.

How so?

Well it comes down to the fact that some people are ALWAYS going to perform badly in a code "testing" setting. May it be someone with a learning disability, an individual with forms of anxiety that affect testing performance, or someone who's background makes the testing material extremely difficult. And know, this is not an exhaustive list but a representative list of limiting factors to coding test performance.

Sadly for me, I've ALWAYS struggled in high pressure testing environments (ask my college GPA about that 😅) and I've also struggled with coding tests because as a self-taught programmer, things like algorithms and CS concepts are something I didn't really encounter until later in life.

Because of those factors, it makes coding tests extremely difficult. Even in coding tests where I have done "okay", it still didn't feel good. And though I have always performed extremely well at any role I have ever had the opportunity to work in, a coding test metric would show me, in most cases, as absolutely incompetent.

There was one occasion where I was going through an interview process and performed will at almost every stage. At the very end of the process, the recruiter was giving me amazing feedback and was about to end the interviews for the day. (NOTE: This was an on-site multi-hour interview.) As I was heading out, I was asked if I'd be willing to do an "extra" interview with one of the "higher-ups" in the company and I was fine with that. What I didn't know, was that they were going to administer a "test" to me. Within the course of 15-minutes, I lost the job. My anxiety destroyed me. EVEN THOUGH, I was found to be competent and qualified by others in the process, my failure of this one "test" ended my chances.

That was a very personal experience, but I have a feeling that I may not be the only person who has been summarily disqualified from a position because of a poor performance on a test (some which don't even cover skills that would be used on the actual job).

3. It's Not Worth It!

I have a personal conviction when it comes to interviews, that goes like this:

"You already know what you know, No need to prepare!"

Once again this is a personal conviction (which I don't expect anyone else to have) so my personal mentality with interviews has always been to do no extra preparation other than maybe reviewing what I already know (and brushing my hair and looking presentable 😅). I want interviewers to have the most honest reflection of my skills. I want them to see me as I would be if I worked for them on a daily-basis.

I have on multiple occasions in the past been told I need to "study for this interview". I would then be sent a set of links or a PDF on how to prepare for this interview and necessary skills I need to learn to "pass" it.

When I was looking for my first job in the industry and I had almost no experience or knowledge that I could exemplify in a demonstrable way, I kind of got it. Companies need to know that I can learn and that I'm willing to take challenges, so fair-play. But interestingly enough, I got my first job in the industry without having to study or take an explicit test; They did test my knowledge but it was relative to the job role itself.

Now, as an experienced professional, I personally refuse to study for a job interview for skills that I will not use on the job. It's just not worth it.

With the busy-ness of life and the difficulty of finding time to study with a full-time job, in most cases, I have been asked to study concepts and materials that I almost never have and never will use in my day-to-day. There is a good chance that after I pass the test that I may NEVER use the knowledge I studied for in a meaningful way when I do land the job.

I have from time-to-time contemplated maybe studying for a coding test and then realized I'd rather spend my time learning skills that I will most likely use in the future. It many ways, I feel that asking to study for a job interview should not be the standard. You should be evaluating the candidate based on their current skill, not on a skill that they don't currently possess.

---

Now, at the end of the day, this is all my personal opinion and comes from my time in the industry but I'd love to hear other people's thoughts. I am not completely against "coding tests" per-say, but I do feel that they are overused and/or misused in the tech industry. At a minimum, I think there should always be an alternative to a coding test OR that it should NOT be used as a "gate-keeping" mechanism in the interview process. If anything, just like behavioral tests that are given during some job interview processes, it should be used as a frame of reference, not the deciding factor.

So do ya'll think coding tests are worth it? Do you disagree with my opinion/take on the matter? Did I get something wrong or make a point that doesn't make much sense?

Please let me know below in the comments. I'd love to hear your thoughts!

Thanks for reading, my friends,

Bradston Henry

---

And If you are interested in checking on what I believe to be better ways to interview, check out my blog on tech interview approaches:

Death of the Coding Test: Interview Methods that Better Evaluate Candidate Competency

Bradston Henry for Developers @ Asurion ・ Sep 21 ・ 16 min read

#career #programming #discuss #interview

===CREDITS===

• Cover Photo: Monstera from Pexels
• Education System Photo: Unknown
• Woman Shrugging Photo: Polina Zimmerman from Pexels
• Disappointed Woman Photo: Andrea Piacquadio from Pexels
• Man's Smartwatch Photo: cottonbro from Pexels

==== FOLLOW ME ON SOCIAL MEDIA ====

https://linktr.ee/bradstondev

Comments

[Luke Shiru]

Every time I see one of those interviews with a coding test like "sort an array of numbers in JavaScript without using sort":

Good post, thanks for sharing!

[lionel-rowe]

I don't think the meme really fits, because it implies answering academic "implement bubble sort" type questions is more difficult than solving real-world software engineering challenges. I don't think it's more or less difficult — it's just a different type of question altogether.

[Katherine Hambley]

When have you ever had to sort an array in JS without using sort on a real job? When have you ever had to implement bubble sort on a real job? They're just trying to trick you because they are jealous they are not wicked smart like you. :)

[lionel-rowe]

Yeah the most difficult thing about sorting arrays in JavaScript is remembering which of a - b vs b - a (or a.localeCompare(b) vs b.localeCompare(a)) means "ascending" and which means "descending" 😅

As for performance, the browser implementation is already optimal (versus anything you could implement in JS itself, at least — possibly you could squeeze out some extra perf with WASM) in almost all cases.

[Luke Shiru]

My way or remembering is using "z" instead of "b" ... want to sort from a to z? a - z .... the other way around? z - a.

[lionel-rowe]

Ooh that's actually a great tip! I never thought of re-imagining the hyphen as "to" instead of "minus"!

And it has the same ordering with localeCompare too: a.localeCompare(z) is ascending, and z.localeCompare(a) is descending.

[Peter Ellis]

Wow, that's genuinely amazing. And I mean it. Thank you, Luke.

[Luke Shiru]

I see the meme more like it says "completely different" than "less difficult", but memes are art, so you can interpret it differently 😅

[Paul Abrams]

Not even a different type of question. It's highly relevant as a test.

[Luke Shiru]

In a CS school, maybe. If you seriously consider a question like this relevant, I really hope you aren't part of an interview process, because if that's the case, then I'm sure you missed great talent, but I'm also sure that talent dodged a bullet as well 😅

[Josh Cheek]

I'm really confused, what kind of coding do you do? In every job I've had, I've needed to implement algorithms. Maybe not sorting algorithms, but sorting algorithms are much easier than many of the algorithms I've had to come up with. And sorting algorithms are widely studied, so rather than asking you to implement a complex esoteric algorithm, they can ask you to implement a well known algorithm that you're probably already familiar with. Sure it's not exactly equivalent to what you'll do on the job, but... IDK, if you can't implement sort, then what happens when you need to implement double entry bookkeeping?

[Luke Shiru]

My point is not that you don't have to do algorithms, my point is that when you have to, you will not do them by memory. 90% of the times it will be something you can Google or tools like Copilot will fill for you. Is way more important that you know when to use a "double entry bookkeeping" than how to. In an interview is way more valuable that given a problem you're able to say what logic would you use, talk about why and even if you had any problems in the past with it, that if you know by memory how to write it. If you're not able to figure out if a candidate is lying about their experience, why are you even interviewing? Not only that, we already talked about this in this comment section, instead of making the candidates code a contrived piece of code just for the interview, ideally you should talk about challenges unique for the position. How is it useful to do a code challenge about sorting when the problems you have daily in the job have nothing to do with sorting?

[Josh Cheek]

NVM, I saw your other comment with the specific sorting question in it, and I get it. It's a question about "are you familiar with or can you come up with this 1 weird trick" rather than "can you implement a sorting algorithm". Yeah, I wouldn't really expect anyone I work with to come up with that solution, and while it's theoretically slower, I just tested them both, and the builtin sort was over twice as fast, and way more obvious.

ruby -e '
   def time
     start = Time.now
     yield
     Time.now - start
   end

   def sort(a)
     counts = Array.new 100, 0
     a.each { |n| counts[n] += 1 }
     sorted = Array.new a.size
     i = -1
     counts.each_with_index do |count, index|
       count.times { sorted[i += 1] = index }
     end
     sorted
   end

   a = 1_000_000.times.map { rand 0...100 }
   puts correct: a.sort == sort(a),
        builtin: time { a.sort },
        custom:  time { sort a }
   '
{:correct=>true, :builtin=>0.039097, :custom=>0.087166}

[Lucian I. Last]

github.com/Financial-Times/polyfil... Look up the polyfill, import, done

[Luke Shiru]

There's a popular company I will not mention, that has the following "problem":

Given an array of 1 million random numbers between 0 and 99, return the sorted array without using native sort.

They expect you to create a new array of 100 elements (indexes from 0 to 99), and then go over the unsorted array counting the amount of appearances of every number and save it in the corresponding index of the newly created array, and finally loop over that array repeating every number the amount of times it appeared. Something like this:

const sort = array =>
    array
        .reduce(
            (counts, item) =>
                Object.assign(counts, { [item]: (counts[item] ?? 0) + 1 }),
            [],
        )
        .flatMap((count, index) => [...Array(count)].fill(index));

Or maybe like this if you prefer a less "functional" approach:

const sort = array => {
    const counts = [];

    for (const item of array) {
        counts[item] = (counts[item] ?? 0) + 1;
    }

    const output = [];

    for (let number = 0; number < counts.length; number++) {
        for (let count = 0; count < counts[number]; count++) {
            output.push(number);
        }
    }

    return output;
};

And supposedly that's faster .... IDK why would you ever have an array of 1.000.000 elements, and why would you ever want to sort that, but yeah ... that kind of shit is what you can expect from a really bad interview.

[Vincent A. Cicirello]

I'm not questioning your view on whether or not this is a good or bad interview coding problem. I'm just commenting on the "supposedly that's faster" part.... The worstcase runtime of the various counting sort algorithms is O(n), whereas the best you can do in worst case with a comparison sort algorithm, like mergesort, is O(n log n). When a counting sort algorithm is applicable to the data, it can be extremely faster than mergesort and the other comparison sorts. In most common cases though, either its not applicable, or the array size isn't long enough to overcome the constant factors, or for other practical purposes.

This is probably not a good interview coding question in general though. Actually it isn't really a coding question at all. It is an algorithms question masquerading as a coding question.

[Luke Shiru]

100% agree. My point is mainly that. When I heard that question from a close friend my immediate reaction was: When will I ever need that sorting logic? I mean it can be faster in some scenarios, but generally you sort stuff that is way more complex than just numbers. My problem with this algorithm is not so much about perf, but actually about usefulness.

[scottshipp]

And supposedly that's faster .... IDK why would you ever have an array of 1.000.000 elements, and why would you ever want to sort that, but yeah ...

In certain domains, this is the kind of thing that comes up all the time and for which you can actually cost (or save) your company a lot of money. The approach you're describing is actually a well known sort called Radix Sort. It has a far better performance, at O(n) compared to O(n log n), not "supposedly" faster. There's a well-known problem/story about Radix Sort in Jon Bentley's famous Programming Pearls.

I've heard a lot of programmers scoff at stuff like this but it is actually useful and practical knowledge.

No argument that it's not necessarily a good interview question. In most cases, that's a poor interview question because it is testing knowledge (do they know about radix sort?) rather than skill or problem-solving ability.

[Luke Shiru]

My "supposedly" was kinda sarcastic, mainly because in real world scenarios you don't sort number arrays (an array of numbers with no data related to those numbers), generally if you have a bunch of numbers, you have data related to those numbers (let's say for example that the numbers are prices, then there are products related to them). Pretty much in any real world scenario, using Array.prototype.sort will be faster.

There are lots of approaches to "problems" that are faster than the commonly used, but the thing is that some of those problems only exist for "demos", or while learning programming or whatnot, but you never see them in real software products.

Reinventing the wheel in an interview is far from useful for both the candidate and the interviewers.

[Blake Schwartz]

In my opinion these kinds of questions are really just meant to serve as a semi-contrived roadblock/sieve to help ensure that only “certain types of candidates” make it through to the final stages. Even though it is nigh inconceivable that you would regularly deal with sorting an array of one million, let alone one thousand, elements in a JavaScript context, unless you really like to re-invent the wheel and/or your boss doesn’t mind paying several hours/days of your time to fiddle around, you’re pretty much never going to come up with a more optimized approach than what the c++ programmers have already baked into V8 or whatever engine you’re using.

Asking a question that requires you to use built in methods creatively seems a lot more useful, but I see these kinds of “in a world where you’re using React, but somehow Array.sort doesn’t exist…” (I.e. frontend roles) questions in job interviews all the time.

Meme is spot on btw 🎯

[Palfalvi Laszlo Lorand]

Another point is that in these days we are kinda working under preassure and we should always choose the fastest tool to implement because what the client wants is to get to market asap. Sometimes you don't even have time for writing tests.

So yea I think it's good to know about different algorithms and how they work but most likely you will need to know when to use them and know the basics pretty well because that's what you will use most of the time.

[Paul Abrams]

If you can't sort numbers without sort, then you don't have the requisite skills.

[Luke Shiru]

Sure mate! Because knowing by memory something that the language does better than you, or that you can get by googling is extremely important to determine if someone is a good dev </sarcasm> ... Seriously, you're like the folks that got mad with Dan Abramov (core member of the React team) for not knowing how to center an element with CSS by memory. Is way more useful you know what tools to use and when to use them, that to know what the underlying algorithm is. If I ask you to sort an array in JavaScript and your first answer isn't using Array.prototype.sort, I would be worried.
There is nothing wrong with knowing how to sort numbers "manually", but you don't "need" to know how to do it in order to be a good JS dev.

[Thomas Cayne]

Yeah. I passed 2 interviewers but I couldn't pass the tech "leads". They wanted me to sort and array manually. So I said, "why sort it manually?" Can you shuffle it? he asked. At that point I fingered out that they were just going through the motion. It had nothing to do with the framework for which I was being interviewed.

[SwapnilBasher]

The company you currently work in , how do they interview candidates without coding interview ? Home assignment or questions from github project or something else ? (I'm undergrad , no idea about job field)

[Luke Shiru]

The first "technical interview" doesn't have any code on it, we just talk about your experience, things you liked and things you didn't liked about previous projects if you have professional experience, and if you don't have professional experience we just talk about what's interesting for you about WebDev, how do you keep up to date, and stuff like that. If we feel you're a good fit "culturally", just then we got to a more "coding step", and is a "home assignment" that's pretty similar to the actual position you'll get (real world problems), and that doesn't even has to be 100% completed, we only care about the decisions you took to solve the problems in the assignment, like why you took those decisions, and if you can think of any improvements, and your overall experience. If you instead prefer to do a "pairing session" instead of the "home assignment" that's ok as well. We want to avoid making you waste your time, so is up to you as a candidate to define how to show your technical skills.

The main things we care about is how good teammate you'll be, not so much about your technical expertise (this doesn't mean we don't care at all, it just means we care less about it). We had to say no to a few candidates a few times that were great technically, but they also shown signs of not being great teammates.

This happens in a lot of descent companies, so if you get into an interview where they ask stuff like: "Implement your own Array.prototype.map" or some useless junk like that, take it as a red flag that the company might not care about you or your team as human beings.

[Moisés Álvaro Blanco]

Couldn't agree more!

[zakwillis]

Great meme.

[Techislife]

I liked this comment, but I really don't understand the meme. 😊🤪

[Luke Shiru]

Memexplained: Sometimes the interview makes it look like a high budget and high quality position, and then the actual job doesn't have anything to do with the interview. It portrays correctly how lots of interviews suck 😅

[Brad]

I find it fascinating that people build coding tests to test what people know, to see if they are fit for a given job.

Then candidates study those tests so they can get the job.

Then people analyze their performance to see if they are fit for a given job and then continue the process with more tests.

The only issue is if the coding test is nothing like the actual job, everyone else is spending time and effort dealing with what is ultimately unrelated to the actual job.

If the actual job is very coding-test-like, then sure that's an excellent representation. If its not, isn't it a huge waste of time for everyone?

[Bradston Henry]

I feel exactly the same way. It's 100% appropriate if the job itself has a lot of tasks that might mimic the problems you have to tackle in a coding test, but thats seems more like the exception than the rule.

And it would be quite a tragedy if you performed extremely well on a coding test, started a job and realized that you didn't have any of the skills needed to do the job.

To think that a coding test could be a waste of time for anyone involved. Not good.

[Marcus Corrêa]

I agree with you, and I've been there.
Three years ago I applied for a company that required a JS test with tests in Jest. The test was basically to input any number and output this number's name written (102 -> one hundred and two). I nailed it, reviewed the tests and the code 20 times, refactored and so. The interviewer loved it!
I got the job and then I realized: they were expecting a much higher level dev which should do JS AND React AND Ruby/Rails (I was aware of the stack needed before interviewing, but the test level was much easier than what I was expected to deliver in the actual job).

[Bradston Henry]

That's tough! So sorry to hear that happened!

How did you handle it when you started working the actual job?

Had to be super discouraging to run into that!

[Marcus Corrêa]

Thank you for your kind words :)
In fact it was tough... I could handle well with the React stuff, but on the backend side I was still a junior dev. As the time passed I could feel that I was not attending their expectations, and one year later they said that they needed someone with more advanced skills in the team. However they were super kind and let me stay in the company until I find a new job, which took two months. Not everything was bad at all :D

[Bradston Henry]

Wow! That's incredible!

Even though I'd never wish that on anyone, I'm actually amazed by their empathy and consideration in your situation.

Really happy things worked out for you! That's an amazing story!

[Ski]

I personally think really a lot of abilities that help you perform well in job also help perform well in coding tests. But it of course also depends how coding interview is done from interviewer side.

I see coding interview more like a pair programming session. While problem is synthetic the methods used to problem solve and the skills that are exposed during this session are all relevant to day to day job.

I had chance to run a number of code interviews over last couple months, and most common problems were:

• Poor communication - inability to express ideas in words, for example I have seen candidates who would explain their ideas literally by citing a pseudo code - this in turn leads to very inefficient communication and candidates lose a lot of time
• No communication at all - candidate just writes code without explaining and after running out of time code just doesn't work. If candidate would had communicated his idea I would had helped and rejected it and saved a lot of time. It's faster to solve problem on conceptual whilst brainstorming and testing number of different ideas with interviewer before delving into coding.
• Not listening to feedback, for example I would see a bug, I would ask candidate to explain how this line works, they would explain one thing, I'd say I'm not convinced it works this way, but they would stuck their head in sand and would insist on their own belief and come back to that line later after wasting 10-20 minutes debugging
• Not being able to simplify code, for example candidate has idea how to solve it, but writes code that's too complicated and runs out of time debugging it
• Being lazy and skipping on decent naming and keeping code simple and readable: sometimes candidates make subtle bugs in conditional statements while putting numeric constants straight in code or by writing conditions that are just too long. These types of bugs are easy to see with good naming but very hard to notice when naming is poor or when naming is not there and numeric values are put right into long if statements. Before coding interview begins I always remind to keep code clean, use good naming, as this would help the candidate and also will help me spot bugs - if I see easy to spot bug - I always help - unfortunately I'm not a robot and can't spot bugs in a messy code.

I think neither of this is in any way some esoteric situation. It's just daily work stuff. On top of that if problem is really more on esoteric side and has some things that candidate needs a "click" in brain to happen, I try to do my best to help guide them so that this click would happen.

[Brad]

It's just daily work stuff.

When I think of daily work stuff I think of doing a feature and PR review.

A common alternative I hear to a whiteboard interview, or a coding challenge is a take home project.

A take home project can not only test more relevant skills, it also provides a more real-world experience for everyone. With PR-like feedback the experience would also provide a similar setup to a work environment, and allow you to uncover all those problems you described above.

This is all without all the stress you'd get from needing to code/solve a problem immediately while someone else watches.

I'm not saying all companies should do this, or that all companies could do this, but it does seem more reasonable and relevant than dropping someone into some "test environment" and pointing out the similarities, rather than just putting them into a real world "work environment" and having a guardrails setup (fake inputs, fake data, relevant but fake project).

[Ski]

With take home tasks problem is that some candidates would spend 1 hr others might spend whole day.

[Ingo Steinke]

exactly the same will happen when they are part of your team, no matter if they work from home or sit in an office. Doesn't matter much when they deliver good quality.
Getting paid by time spent is not much better than getting paid by lines of code, but it seems to be the most common measurement in our business somehow.

[Amantel]

I think it's the other way around. Company is much more intetested in someone who can do a task in an hour, than in one that can do same stuff in 8 hours.

[Jeehoo Ahn]

The last bullet point:
have you run across companies that seem to not care about or even think programmers should be able to read and comprehend and manage bad naming and obscure code such as literal numeric constants in conditions?

It sometimes feels like I get the short end of the stick either way:

• if i try to push for better naming, both when it's already written and preventively as a matter of following best practices, I'll get resistance, but it ends up taking me longer to do a task and this will occur whenever i try to read various coworkers' obscure code
• it also feels like companies and managers want me to be able to deal with such code anyway, or i won't get the job / project contract

[Davide de Paolis]

besides the fact that NO, SORRY, NOT EVERYBODY IS A GENIUS, (maybe different, maybe special but not genius) I always loved the quote:

"if you judge a fish by its ability to climb a tree, it will live its whole life believing that it is stupid."

and found arrogant and patronizing when people "judge" a person in all their complexity based on just few parameters.

But I love it only when applied in general or just to the education system.

A kid could be good at Math, and suck at art.
A kid could be good at Drawing and suck at Science.
Both are very intelligent and special kids, teachers should remember them, respect their different abilities and natures. Try to foster where they thrive and support them were they lack.

But.. If we are talking about a specific thing, a specific skill that we are assessing, this quote is meaningless.

If I visit a Music school, and I suck at anything musical, I cannot just say:

Ehi you are being mean, I can't play the piano, but I am special, I am a genius, I can run 100m in 10seconds, and... and.. I was very good at Math too!"

If the fish applied to a Tree Climbing competition, should we really take into account his ability to swim? Or should we, with good reason, consider him stupid for even applying?

I am sorry, but here we are talking about hiring someone to become a Software Engineer, so the ability of coding must be assessed.

We can discuss how, in which ways, with which tools, but a "coding assessment" is necessary.
It might be stressful, it might be timeconsuming, it might possibly weed out valid candidates, but it is necessary.

as everything in Software Engineering, and basically in life, IT DEPENDS, mainly

• on the position you are applying for
• on the type of coding test

If you apply for a Solutions Architect position, you can't just say: well... I am not good a drawing diagrams and not so good at naming System Design principles..."
If you apply for Frontend Engineer with React experience, you must be able to quickly jot down a component with a couple of hooks.
If you are a Javascript Engineer I expect you to know about Currying and Partial application.

Of course there are loads of stupid companies and interviewers that fail because the apply interviewing patterns that make no sense for their company and daily job. But it is not a flaw of the coding test.

If the job deals with algorithms, performance and really complex low level stuff, you must have a CS background. (be it self taught or official). If the job envolves solving problems with code ( and namely, building some components, and connecting apis and using services already built for the job - be it a library, or a Cloud provider service) way less.

I also had interviews where i was asked to reverse a string or sorting an array without using any native methods.. which is something that makes no freaking sense in a real world scenario. I failed them miserably, but I left without any sense of defeat - either I would not be a fit for their daily tasks, or the company mindset would not be a fit for me.

The point is, hiring is hard ( and being hired too).

• No coding test? you might be hiring someone that is just bragging or is good at selling themselves.
• CS / Algo-Datastructure Test? you might be hiring someone that is very skilled but has no idea how an avarage day at any gig works ( or simply, they are very good at solving quizzes)
• Takehome real life coding test? you might discriminate people that have not so much time outside their working hours).
• Just friendly chat about coding and technical questions? people gets stressed during interviews, might be shy, or have speaking difficulties, but still very good at programming.

There is no right way to do that. So in the end, it is really up to your gut feeling, and the company that you applied for.

[Bradston Henry]

I really like your thoughts on this!

I think it's interesting because the issue isn't the Coding Tests in their purest form, it may be an implementation problem.

At the end of the day, Recruiters, HR professional, and development teams NEED some way to judge if a candidate is competent. And sadly, just taking a person at their word is probably NOT the way to go 😅

I think you statement really rings true:

There is no right way to do that. So in the end, it is really up to your gut
feeling, and the company that you applied for.

I think it's INCREDIBLY important to trust yourself and know what is best for you and evaluating your skills. For me, it's not coding test but for others it may be.

[Meat Boy]

Exactly my thoughts. No matter what companies will do, always will be someone complaining about the recruitment process. Currently is most visible as the project-based vs test-based check of experience.

[Federico Navarrete]

This is for me, one of the best answers I have read. Personally, I agree, you cannot be a solution architect and don't know how to create diagrams. I am a "huge" fan of the C4 model.

Furthermore, I agree with the point that if you are applying for jobs working with metal or I don't know nuclear plants, I agree that you must have certain skills. However, the main issue I have experienced in Europe is that I used to have too theoretical code interviews for things you would never use. When I was searching for a job, I never applied for optimizing OSs or building Machine Learning Models positions, and I still had questions of topics I hadn't used in 10 years since I had graduated from CS.

In my first interviews I failed horribly, and I was forced to study things that I don't use, and I already forget again. I still have a crazy sticky note in my phone if I ever need it again with all potential questions that I got that I didn't remember in detail.

I can just add that I'm not against code interviews, in the only company I worked in Poland that was a large corporation, we had technical tests, but they were about a small business case that was related to your future job. We had some CRUDs and other stuff, and you could choose the .NET tech you wanted. After your solution was done in 1h more or less, you will have questions and based on that we proceed. I consider these tests fairer and with more sense.

We avoided extremely theoretical tests where people had no idea about .NET, LINQ, T-SQL, etc. but they knew all sorting algorithms ever, and we focused on the people that our business needs. Sadly, I had seen more comparably sized companies doing the same Quick Sort or related algorithms for unrelated positions that when the times come and you start your job, they tell you things like this one: "You must migrate an entire DB instance on your own for X date." This happened to me, and I succeeded, but not everyone is so lucky.

[Ali Adnan]

agreed 100% with you . best comment and answer

[Alex Lohr]

We usually do coding tests not as a test of knowledge and skill, but more as a test of approach and methods. Applicants may ask as many questions as they want and a working solution at the end of the test is not a requirement. We'll rather have someone to ask the right questions than someone who thinks they know it all.

[Karel De Smet]

Totally agree with this one. When the interview is scheduled, it should be made clear that a coding test will be part of the interview, so that the interviewee isn't caught off-guard. Then, during the interview it should be made clear that the interviewee can take as much time as required and that it is advisable to ask questions which will help clarify the problem at hand. If these conditions are met, I don't see any issue in taking/conducting a coding test.

[Alex Lohr]

I once advised my manager against taking a senior developer who had aced the coding test with ease, but was also the most opinionated person I ever had the misfortune to meet and the quintessential antithesis to a team player. I really wouldn't have wished him on any of our teams – and the colleague who was conducting the interview with me immediately agreed.

On the other hand, an interviewee with a speech impediment who was obviously very nervous and barely managed to finish the coding test, but had asked the right questions, I could wholeheartedly recommend; he is now an estimated team member and was promoted to senior developer last year.

[trailrun80]

Oh, the gatekeepers, where would we be without them.

[Alex Lohr]

When hiring for jobs that have more applicants than openings, you always get gatekeepers as a necessity.

If you're lucky, they will also look for things like team dynamics and methodology. If not, I'd hope you'll find another job soon.

[Graham Crocker]

My argument is that if you're choosing people to run in a team for a marathon, its only fair to ask them to walk or jog on the spot first, rather than saying 'Oh he's a nice guy/gal' go through the whole process to onboard them and later find out on practice runs through office gossip that they are actually lame and they'll need a guy in the team to carry them across the finish line. Its only unfair if your tests are, dance the salsa or juggle footballs, as while it shows proficiency in foot use, but not in running.

The issue is not with code tests, but how they're designed. If you're talking about getting asked to whiteboad BST re-ordering or to write a string palindrome without google or what is cyclometric complexity of this project or whatever, then yes you're totally right to walk out of there.

We do them. We had bad experience with a senior getting fired while on probation. Not a story to tell here. Another important reason to do them is to see code and git commit quality.

I designed some of the technical tests for my employer. They're based individual actions we need to be familiar with doing our job. For example backend,to show you can understand OOL, expose a function, commit history, etc. For front end its like heres a form, an api, make it look like this in html and css. 30-60 minute stuff.

We had a guy we really liked who aced the non tech interview until we gave him a 60 min test (reality 15 min test) and with ability to google what to do. Barely did anything, blamed the test, so I did in front of him while explaining what I'm doing. I could have been typing in Sanskrit as he couldn't explain what I did to solve it.

Amount of time working in industry means nothing in software development as its so varied and tech changing all the time.

[Bradston Henry]

That's a good point! I think it's important to not "toss the baby out with the bath water".

Personally, coding tests have never been a good metric to test my competence BUT that doesn't mean it's not a great one for others. At a minimum, if administered correctly, it can give you some insight into the developer and how they code and what is their "technical proficiency" in a subset of skills.

Amount of time working in industry means nothing in software development
as its so varied and tech changing all the time.

Sadly, this is BEYOND the truth. I think an important question to ask any developer (especially senior level) is how their skills have progressed over the years and also to see their technical skills in a demonstrable way.

It's sad to say, but I have worked with people who are much more "seasoned" than me but REFUSE to learn how things like Git works because it's not something they like, or, the worst excuse, because they are "not interested in that technology".

Thanks for sharing your thoughts. Always love to hear different and well-thought our perspectives.

[Maciek Grzybek]

Amen to that, a voice of reason :)

[cubiclesocial]

Coding tests are unfortunately necessary. There are plenty of people out there who can't code their way out of a wet paper bag but have impressive looking resumes. Search for "The Brillant Paula Bean."

That said, the best coding tests are those that reflect the sort of work that will be accomplished on the job itself. The existing developers at the company should take a specific but relatively simple task that they themselves have done on several occasions (e.g. a data processing exercise), remove business-specific information from the task, and then have potential candidates submit code examples. Record how long it took each candidate to submit their solution to the problem - shortest time is not necessarily the best candidate. Since it is a company-specific exercise and pretty easily changed, it is unlikely to have a ready-made solution when searching on SO.

Companies should also have an alternative to the coding test: The GitHub account. If someone has 10 or more relatively high-quality projects on GitHub that they created and actively maintain themselves and can provide in-depth details about the structure of each project, that is more than enough information about them as a developer and they should be able to bypass the coding test altogether. It's still a good idea to take the test as a mere formality just to prove their capabilities to the other side. But both sides can roll their eyes over the necessity of the test.

Another great alternative to the coding test is having the interviewee describe how they would go about solving a specific business problem. Most software development isn't the actual writing of code, which is important, but rather interfacing with other humans to gather requirements and work through whether some requirement needs to be rigidly coded into the software OR can simply be a policy and therefore the software can be a bit more flexible to accommodate a wider variety of unforeseen use-cases. I prefer software that is minimally invasive (i.e. few enforcements) and just relies on people to use their good judgement and only occasionally apply policies as the need arises. People tend to enjoy using less restrictive software that does that.

[Luke Shiru]

There are plenty of people out there who can't code their way out of a wet paper bag but have impressive looking resumes.

There are better ways to evaluate the skill level of a candidate. If a company needs to do a coding test to figure out the level of a candidate, then that's a company you should avoid.

The existing developers at the company should take a specific but relatively simple task that they themselves have done on several occasions [...]

You're basically making the candidate work on your project already, but without real data and without paying for it, and once they get into the real project, then they'll have to do the "same work" again but with real data this time.

You're kinda going in the right direction using "close to real world" problems, but you don't need to waste the candidate's time making them code the solution. You can figure out talking with them how they will solve certain problems, and if you go into a coding part, it can be minimal just to "illustrate" something that can't be put into words.

it is unlikely to have a ready-made solution when searching on SO.

What would be the problem if the developer is resourceful enough to get solutions from somewhere else. If the solution they find is a good solution, how is that less valuable than thinking the solution themselves. One of the biggest differences between seniors and juniors nowadays, is that seniors are better at googling ... but we all google stuff. It will be very hypocritical to say you never googled anything.

Companies should also have an alternative to the coding test: The GitHub account [...]

And what happens if the developer has done the great pieces of software in private or company owned repos? Some folks work exclusively for companies, and that doesn't make them less valuable than folks working on open source projects.

Another great alternative to the coding test is having the interviewee describe how they would go about solving a specific business problem [...]

This! Why would you need an exercise having this! You have the best approach as an alternative to an approach that wastes your and your candidate's time. Why not start with this, and if you really can't figure out the skill level of the candidate, just then maybe go to the coding exercise?

---

I believe that you aren't wrong, you just have the order of your priorities when interviewing kinda upside down. My main position is to avoid stuff that will annoy me if I was the candidate.

[marysonthego]

Hi Bradston! I like this Einstein quote too, "I never commit to memory anything that can easily be looked up in a book" and "Never memorize what you can look up in books." (The second version is found in "Recording the Experience" (10 June 2004) at The Library of Congress, but no citation to Einstein's writings is given).
en.wikiquote.org/wiki/Albert_Einstein

This is important. I think a better test might be to present some problem to the interviewee and ask them to explain how they would go about solving it. You would get a good idea of their problem-solving approach. Asking them to regurgitate the syntax for some specific algorithm says nothing about their range of problem solving approaches, their creativity, or their willingness to ask others for help.

[Canu667]

Leetcode - all big tech, FAANG (MAANG?) companies use it. I have the impression that generally, people hate them.

I am personally divided on this subject. Yeah, if a company gives you a HackerRank link and the feeling that "if you can't do these then don't even bother" then this sends a really bad image. However, if you are tasked with solving the puzzle together with a second engineer then it makes more sense. Sure, you can always end up with a bad interviewer (the interviewer trainings are laughable IMO), but the objective, IMO, should be to find out how the candidate thinks, how he approches problems, asking questions, reasoning - yeah, it is nice to solve the coding puzzle, but I think this should not be a pass/fail approach.

What is more, coding tests show your willingness to learn things/standards. "I don't use it at my job" is a poor excuse. Sure, maybe DFS and BFS, but memoization? Also, I've never used/touched cryptocurrencies at my work - well, I am learning this right now.

Lastly, but not least, big companies have so many candidates that they can use whichever standard they want. Most of them use their own stack, so they need more good generalists to fill the body count.

I'd rather this than taking an assignment home, especially, when I am not paid for it.

[Thomas Cayne]

This is a great article and you make some excellent points. I started, self-taught, in software development (SD) in 1988. I have never gotten a job where 2 things factored in: 1) multiple interviewers, like a panel [always only 1-on-1] and 2) after taking a coding test.

Lately I found out (and think about this) that the developers who write these coding test/exam websites take literally MONTHS to write them. They have to go through ALL the process of the SD life cyce, not to mention the bugs they have to fix when it is in production.

Basically, candidates are required to take coding tests from platform that their very own developers cannot complete within the time required for us to do the test they wrote. Each time I take these test I feel heat in my head and I can't keep my eyes of the timer. I swear sometimes I hear the seconds ticking away. I tell myself one these days I would pass.

I quote from Stack Overflow: "...For the stuff I imagine that you want, the answer is that someone else probably already figured out how to do it...." So, me now, why "r_e-invent the wheel_"? The best part of SD today is "it probably already exist out there so we don't have to rewrite". Why do an array sort manually? Knowing how to find the right tools is a key part of the SDLC.

[Imaduddin Haetami]

I think coding test is necessary as there's a lot of fraud out there. People are claiming things that they cannot do or did not do. Unless the employer is very certain on someone's capability, then coding test may be skipped. In fact, coding test is helpful for beginners without professional experience to enter the job market.

[Luke Shiru]

If a company needs a coding exercise to figure out the skill level of a candidate, then that talks more about the company than the candidate. There are better ways of figuring out the skill level of a candidate, without waisting their time ... and about that "coding test is helpful for beginners without professional experience" ... its way more useful to hire them and put them to work in a real project instead of making them do coding tests that will only be "useful" for that interview.

[Abdelellah]

@lukaShiru, i just started following you. cant agree more

[Jon Randy 🎖️]

"You already know what you know, No need to prepare!"

This is so, so true. I live by this too. I never prepare for interviews. This advice first came to me from my A-Level physics teacher - in this form:

Revision is for wimps

For the most part, all last minute preparation does is stress you out and make you doubt your own abilities. Relax and trust yourself - that's the best plan

[Bradston Henry]

For the most part, all last minute preparation does is stress you out and
make you doubt your own abilities. Relax and trust yourself - that's the best
plan

I feel like this describes me exactly some times. I can get into my head and then everything goes out the window. I think that's why tests in general are a struggle.

[Bruno Paz]

"You already know what you know, No need to prepare!"

Great quote that matches exactly my way of thinking as well. :)

You definitely raise some interesting points in the article.

I have started doing interviews for my company some months ago, so I can see both sides now, here is my take on this:

No matter what interview process you have, it will be impossible to have a process that works for everyone. Whiteboard and leet code exercises favors candidates with CS Background or "geeks" that like to do this kind of things in their free time. Take Home exercises discriminates parents or people with lot´s of other responsibilities. Live coding discriminates against anxious persons or that cannot perform well under pressure.

Even "discussion based" interviews, can discriminate against introverts or people with vocal related disabilities for example.

So IMO, the best way to approach is to always give the candidate choice, what it works better for them. I agree in having a positive and inclusive process where the candidate can show their strengths, instead of an exclusive process or the interviews are designed for the candidates to fail.

My current company does a take home exercise as the 2nd stage of the hiring process, after a first technical interview. Personally I would gladly don´t require these if a candidate has some other code examples that they can show (like open source / personal projects or projects from his previous job, if possible.

But I have to see sort some code., if possible in a similar type of what they would do in the job.

I love great technical and architectural discussions about real world scenarios with the candidates and you can definitively get a lot of information from this, but there are some details I have not found yet a way to replicate in other form, without seeing code. And I also think these types of interviews are probably more appropriate for more senior level engineers, where we can really go deep in many topics.

Some things I access in the coding exercise, include :

• Tests. The candidate might say he know all the types of tests and do TDD and all that. but what they do in practice? The coding exercise allows me to asset that. How they think about the test scenarios? what kind of tests they do? Unit? Integration? An exercise without any tests is a red flag for me.

• Documentation - Does they write documentation and explain their thoughts (a Readme file indicating how to start the project is enough in most cases).

• Code structure - How they structure their code? Is it readable or they like to over engineer? do they provide meaningful names on the variables/functions? How they handle errors, logging, etc.

• Infrastructure - Can we run the application without errors? Candidates that provide a Dockerfile/Docker-compose or some sort of script to boot the application are appreciated.

A good example, I have from a candidate that I hired last month. The exercise my company provides, includes a "list" REST endpoint with Pagination. From like 10 candidates, I saw one, that have handled edge cases like negative numbers on the pagination for example. While definitely not a requirement, it´s a nice touch that caught my attention and shows some attention to detail and that differentiates from other candidates.

This is the kind of things I look on the code exercise. I don´t need them to solve a very complex problem or algorithm. Even an "hello world" like API project, could give me enough information.

I am of course, willing to learn new ways to make the process the better possible for the candidate and for the company.

[Bradston Henry]

Incredibly well said and great insight!

So IMO, the best way to approach is to always give the candidate choice,
what it works better for them.

I agree in having a positive and inclusive process where the candidate can show >their strengths, instead of an exclusive process or the interviews are designed for >the candidates to fail.

I love the idea of having a deep consideration about what works best for the candidate for them to shine in the interview scenario. Our goal is to hire the best candidate for the role. not the person who fails the least.

I read somewhere that one tendency that an interviewer has to try to avoid is the tendency to hire "Yourself". It's easy to sit across from a person and try to see if they pass the test of being as "good as you".

In almost every discipline that exists, diversity of thought is ALWAYS an advantage and I think overly rigid interviewing requirements or the lack of flexibility to approach candidates in a way that helps them shine is detrimental to the whole process.

It takes more works and "care" but engineering your interview process to best suit the various candidates is absolutely the best approach.

Thanks for sharing your thoughts. Gave me more insight into other's interviewing processes and things that I think every interviewer (and interviewee) should consider.

[yw662]

I think you are absolutely right but how should I get a job if I reject all those companies ?

[Ingo Steinke]

You can try until you find a company that trusts you upfront. You can make compromise and be less strict that @bradstondev , but refuse to do "homework" assignments for hours or days that give them free work even if they don't accept you, but it might be okay for you to take a small assignment.
You could try to start with a free, unpaid, intership wich might be turned into a regular, paid, employment.
You could, and should anyway, post some of your work in a public place: GitHub, a portfolio website, or even better some small app / theme / extension that has to pass external code review (like writing a WordPress plugin - code review is not very strict at all, but it still takes some effort) so you can prove that you can actually code. Use that code for interviews and explain what you did, and why, how it works, and what you would improve if you had time.

[Bradston Henry]

That's a GREAT question!! @yw662

@ingosteinke Makes great points. There are other avenues that can show your skillset that can be a "substitute" to what would be considered the traditional coding test methods. I didn't mention this in the blog post but one thing I did once I noticed that I was never going to be in consideration for certain roles because I wouldn't perform well on coding tests was begin to build a Github presence.

I have in the past been in the interview process that had a mandatory coding test where it was skipped because they saw my code in my Github repo and didn't feel it necessary to do it (I personally didn't even ask for the exemption). In the end, I didn't get the job (there was better candidates with more practical experience) but it accidentally "confirmed my bias" about why i don't think coding tests are required.

And just another piece of personal advice, it is ABSOLUTELY okay to ask early in the process if it is possible to do an alternative to coding tests. I have talked to recruiters in the past who when I asked, they mentioned that it was an option. You never know!

Also, sometimes there are harsh realities where you have to be subject to a system that puts you at a disadvantage until you have the "power" to change or "affect" it in a positive direction. In some ways, I was lucky to find a job opportunity that didn't require coding tests early in my career but know that I wouldn't have let that stop me either way. I truly believe that persistence takes you a long way. And as a programmer, persistence is something that really helps you be successful because sometimes you are going to run into those coding problems that require it.

I'm not sure if you are looking for a job or considering a new one, but I encourage you not got give up! Even if it does mean you have to do the thing I personally dislike (preparing for the interview beforehand), you never know what opportunities await you one the other side. And hopefully, you'll have the opportunity to be more "strict" in the future.

If you ever need counsel or advice, feel free to DM on Twitter!

[Ingo Steinke]

I had been an employee for about ten years after leaving my startup (which had not been a great success, but I have learned a lot) and have been considering changing jobs quite often, but mostly did not apply after recruiters told me about job details. When I did apply, there were some good and some bad interview situations, including clueless interviewers and standard questions like how would your perfect day look like...

After leaving the last company and getting the feeling that it was still nearly impossible to get a part time dev job in Germany, and that company culture sucks once you rather want more freedom than climb on the career ladder, so I found out it's a good time to freelance.

Now I have everything from very small individual customers to large corporate projects. Some companies do mostly the same interview process with freelancers that they do with aspiring employees. So even without "looking for jobs", I'm still looking for jobs sometimes and find myself in a candidate situation, but it will only be for a short time.

Indiviudual customers have been totally different, like we've seen your work, we liked it, we trust you to make something similar for us, how much will it cost?

[Bikash9609]

From my experience, in most of the companies, in the first round you are generally asked questions related to your skill. Questions which are easily answerable and don't need writing or practice, only to see that you know what you do.

In the second round, they just see if you are able to implement those skills in the best way possible through code - they give you a very simple or a bit complex question ( not DS or ALGORITHM ) real practical situation.

And, Done..!

[Tony B]

I prefer the "go home and write a program to do X" then bring it back in a week and explain the approach taken, and answer questions about it.

Sorts out the wheat from the chaff.

[Roger Filomeno]

Refactoring an existing code is also a good test.

[Matthew O'Brien]

The only thing that I can think of here is that, the quote from Albert Einstein doesn’t seem to stand here. Coding tests stand to measure the ability of people coding and their problem solving. Ie. you’re judging a fish on its ability to swim. That’s kinda the way I see it. Although you do bring up a good point. I do feel that testing programmers on complex data structure problems when they’re going to be a front end developer can be a tad annoying and not needed. However if you’re hiring someone for back end and they don’t know how to store items in a data structure to optimize for efficiency. That could lead to problems.