You may be familiar with a famous quote thats has been attributed to Albert Einstein:
"Everyone is a genius. But if you judge a fish by its ability to climb a tree, it will live its whole life believing that it is stupid."
Now there is some debate if Einstein actually said or wrote this BUT the wisdom, nonetheless, rings true:
If you judge everyone by the same metric, there is a chance that you may not get an accurate measure of success.
Sadly, during my 7+ years working as a professional developer in the tech industry, I have seen a metric used to measure developer and programmer success that in MOST cases does not match the actual skills needed to be successful.
And if you have at any point attempted to apply for a job or interview for a position, there is a STRONG possibility you have encountered this metric; Coding Tests.
Coding tests come in several different forms but at their core, their goal is to test your proficiency with computer science principles, algorithms, and general CS knowledge. Coding tests can be delivered in-person on a whiteboard, virtually in a coding sandbox environment or in live-virtual environment where an interviewer asks you to share your screen and watches you work.
Fundamentally, coding tests are non-problematic. They are just a tool to measure a developer or programmer's competency and can be administered to most any job applicant fairly consistently.
But as with any tool, when used inappropriately or without the proper understanding of its usefulness, they can do more harm than good.
So let me briefly summarize why I no longer interview with companies that require a coding test.
NOTE: I am specifically using the word "REQUIRE" as I do not have an issue with a company that uses coding tests as one of their many metrics to evaluate candidates. I am particularly talking about companies that evaluate all technical developer/programming candidates via a coding test and DO NOT have an alternative option for a coding test in place.
1. It Shows a Lack of Care or Concern
So I can understand in the grand scheme of things why coding tests are useful. They allow a hiring team to quickly screen a large amount of candidates and gives a fairly easy metric to judge candidates by. It it by no coincidence that most very large tech companies use them in their initial stage of the interviewing process as a means to weed out "low competency" candidates.
But it is this EXACT reason that coding tests being a requirement in an interview process shows a lack of care or concern. It shows that the goal in this interview process is to weed out candidates, regardless of their wholistic strengths.
It doesn't matter if the candidate is a self-taught programmer, a high-performing developer who has worked in the industry for decades, or a college student pursuing their first opportunity, coding tests only evaluate you on a single metric. And if you have spoken with many recruiters or HR professionals at some of the bigger companies, it acts as the ultimate gate keeper to opportunities.
Being a tech professional who has been interviewee, interviewer, and sometimes just observed technical interviews, I have found that coding tests can be used to invalidate a developers experience and skill in an instance.
I have watched as EXTREMELY competent coder crumble when they were given a coding question that they struggled to answer and that I later found out, the interviewer didn't know how to solve. The interviewer simply went online found common coding questions and presented them to candidates. Luckily, I was able to participate in the interview and "intervened". This particular story ended up having a happy ending but not every interview will have an "observer" associated to it.
I have personally been in interviews where I have done stellar on all technical questions related to the actual role (as explicitly stated by the interviewers in the process) but was no longer considered a viable candidate because I had a hiccup trying to solve an esoteric algorithm coding question that would NEVER be used on the job.
Personally, a coding test requirement in the interview process indicates to me that an employer and an hiring team aren't really in touch with what makes a quality developer and how to measure actual success. But are looking for any easy way to comb through candidates and minimize interactions with candidates until they are further down in the process.
2. Some Candidates are at a Massive Disadvantage
This point personally hits home for me, but there are circumstances that can make a coding test extremely unfair to candidates looking for an opportunity at a company.
Imagine we threw away everything I said in my first point. Imagine that all coding tests were administered by hiring professionals and teams that genuinely cared about their prospective candidates. Even if the hearts of every interviewer and person in the hiring process was absolutely pure, coding tests would still put an undue burden on otherwise competent and valuable candidates.
How so?
Well it comes down to the fact that some people are ALWAYS going to perform badly in a code "testing" setting. May it be someone with a learning disability, an individual with forms of anxiety that affect testing performance, or someone who's background makes the testing material extremely difficult. And know, this is not an exhaustive list but a representative list of limiting factors to coding test performance.
Sadly for me, I've ALWAYS struggled in high pressure testing environments (ask my college GPA about that 😅) and I've also struggled with coding tests because as a self-taught programmer, things like algorithms and CS concepts are something I didn't really encounter until later in life.
Because of those factors, it makes coding tests extremely difficult. Even in coding tests where I have done "okay", it still didn't feel good. And though I have always performed extremely well at any role I have ever had the opportunity to work in, a coding test metric would show me, in most cases, as absolutely incompetent.
There was one occasion where I was going through an interview process and performed will at almost every stage. At the very end of the process, the recruiter was giving me amazing feedback and was about to end the interviews for the day. (NOTE: This was an on-site multi-hour interview.) As I was heading out, I was asked if I'd be willing to do an "extra" interview with one of the "higher-ups" in the company and I was fine with that. What I didn't know, was that they were going to administer a "test" to me. Within the course of 15-minutes, I lost the job. My anxiety destroyed me. EVEN THOUGH, I was found to be competent and qualified by others in the process, my failure of this one "test" ended my chances.
That was a very personal experience, but I have a feeling that I may not be the only person who has been summarily disqualified from a position because of a poor performance on a test (some which don't even cover skills that would be used on the actual job).
3. It's Not Worth It!
I have a personal conviction when it comes to interviews, that goes like this:
"You already know what you know, No need to prepare!"
Once again this is a personal conviction (which I don't expect anyone else to have) so my personal mentality with interviews has always been to do no extra preparation other than maybe reviewing what I already know (and brushing my hair and looking presentable 😅). I want interviewers to have the most honest reflection of my skills. I want them to see me as I would be if I worked for them on a daily-basis.
I have on multiple occasions in the past been told I need to "study for this interview". I would then be sent a set of links or a PDF on how to prepare for this interview and necessary skills I need to learn to "pass" it.
When I was looking for my first job in the industry and I had almost no experience or knowledge that I could exemplify in a demonstrable way, I kind of got it. Companies need to know that I can learn and that I'm willing to take challenges, so fair-play. But interestingly enough, I got my first job in the industry without having to study or take an explicit test; They did test my knowledge but it was relative to the job role itself.
Now, as an experienced professional, I personally refuse to study for a job interview for skills that I will not use on the job. It's just not worth it.
With the busy-ness of life and the difficulty of finding time to study with a full-time job, in most cases, I have been asked to study concepts and materials that I almost never have and never will use in my day-to-day. There is a good chance that after I pass the test that I may NEVER use the knowledge I studied for in a meaningful way when I do land the job.
I have from time-to-time contemplated maybe studying for a coding test and then realized I'd rather spend my time learning skills that I will most likely use in the future. It many ways, I feel that asking to study for a job interview should not be the standard. You should be evaluating the candidate based on their current skill, not on a skill that they don't currently possess.
Now, at the end of the day, this is all my personal opinion and comes from my time in the industry but I'd love to hear other people's thoughts. I am not completely against "coding tests" per-say, but I do feel that they are overused and/or misused in the tech industry. At a minimum, I think there should always be an alternative to a coding test OR that it should NOT be used as a "gate-keeping" mechanism in the interview process. If anything, just like behavioral tests that are given during some job interview processes, it should be used as a frame of reference, not the deciding factor.
So do ya'll think coding tests are worth it? Do you disagree with my opinion/take on the matter? Did I get something wrong or make a point that doesn't make much sense?
Please let me know below in the comments. I'd love to hear your thoughts!
Thanks for reading, my friends,
Bradston Henry
And If you are interested in checking on what I believe to be better ways to interview, check out my blog on tech interview approaches:
===CREDITS===
- Cover Photo: Monstera from Pexels
- Education System Photo: Unknown
- Woman Shrugging Photo: Polina Zimmerman from Pexels
- Disappointed Woman Photo: Andrea Piacquadio from Pexels
- Man's Smartwatch Photo: cottonbro from Pexels
Top comments (139)
I find it fascinating that people build coding tests to test what people know, to see if they are fit for a given job.
Then candidates study those tests so they can get the job.
Then people analyze their performance to see if they are fit for a given job and then continue the process with more tests.
The only issue is if the coding test is nothing like the actual job, everyone else is spending time and effort dealing with what is ultimately unrelated to the actual job.
If the actual job is very coding-test-like, then sure that's an excellent representation. If its not, isn't it a huge waste of time for everyone?
I feel exactly the same way. It's 100% appropriate if the job itself has a lot of tasks that might mimic the problems you have to tackle in a coding test, but thats seems more like the exception than the rule.
And it would be quite a tragedy if you performed extremely well on a coding test, started a job and realized that you didn't have any of the skills needed to do the job.
To think that a coding test could be a waste of time for anyone involved. Not good.
I agree with you, and I've been there.
Three years ago I applied for a company that required a JS test with tests in Jest. The test was basically to input any number and output this number's name written (102 -> one hundred and two). I nailed it, reviewed the tests and the code 20 times, refactored and so. The interviewer loved it!
I got the job and then I realized: they were expecting a much higher level dev which should do JS AND React AND Ruby/Rails (I was aware of the stack needed before interviewing, but the test level was much easier than what I was expected to deliver in the actual job).
That's tough! So sorry to hear that happened!
How did you handle it when you started working the actual job?
Had to be super discouraging to run into that!
Thank you for your kind words :)
In fact it was tough... I could handle well with the React stuff, but on the backend side I was still a junior dev. As the time passed I could feel that I was not attending their expectations, and one year later they said that they needed someone with more advanced skills in the team. However they were super kind and let me stay in the company until I find a new job, which took two months. Not everything was bad at all :D
Wow! That's incredible!
Even though I'd never wish that on anyone, I'm actually amazed by their empathy and consideration in your situation.
Really happy things worked out for you! That's an amazing story!
I personally think really a lot of abilities that help you perform well in job also help perform well in coding tests. But it of course also depends how coding interview is done from interviewer side.
I see coding interview more like a pair programming session. While problem is synthetic the methods used to problem solve and the skills that are exposed during this session are all relevant to day to day job.
I had chance to run a number of code interviews over last couple months, and most common problems were:
I think neither of this is in any way some esoteric situation. It's just daily work stuff. On top of that if problem is really more on esoteric side and has some things that candidate needs a "click" in brain to happen, I try to do my best to help guide them so that this click would happen.
When I think of daily work stuff I think of doing a feature and PR review.
A common alternative I hear to a whiteboard interview, or a coding challenge is a take home project.
A take home project can not only test more relevant skills, it also provides a more real-world experience for everyone. With PR-like feedback the experience would also provide a similar setup to a work environment, and allow you to uncover all those problems you described above.
This is all without all the stress you'd get from needing to code/solve a problem immediately while someone else watches.
I'm not saying all companies should do this, or that all companies could do this, but it does seem more reasonable and relevant than dropping someone into some "test environment" and pointing out the similarities, rather than just putting them into a real world "work environment" and having a guardrails setup (fake inputs, fake data, relevant but fake project).
With take home tasks problem is that some candidates would spend 1 hr others might spend whole day.
exactly the same will happen when they are part of your team, no matter if they work from home or sit in an office. Doesn't matter much when they deliver good quality.
Getting paid by time spent is not much better than getting paid by lines of code, but it seems to be the most common measurement in our business somehow.
I think it's the other way around. Company is much more intetested in someone who can do a task in an hour, than in one that can do same stuff in 8 hours.
The last bullet point:
have you run across companies that seem to not care about or even think programmers should be able to read and comprehend and manage bad naming and obscure code such as literal numeric constants in conditions?
It sometimes feels like I get the short end of the stick either way:
besides the fact that NO, SORRY, NOT EVERYBODY IS A GENIUS, (maybe different, maybe special but not genius) I always loved the quote:
and found arrogant and patronizing when people "judge" a person in all their complexity based on just few parameters.
But I love it only when applied in general or just to the education system.
A kid could be good at Math, and suck at art.
A kid could be good at Drawing and suck at Science.
Both are very intelligent and special kids, teachers should remember them, respect their different abilities and natures. Try to foster where they thrive and support them were they lack.
But.. If we are talking about a specific thing, a specific skill that we are assessing, this quote is meaningless.
If I visit a Music school, and I suck at anything musical, I cannot just say:
If the fish applied to a Tree Climbing competition, should we really take into account his ability to swim? Or should we, with good reason, consider him stupid for even applying?
I am sorry, but here we are talking about hiring someone to become a Software Engineer, so the ability of coding must be assessed.
We can discuss how, in which ways, with which tools, but a "coding assessment" is necessary.
It might be stressful, it might be timeconsuming, it might possibly weed out valid candidates, but it is necessary.
as everything in Software Engineering, and basically in life, IT DEPENDS, mainly
If you apply for a Solutions Architect position, you can't just say: well... I am not good a drawing diagrams and not so good at naming System Design principles..."
If you apply for Frontend Engineer with React experience, you must be able to quickly jot down a component with a couple of hooks.
If you are a Javascript Engineer I expect you to know about Currying and Partial application.
Of course there are loads of stupid companies and interviewers that fail because the apply interviewing patterns that make no sense for their company and daily job. But it is not a flaw of the coding test.
If the job deals with algorithms, performance and really complex low level stuff, you must have a CS background. (be it self taught or official). If the job envolves solving problems with code ( and namely, building some components, and connecting apis and using services already built for the job - be it a library, or a Cloud provider service) way less.
I also had interviews where i was asked to reverse a string or sorting an array without using any native methods.. which is something that makes no freaking sense in a real world scenario. I failed them miserably, but I left without any sense of defeat - either I would not be a fit for their daily tasks, or the company mindset would not be a fit for me.
The point is, hiring is hard ( and being hired too).
There is no right way to do that. So in the end, it is really up to your gut feeling, and the company that you applied for.
I really like your thoughts on this!
I think it's interesting because the issue isn't the Coding Tests in their purest form, it may be an implementation problem.
At the end of the day, Recruiters, HR professional, and development teams NEED some way to judge if a candidate is competent. And sadly, just taking a person at their word is probably NOT the way to go 😅
I think you statement really rings true:
I think it's INCREDIBLY important to trust yourself and know what is best for you and evaluating your skills. For me, it's not coding test but for others it may be.
Exactly my thoughts. No matter what companies will do, always will be someone complaining about the recruitment process. Currently is most visible as the project-based vs test-based check of experience.
This is for me, one of the best answers I have read. Personally, I agree, you cannot be a solution architect and don't know how to create diagrams. I am a "huge" fan of the C4 model.
Furthermore, I agree with the point that if you are applying for jobs working with metal or I don't know nuclear plants, I agree that you must have certain skills. However, the main issue I have experienced in Europe is that I used to have too theoretical code interviews for things you would never use. When I was searching for a job, I never applied for optimizing OSs or building Machine Learning Models positions, and I still had questions of topics I hadn't used in 10 years since I had graduated from CS.
In my first interviews I failed horribly, and I was forced to study things that I don't use, and I already forget again. I still have a crazy sticky note in my phone if I ever need it again with all potential questions that I got that I didn't remember in detail.
I can just add that I'm not against code interviews, in the only company I worked in Poland that was a large corporation, we had technical tests, but they were about a small business case that was related to your future job. We had some CRUDs and other stuff, and you could choose the .NET tech you wanted. After your solution was done in 1h more or less, you will have questions and based on that we proceed. I consider these tests fairer and with more sense.
We avoided extremely theoretical tests where people had no idea about .NET, LINQ, T-SQL, etc. but they knew all sorting algorithms ever, and we focused on the people that our business needs. Sadly, I had seen more comparably sized companies doing the same Quick Sort or related algorithms for unrelated positions that when the times come and you start your job, they tell you things like this one: "You must migrate an entire DB instance on your own for X date." This happened to me, and I succeeded, but not everyone is so lucky.
agreed 100% with you . best comment and answer
I don't think the meme really fits, because it implies answering academic "implement bubble sort" type questions is more difficult than solving real-world software engineering challenges. I don't think it's more or less difficult — it's just a different type of question altogether.
When have you ever had to sort an array in JS without using sort on a real job? When have you ever had to implement bubble sort on a real job? They're just trying to trick you because they are jealous they are not wicked smart like you. :)
Yeah the most difficult thing about sorting arrays in JavaScript is remembering which of
a - bvsb - a(ora.localeCompare(b)vsb.localeCompare(a)) means "ascending" and which means "descending" 😅As for performance, the browser implementation is already optimal (versus anything you could implement in JS itself, at least — possibly you could squeeze out some extra perf with WASM) in almost all cases.
Disagree: The meme is perfect. The difference is the very short time constraint and people watching you code. I can solve way more difficult problems given more time and space to figure things out. But in an interview even basic simple things can feel very difficult.
Exactly that! The interview is anything but a relaxed setup, no one will start cutting the code immediately when faced with a new challenge while someone else is counting his breathes! During the interview the anxiety high and the focus is low, the brain is surviving mode.
I am ok to read and explain a code, but not to cut code during an interview.
Not even a different type of question. It's highly relevant as a test.
We usually do coding tests not as a test of knowledge and skill, but more as a test of approach and methods. Applicants may ask as many questions as they want and a working solution at the end of the test is not a requirement. We'll rather have someone to ask the right questions than someone who thinks they know it all.
Totally agree with this one. When the interview is scheduled, it should be made clear that a coding test will be part of the interview, so that the interviewee isn't caught off-guard. Then, during the interview it should be made clear that the interviewee can take as much time as required and that it is advisable to ask questions which will help clarify the problem at hand. If these conditions are met, I don't see any issue in taking/conducting a coding test.
I once advised my manager against taking a senior developer who had aced the coding test with ease, but was also the most opinionated person I ever had the misfortune to meet and the quintessential antithesis to a team player. I really wouldn't have wished him on any of our teams – and the colleague who was conducting the interview with me immediately agreed.
On the other hand, an interviewee with a speech impediment who was obviously very nervous and barely managed to finish the coding test, but had asked the right questions, I could wholeheartedly recommend; he is now an estimated team member and was promoted to senior developer last year.
Oh, the gatekeepers, where would we be without them.
When hiring for jobs that have more applicants than openings, you always get gatekeepers as a necessity.
If you're lucky, they will also look for things like team dynamics and methodology. If not, I'd hope you'll find another job soon.
My argument is that if you're choosing people to run in a team for a marathon, its only fair to ask them to walk or jog on the spot first, rather than saying 'Oh he's a nice guy/gal' go through the whole process to onboard them and later find out on practice runs through office gossip that they are actually lame and they'll need a guy in the team to carry them across the finish line. Its only unfair if your tests are, dance the salsa or juggle footballs, as while it shows proficiency in foot use, but not in running.
The issue is not with code tests, but how they're designed. If you're talking about getting asked to whiteboad BST re-ordering or to write a string palindrome without google or what is cyclometric complexity of this project or whatever, then yes you're totally right to walk out of there.
We do them. We had bad experience with a senior getting fired while on probation. Not a story to tell here. Another important reason to do them is to see code and git commit quality.
I designed some of the technical tests for my employer. They're based individual actions we need to be familiar with doing our job. For example backend,to show you can understand OOL, expose a function, commit history, etc. For front end its like heres a form, an api, make it look like this in html and css. 30-60 minute stuff.
We had a guy we really liked who aced the non tech interview until we gave him a 60 min test (reality 15 min test) and with ability to google what to do. Barely did anything, blamed the test, so I did in front of him while explaining what I'm doing. I could have been typing in Sanskrit as he couldn't explain what I did to solve it.
Amount of time working in industry means nothing in software development as its so varied and tech changing all the time.
That's a good point! I think it's important to not "toss the baby out with the bath water".
Personally, coding tests have never been a good metric to test my competence BUT that doesn't mean it's not a great one for others. At a minimum, if administered correctly, it can give you some insight into the developer and how they code and what is their "technical proficiency" in a subset of skills.
Sadly, this is BEYOND the truth. I think an important question to ask any developer (especially senior level) is how their skills have progressed over the years and also to see their technical skills in a demonstrable way.
It's sad to say, but I have worked with people who are much more "seasoned" than me but REFUSE to learn how things like Git works because it's not something they like, or, the worst excuse, because they are "not interested in that technology".
Thanks for sharing your thoughts. Always love to hear different and well-thought our perspectives.
Amen to that, a voice of reason :)
Coding tests are unfortunately necessary. There are plenty of people out there who can't code their way out of a wet paper bag but have impressive looking resumes. Search for "The Brillant Paula Bean."
That said, the best coding tests are those that reflect the sort of work that will be accomplished on the job itself. The existing developers at the company should take a specific but relatively simple task that they themselves have done on several occasions (e.g. a data processing exercise), remove business-specific information from the task, and then have potential candidates submit code examples. Record how long it took each candidate to submit their solution to the problem - shortest time is not necessarily the best candidate. Since it is a company-specific exercise and pretty easily changed, it is unlikely to have a ready-made solution when searching on SO.
Companies should also have an alternative to the coding test: The GitHub account. If someone has 10 or more relatively high-quality projects on GitHub that they created and actively maintain themselves and can provide in-depth details about the structure of each project, that is more than enough information about them as a developer and they should be able to bypass the coding test altogether. It's still a good idea to take the test as a mere formality just to prove their capabilities to the other side. But both sides can roll their eyes over the necessity of the test.
Another great alternative to the coding test is having the interviewee describe how they would go about solving a specific business problem. Most software development isn't the actual writing of code, which is important, but rather interfacing with other humans to gather requirements and work through whether some requirement needs to be rigidly coded into the software OR can simply be a policy and therefore the software can be a bit more flexible to accommodate a wider variety of unforeseen use-cases. I prefer software that is minimally invasive (i.e. few enforcements) and just relies on people to use their good judgement and only occasionally apply policies as the need arises. People tend to enjoy using less restrictive software that does that.
I'm not questioning your view on whether or not this is a good or bad interview coding problem. I'm just commenting on the "supposedly that's faster" part.... The worstcase runtime of the various counting sort algorithms is O(n), whereas the best you can do in worst case with a comparison sort algorithm, like mergesort, is O(n log n). When a counting sort algorithm is applicable to the data, it can be extremely faster than mergesort and the other comparison sorts. In most common cases though, either its not applicable, or the array size isn't long enough to overcome the constant factors, or for other practical purposes.
This is probably not a good interview coding question in general though. Actually it isn't really a coding question at all. It is an algorithms question masquerading as a coding question.
Hi Bradston! I like this Einstein quote too, "I never commit to memory anything that can easily be looked up in a book" and "Never memorize what you can look up in books." (The second version is found in "Recording the Experience" (10 June 2004) at The Library of Congress, but no citation to Einstein's writings is given).
en.wikiquote.org/wiki/Albert_Einstein
This is important. I think a better test might be to present some problem to the interviewee and ask them to explain how they would go about solving it. You would get a good idea of their problem-solving approach. Asking them to regurgitate the syntax for some specific algorithm says nothing about their range of problem solving approaches, their creativity, or their willingness to ask others for help.
Leetcode - all big tech, FAANG (MAANG?) companies use it. I have the impression that generally, people hate them.
I am personally divided on this subject. Yeah, if a company gives you a HackerRank link and the feeling that "if you can't do these then don't even bother" then this sends a really bad image. However, if you are tasked with solving the puzzle together with a second engineer then it makes more sense. Sure, you can always end up with a bad interviewer (the interviewer trainings are laughable IMO), but the objective, IMO, should be to find out how the candidate thinks, how he approches problems, asking questions, reasoning - yeah, it is nice to solve the coding puzzle, but I think this should not be a pass/fail approach.
What is more, coding tests show your willingness to learn things/standards. "I don't use it at my job" is a poor excuse. Sure, maybe DFS and BFS, but memoization? Also, I've never used/touched cryptocurrencies at my work - well, I am learning this right now.
Lastly, but not least, big companies have so many candidates that they can use whichever standard they want. Most of them use their own stack, so they need more good generalists to fill the body count.
I'd rather this than taking an assignment home, especially, when I am not paid for it.
hey man. sign on EVERY of your words and each paragraph. HR industry in software engineering is fucked up. I am waiting when these all the hiring models will be destroyed and all of us come back to the same what was 15 years ago. So I can start working normally again.
The current models are very much suitable for new comers engineers, juniors, => which leads disproportional amount of them in the market comparing to leads and seniour=> which leads to actual fuck up of all engineering process. Once heads see it and understand they will revert process back to hiring competency.