You may be familiar with a famous quote thats has been attributed to Albert Einstein:
"Everyone is a genius. But if you judge a fish by its ability to climb a tree, it will live its whole life believing that it is stupid."
Now there is some debate if Einstein actually said or wrote this BUT the wisdom, nonetheless, rings true:
If you judge everyone by the same metric, there is a chance that you may not get an accurate measure of success.
Sadly, during my 7+ years working as a professional developer in the tech industry, I have seen a metric used to measure developer and programmer success that in MOST cases does not match the actual skills needed to be successful.
And if you have at any point attempted to apply for a job or interview for a position, there is a STRONG possibility you have encountered this metric; Coding Tests.
Coding tests come in several different forms but at their core, their goal is to test your proficiency with computer science principles, algorithms, and general CS knowledge. Coding tests can be delivered in-person on a whiteboard, virtually in a coding sandbox environment or in live-virtual environment where an interviewer asks you to share your screen and watches you work.
Fundamentally, coding tests are non-problematic. They are just a tool to measure a developer or programmer's competency and can be administered to most any job applicant fairly consistently.
But as with any tool, when used inappropriately or without the proper understanding of its usefulness, they can do more harm than good.
So let me briefly summarize why I no longer interview with companies that require a coding test.
NOTE: I am specifically using the word "REQUIRE" as I do not have an issue with a company that uses coding tests as one of their many metrics to evaluate candidates. I am particularly talking about companies that evaluate all technical developer/programming candidates via a coding test and DO NOT have an alternative option for a coding test in place.
1. It Shows a Lack of Care or Concern
So I can understand in the grand scheme of things why coding tests are useful. They allow a hiring team to quickly screen a large amount of candidates and gives a fairly easy metric to judge candidates by. It it by no coincidence that most very large tech companies use them in their initial stage of the interviewing process as a means to weed out "low competency" candidates.
But it is this EXACT reason that coding tests being a requirement in an interview process shows a lack of care or concern. It shows that the goal in this interview process is to weed out candidates, regardless of their wholistic strengths.
It doesn't matter if the candidate is a self-taught programmer, a high-performing developer who has worked in the industry for decades, or a college student pursuing their first opportunity, coding tests only evaluate you on a single metric. And if you have spoken with many recruiters or HR professionals at some of the bigger companies, it acts as the ultimate gate keeper to opportunities.
Being a tech professional who has been interviewee, interviewer, and sometimes just observed technical interviews, I have found that coding tests can be used to invalidate a developers experience and skill in an instance.
I have watched as EXTREMELY competent coder crumble when they were given a coding question that they struggled to answer and that I later found out, the interviewer didn't know how to solve. The interviewer simply went online found common coding questions and presented them to candidates. Luckily, I was able to participate in the interview and "intervened". This particular story ended up having a happy ending but not every interview will have an "observer" associated to it.
I have personally been in interviews where I have done stellar on all technical questions related to the actual role (as explicitly stated by the interviewers in the process) but was no longer considered a viable candidate because I had a hiccup trying to solve an esoteric algorithm coding question that would NEVER be used on the job.
Personally, a coding test requirement in the interview process indicates to me that an employer and an hiring team aren't really in touch with what makes a quality developer and how to measure actual success. But are looking for any easy way to comb through candidates and minimize interactions with candidates until they are further down in the process.
2. Some Candidates are at a Massive Disadvantage
This point personally hits home for me, but there are circumstances that can make a coding test extremely unfair to candidates looking for an opportunity at a company.
Imagine we threw away everything I said in my first point. Imagine that all coding tests were administered by hiring professionals and teams that genuinely cared about their prospective candidates. Even if the hearts of every interviewer and person in the hiring process was absolutely pure, coding tests would still put an undue burden on otherwise competent and valuable candidates.
How so?
Well it comes down to the fact that some people are ALWAYS going to perform badly in a code "testing" setting. May it be someone with a learning disability, an individual with forms of anxiety that affect testing performance, or someone who's background makes the testing material extremely difficult. And know, this is not an exhaustive list but a representative list of limiting factors to coding test performance.
Sadly for me, I've ALWAYS struggled in high pressure testing environments (ask my college GPA about that 😅) and I've also struggled with coding tests because as a self-taught programmer, things like algorithms and CS concepts are something I didn't really encounter until later in life.
Because of those factors, it makes coding tests extremely difficult. Even in coding tests where I have done "okay", it still didn't feel good. And though I have always performed extremely well at any role I have ever had the opportunity to work in, a coding test metric would show me, in most cases, as absolutely incompetent.
There was one occasion where I was going through an interview process and performed will at almost every stage. At the very end of the process, the recruiter was giving me amazing feedback and was about to end the interviews for the day. (NOTE: This was an on-site multi-hour interview.) As I was heading out, I was asked if I'd be willing to do an "extra" interview with one of the "higher-ups" in the company and I was fine with that. What I didn't know, was that they were going to administer a "test" to me. Within the course of 15-minutes, I lost the job. My anxiety destroyed me. EVEN THOUGH, I was found to be competent and qualified by others in the process, my failure of this one "test" ended my chances.
That was a very personal experience, but I have a feeling that I may not be the only person who has been summarily disqualified from a position because of a poor performance on a test (some which don't even cover skills that would be used on the actual job).
3. It's Not Worth It!
I have a personal conviction when it comes to interviews, that goes like this:
"You already know what you know, No need to prepare!"
Once again this is a personal conviction (which I don't expect anyone else to have) so my personal mentality with interviews has always been to do no extra preparation other than maybe reviewing what I already know (and brushing my hair and looking presentable 😅). I want interviewers to have the most honest reflection of my skills. I want them to see me as I would be if I worked for them on a daily-basis.
I have on multiple occasions in the past been told I need to "study for this interview". I would then be sent a set of links or a PDF on how to prepare for this interview and necessary skills I need to learn to "pass" it.
When I was looking for my first job in the industry and I had almost no experience or knowledge that I could exemplify in a demonstrable way, I kind of got it. Companies need to know that I can learn and that I'm willing to take challenges, so fair-play. But interestingly enough, I got my first job in the industry without having to study or take an explicit test; They did test my knowledge but it was relative to the job role itself.
Now, as an experienced professional, I personally refuse to study for a job interview for skills that I will not use on the job. It's just not worth it.
With the busy-ness of life and the difficulty of finding time to study with a full-time job, in most cases, I have been asked to study concepts and materials that I almost never have and never will use in my day-to-day. There is a good chance that after I pass the test that I may NEVER use the knowledge I studied for in a meaningful way when I do land the job.
I have from time-to-time contemplated maybe studying for a coding test and then realized I'd rather spend my time learning skills that I will most likely use in the future. It many ways, I feel that asking to study for a job interview should not be the standard. You should be evaluating the candidate based on their current skill, not on a skill that they don't currently possess.
Now, at the end of the day, this is all my personal opinion and comes from my time in the industry but I'd love to hear other people's thoughts. I am not completely against "coding tests" per-say, but I do feel that they are overused and/or misused in the tech industry. At a minimum, I think there should always be an alternative to a coding test OR that it should NOT be used as a "gate-keeping" mechanism in the interview process. If anything, just like behavioral tests that are given during some job interview processes, it should be used as a frame of reference, not the deciding factor.
So do ya'll think coding tests are worth it? Do you disagree with my opinion/take on the matter? Did I get something wrong or make a point that doesn't make much sense?
Please let me know below in the comments. I'd love to hear your thoughts!
Thanks for reading, my friends,
Bradston Henry
And If you are interested in checking on what I believe to be better ways to interview, check out my blog on tech interview approaches:
===CREDITS===
- Cover Photo: Monstera from Pexels
- Education System Photo: Unknown
- Woman Shrugging Photo: Polina Zimmerman from Pexels
- Disappointed Woman Photo: Andrea Piacquadio from Pexels
- Man's Smartwatch Photo: cottonbro from Pexels
Oldest comments (145)
I find it fascinating that people build coding tests to test what people know, to see if they are fit for a given job.
Then candidates study those tests so they can get the job.
Then people analyze their performance to see if they are fit for a given job and then continue the process with more tests.
The only issue is if the coding test is nothing like the actual job, everyone else is spending time and effort dealing with what is ultimately unrelated to the actual job.
If the actual job is very coding-test-like, then sure that's an excellent representation. If its not, isn't it a huge waste of time for everyone?
I feel exactly the same way. It's 100% appropriate if the job itself has a lot of tasks that might mimic the problems you have to tackle in a coding test, but thats seems more like the exception than the rule.
And it would be quite a tragedy if you performed extremely well on a coding test, started a job and realized that you didn't have any of the skills needed to do the job.
To think that a coding test could be a waste of time for anyone involved. Not good.
@bradstondev So cool running across this article, because I just backed out of an interview today because of it. Before yesterday I was contacted by a pretty huge and growing entity which I thought would be a cool space to work in. The 1st screening went well, well enough that the interviewer I talked with felt very positive about moving me forward to the next stage of the interview. Cool. No foreseen issues, right? When I received an invitation to the next stage of the interview, I was directed to familiarize myself with CoderPad. Never heard of it, but after reading through the documentation and dug deeper to try to understand as a light programmer, why I needed to get grilled on a coding-skill checker such as CoderPad. Very intuitively it became apparent that this wasn't really about why they contacted me to seriously consider me as a candidate to hire, but a way to weed out whether or not I could code or not. Transparently, I'm not a coder but well-skilled with a handful of high-level languages that I utilize in my day-to-day engineering role(s) with my current employer and previous employers who grew and presented profitable projects to me to initiate, architect and launch with careful production success.
I've utilized the tools within each entity that I've been apart of per the requirements of the companies that advance with them, so my resume reflects honestly. However the goal I found out from this particular interview journey I was very skeptical of, while it truly being a tool to vet true coders from non-coders, the job description I had applied for only required less than a handful of those tools having the need to build raw code in whatever flavor that was needed to leverage the role. This particular role, maybe less than a 1/4%. The tool I'm most familiar with already has tested and proven code that could be inserted or modified to fit the automation tasks to perform for deployment to endpoints, but nothing heavy duty. The CoderPad they use for interviewing was a bit of a turn off and a diversion to weed out cheaters, copy and paster's and non-coders...no encouragement to help potential prospects to grow without intimidation. Don't get me wrong I do understand that the tool DOES weed out the non-coders but it just felt really negative based on the video I later found that was more of a interview from hell vetter.
So, this was helpful to cancel the arduous, mind-numbing stupidity of discarding really talented individuals due to the lack of face-to-face skill that really helped folks land good roles and keep that person-to-person interaction that's almost long gone in today's culture. Not worth it, and the compensation sucked not to mention.
I agree with you, and I've been there.
Three years ago I applied for a company that required a JS test with tests in Jest. The test was basically to input any number and output this number's name written (102 -> one hundred and two). I nailed it, reviewed the tests and the code 20 times, refactored and so. The interviewer loved it!
I got the job and then I realized: they were expecting a much higher level dev which should do JS AND React AND Ruby/Rails (I was aware of the stack needed before interviewing, but the test level was much easier than what I was expected to deliver in the actual job).
That's tough! So sorry to hear that happened!
How did you handle it when you started working the actual job?
Had to be super discouraging to run into that!
Thank you for your kind words :)
In fact it was tough... I could handle well with the React stuff, but on the backend side I was still a junior dev. As the time passed I could feel that I was not attending their expectations, and one year later they said that they needed someone with more advanced skills in the team. However they were super kind and let me stay in the company until I find a new job, which took two months. Not everything was bad at all :D
Wow! That's incredible!
Even though I'd never wish that on anyone, I'm actually amazed by their empathy and consideration in your situation.
Really happy things worked out for you! That's an amazing story!
I personally think really a lot of abilities that help you perform well in job also help perform well in coding tests. But it of course also depends how coding interview is done from interviewer side.
I see coding interview more like a pair programming session. While problem is synthetic the methods used to problem solve and the skills that are exposed during this session are all relevant to day to day job.
I had chance to run a number of code interviews over last couple months, and most common problems were:
I think neither of this is in any way some esoteric situation. It's just daily work stuff. On top of that if problem is really more on esoteric side and has some things that candidate needs a "click" in brain to happen, I try to do my best to help guide them so that this click would happen.
When I think of daily work stuff I think of doing a feature and PR review.
A common alternative I hear to a whiteboard interview, or a coding challenge is a take home project.
A take home project can not only test more relevant skills, it also provides a more real-world experience for everyone. With PR-like feedback the experience would also provide a similar setup to a work environment, and allow you to uncover all those problems you described above.
This is all without all the stress you'd get from needing to code/solve a problem immediately while someone else watches.
I'm not saying all companies should do this, or that all companies could do this, but it does seem more reasonable and relevant than dropping someone into some "test environment" and pointing out the similarities, rather than just putting them into a real world "work environment" and having a guardrails setup (fake inputs, fake data, relevant but fake project).
With take home tasks problem is that some candidates would spend 1 hr others might spend whole day.
exactly the same will happen when they are part of your team, no matter if they work from home or sit in an office. Doesn't matter much when they deliver good quality.
Getting paid by time spent is not much better than getting paid by lines of code, but it seems to be the most common measurement in our business somehow.
I think it's the other way around. Company is much more intetested in someone who can do a task in an hour, than in one that can do same stuff in 8 hours.
The last bullet point:
have you run across companies that seem to not care about or even think programmers should be able to read and comprehend and manage bad naming and obscure code such as literal numeric constants in conditions?
It sometimes feels like I get the short end of the stick either way:
I also am not a big fan of coding tests.
What I'd rather see is "we'll hire you as a contractor for 3 months, and if it works out we'll offer you a full time position". (Hmmm, I've seen internships turn out that way....)
But that "try-to-buy" probably would not work, since it is a gamble for the potential new hire who takes on all the risk, and of no risk to the company.
yea, the "try-to-buy" method is an interesting thought but like you said it's kinda risky.
Definitely risky for the potential hire and I do think it's risky for the company as well. They'll have to devout resources to train you, onboard you, etc and then you might be a bad fit.
Maybe it would have been better to just wait it out and find a "better" candidate. It's a difficult conundrum.
Some companies do the interview process really well while others just phone it in with coding tests (imo)
Hi Bradston! I like this Einstein quote too, "I never commit to memory anything that can easily be looked up in a book" and "Never memorize what you can look up in books." (The second version is found in "Recording the Experience" (10 June 2004) at The Library of Congress, but no citation to Einstein's writings is given).
en.wikiquote.org/wiki/Albert_Einstein
This is important. I think a better test might be to present some problem to the interviewee and ask them to explain how they would go about solving it. You would get a good idea of their problem-solving approach. Asking them to regurgitate the syntax for some specific algorithm says nothing about their range of problem solving approaches, their creativity, or their willingness to ask others for help.
This is so, so true. I live by this too. I never prepare for interviews. This advice first came to me from my A-Level physics teacher - in this form:
For the most part, all last minute preparation does is stress you out and make you doubt your own abilities. Relax and trust yourself - that's the best plan
I feel like this describes me exactly some times. I can get into my head and then everything goes out the window. I think that's why tests in general are a struggle.
I think coding test is necessary as there's a lot of fraud out there. People are claiming things that they cannot do or did not do. Unless the employer is very certain on someone's capability, then coding test may be skipped. In fact, coding test is helpful for beginners without professional experience to enter the job market.
Coding tests are unfortunately necessary. There are plenty of people out there who can't code their way out of a wet paper bag but have impressive looking resumes. Search for "The Brillant Paula Bean."
That said, the best coding tests are those that reflect the sort of work that will be accomplished on the job itself. The existing developers at the company should take a specific but relatively simple task that they themselves have done on several occasions (e.g. a data processing exercise), remove business-specific information from the task, and then have potential candidates submit code examples. Record how long it took each candidate to submit their solution to the problem - shortest time is not necessarily the best candidate. Since it is a company-specific exercise and pretty easily changed, it is unlikely to have a ready-made solution when searching on SO.
Companies should also have an alternative to the coding test: The GitHub account. If someone has 10 or more relatively high-quality projects on GitHub that they created and actively maintain themselves and can provide in-depth details about the structure of each project, that is more than enough information about them as a developer and they should be able to bypass the coding test altogether. It's still a good idea to take the test as a mere formality just to prove their capabilities to the other side. But both sides can roll their eyes over the necessity of the test.
Another great alternative to the coding test is having the interviewee describe how they would go about solving a specific business problem. Most software development isn't the actual writing of code, which is important, but rather interfacing with other humans to gather requirements and work through whether some requirement needs to be rigidly coded into the software OR can simply be a policy and therefore the software can be a bit more flexible to accommodate a wider variety of unforeseen use-cases. I prefer software that is minimally invasive (i.e. few enforcements) and just relies on people to use their good judgement and only occasionally apply policies as the need arises. People tend to enjoy using less restrictive software that does that.
We usually do coding tests not as a test of knowledge and skill, but more as a test of approach and methods. Applicants may ask as many questions as they want and a working solution at the end of the test is not a requirement. We'll rather have someone to ask the right questions than someone who thinks they know it all.
Totally agree with this one. When the interview is scheduled, it should be made clear that a coding test will be part of the interview, so that the interviewee isn't caught off-guard. Then, during the interview it should be made clear that the interviewee can take as much time as required and that it is advisable to ask questions which will help clarify the problem at hand. If these conditions are met, I don't see any issue in taking/conducting a coding test.
I once advised my manager against taking a senior developer who had aced the coding test with ease, but was also the most opinionated person I ever had the misfortune to meet and the quintessential antithesis to a team player. I really wouldn't have wished him on any of our teams – and the colleague who was conducting the interview with me immediately agreed.
On the other hand, an interviewee with a speech impediment who was obviously very nervous and barely managed to finish the coding test, but had asked the right questions, I could wholeheartedly recommend; he is now an estimated team member and was promoted to senior developer last year.
Oh, the gatekeepers, where would we be without them.
When hiring for jobs that have more applicants than openings, you always get gatekeepers as a necessity.
If you're lucky, they will also look for things like team dynamics and methodology. If not, I'd hope you'll find another job soon.
github.com/Financial-Times/polyfil... Look up the polyfill, import, done
I think you are absolutely right but how should I get a job if I reject all those companies ?
From my experience, in most of the companies, in the first round you are generally asked questions related to your skill. Questions which are easily answerable and don't need writing or practice, only to see that you know what you do.
In the second round, they just see if you are able to implement those skills in the best way possible through code - they give you a very simple or a bit complex question ( not DS or ALGORITHM ) real practical situation.
And, Done..!
You can try until you find a company that trusts you upfront. You can make compromise and be less strict that @bradstondev , but refuse to do "homework" assignments for hours or days that give them free work even if they don't accept you, but it might be okay for you to take a small assignment.
You could try to start with a free, unpaid, intership wich might be turned into a regular, paid, employment.
You could, and should anyway, post some of your work in a public place: GitHub, a portfolio website, or even better some small app / theme / extension that has to pass external code review (like writing a WordPress plugin - code review is not very strict at all, but it still takes some effort) so you can prove that you can actually code. Use that code for interviews and explain what you did, and why, how it works, and what you would improve if you had time.
That's a GREAT question!! @yw662
@ingosteinke Makes great points. There are other avenues that can show your skillset that can be a "substitute" to what would be considered the traditional coding test methods. I didn't mention this in the blog post but one thing I did once I noticed that I was never going to be in consideration for certain roles because I wouldn't perform well on coding tests was begin to build a Github presence.
I have in the past been in the interview process that had a mandatory coding test where it was skipped because they saw my code in my Github repo and didn't feel it necessary to do it (I personally didn't even ask for the exemption). In the end, I didn't get the job (there was better candidates with more practical experience) but it accidentally "confirmed my bias" about why i don't think coding tests are required.
And just another piece of personal advice, it is ABSOLUTELY okay to ask early in the process if it is possible to do an alternative to coding tests. I have talked to recruiters in the past who when I asked, they mentioned that it was an option. You never know!
Also, sometimes there are harsh realities where you have to be subject to a system that puts you at a disadvantage until you have the "power" to change or "affect" it in a positive direction. In some ways, I was lucky to find a job opportunity that didn't require coding tests early in my career but know that I wouldn't have let that stop me either way. I truly believe that persistence takes you a long way. And as a programmer, persistence is something that really helps you be successful because sometimes you are going to run into those coding problems that require it.
I'm not sure if you are looking for a job or considering a new one, but I encourage you not got give up! Even if it does mean you have to do the thing I personally dislike (preparing for the interview beforehand), you never know what opportunities await you one the other side. And hopefully, you'll have the opportunity to be more "strict" in the future.
If you ever need counsel or advice, feel free to DM on Twitter!
I had been an employee for about ten years after leaving my startup (which had not been a great success, but I have learned a lot) and have been considering changing jobs quite often, but mostly did not apply after recruiters told me about job details. When I did apply, there were some good and some bad interview situations, including clueless interviewers and standard questions like how would your perfect day look like...
After leaving the last company and getting the feeling that it was still nearly impossible to get a part time dev job in Germany, and that company culture sucks once you rather want more freedom than climb on the career ladder, so I found out it's a good time to freelance.
Now I have everything from very small individual customers to large corporate projects. Some companies do mostly the same interview process with freelancers that they do with aspiring employees. So even without "looking for jobs", I'm still looking for jobs sometimes and find myself in a candidate situation, but it will only be for a short time.
Indiviudual customers have been totally different, like we've seen your work, we liked it, we trust you to make something similar for us, how much will it cost?
Great article!
I completely agree that there should be no reason to 'study' for an interview.
If the test assesses you for the job (which it should), then you will already have a good understanding on what to do because of all your experience as a dev.
Some comments may only be visible to logged-in visitors. Sign in to view all comments.