As the primary hiring manager for Software Engineers at DealerOn, I often get push-back from recruiters and questions from candidates about why we ask our candidates to complete a coding test prior to coming in for an interview. To put it simply, we believe that the benefits outweigh the drawbacks. But what are they? Let me explain…
Over the course of my career, I have seen several different hiring processes from companies. All of them start with a resume. Many ask for a cover letter. You or the recruiter representing you submits these to the hiring manager or HR recruiters at the hiring company and you wait. If you are one of the lucky few, you will be scheduled for a face-to-face interview with any number of technical and non-technical people that you would (presumably) work with should you get the job. This was the way it was done early in my career. Later, I noticed that companies were adding technical screening calls before the face-to-face, to eliminate any candidates who didn’t have the technical skills required.
During several interviews throughout my career I have been asked to code things on whiteboards or even with a pencil and paper. This always seemed odd to me, as we don’t write software that way. We have compilers and feature-rich IDEs with Intelli-sense to keep us from making silly mistakes and Google and Stack Overflow to look up things that we might know about but have forgotten exactly how to do. These methods also tend to make candidates very nervous, as it puts them on the spot in a way that will never happen in the actual work environment.
The kind of code test that DealerOn asks for is not done during the face-to-face interview. We provide the problem days in advance and give you ample time to solve the problem in any way you wish. The only requirements we place are that you use C# or Java, that it compiles, that it doesn’t use 3rd-party libraries, and that you write it yourself. You can use whatever online or in-person resources you desire, and the result can be a web application, a windows application, or a console application. However, keep in mind that as part of our interview process we ask candidates about their design choices in their code test, so you need to understand the choices you made. Candidates have complete freedom to use whatever design patterns they feel best suit the problem and can write as simple or complex code as they wish. Our preference is for C#, since that is the language that we primarily use here at DealerOn. However, Java is syntactically and structurally very similar, so a test written in Java can showcase your design abilities as well.
Why don’t all companies use a code test?
Not all companies use a code test. Most don’t, in fact. The reasons are pretty obvious.
It reduces candidate count
Some candidates, especially those that are highly-sought after, don’t feel they have time to do a code test. After all, they have some companies that don’t require a code test calling them back to do a face-to-face interview. I’m sure that some of those were great candidates who could have excelled here, technically, but the code test is part of our culture, so if you aren’t willing to take the time to do the test, you likely wouldn’t have fit in anyway.
It lengthens the hiring process
When the market is hot, as it is now, time is often not on our side. Candidates can get swooped up by other companies. Having a longer, more complex hiring process can cause us to lose out on candidates.
Our decision to use a code test as part of our hiring process has been the result of weighing the benefits against the drawbacks. At this point, even with the tight labor market, we still feel that benefit of getting the right candidates is worth potentially losing out on others from time to time.
Why does DealerOn use a code test?
We like the code test because we feel that the benefits outweigh the drawbacks. The main benefits we see are:
It allows us to consider candidates independent of their resume.
As part of our process, the engineers reviewing code tests do not have any fore-knowledge about the candidate. They don’t look at their resume or sneak a peek at the code before the evaluation begins. DealerOn doesn’t limit junior developers to menial tasks, and we don’t pigeonhole you based on the number of years that you’ve worked in the industry. If you can write high-quality software, we can provide a place for you to grow.
Candidates that do it really want to work here.
Think about this: If every company asked you to complete a code test, how many would you apply to? By asking for a code test, we know that those who submit one back to us really are motivated to work for us. Furthermore, I have heard from many candidates that they really enjoyed doing the code test. These are the kind of people we want here: People that enjoy writing software and are eager to show us their abilities.
It shows that your knowledge isn’t strictly theoretical.
Yes, the code test isn’t super complex. As a result, we cannot tell for sure, based upon your submission, that you can design highly-scalable web sites and services that can serve up our customers’ data in a reasonable time. But it does show us how much you care about the code you write, and that you might be able to succeed here. Coding style, maintainability, testability, code cleanliness, design pattern usage/knowledge all can be fully captured with our code test.
It gives candidates another way to set themselves apart.
How do we distinguish ourselves to prospective employers? The experience on our resume is part of it, as is the formatting and design of that resume, although often recruiters re-write them without our permission and format them with their style and company letterhead. Presenting yourself well at the face-to-face interview is essential of course, but what if you never get invited to one? Our code test allows candidates to set themselves apart from the pack by showcasing their coding skills. We’ve seen many creative and elegant solutions to our test’s fairly basic problems. We’ve also seen code tests riddled with mistakes that didn’t follow the instructions and solve the problem as stated. The more information we have in the hiring process, the more likely we are to make the correct decision about who to hire, and that’s a good thing.
Advice to Potential Candidates
Whether you intend to apply at DealerOn, or another company that has a code test as part of their hiring process, I have a few pieces of advice:
Take it seriously
This seems obvious, but I can’t tell you how many times I’ve seen submissions that looked like the candidate just “threw it together” in an hour. This is part of the hiring process. The more effort you put in here, the more likely you are to get the job. You wouldn’t show up to a face-to-face interview in your pajamas, would you? Submitting a code test without putting forth your best effort gives the same impression. Spelling errors, inconsistent variable and method naming, or non-object-oriented code show that you didn’t put much thought or care into your work and are red flags.
Follow the instructions
You’d be surprised how often this comes up, but it’s key. If you can’t follow the simple instructions we provide, how do we know that you’ll follow instructions if we hire you? Code that ignores constraints or gives invalid output are huge red flags.
Show off a little
Your resume tells us that you are a C# expert, or that you know all about the Factory design pattern. Show us! Use dependency injection, make unit tests - stand out! We really enjoy looking at well-crafted solutions. It shows us that you really enjoy what you do and would likely be a good fit here.
Conclusion
A code test can be seen as a burdensome extra step in the hiring process, but for a growing number of companies, including DealerOn, it is a way to get a better sense of who you are, how you think, and how skillful of a developer you are. A well-written and thoughtfully designed code test submission can move your resume to the top of the stack, regardless of your experience. Taking the time to do a good job tells us that you are likely a great fit for us.
Top comments (4)
I don't disagree with the code test in principle, but I think it makes a lot of assumptions about candidates. When I am looking for a new job because I am working 60+ hours a week, and I have 5 companies asking for a coding test, I have to prioritize which ones I attempt and which ones I abandon because short of sacrificing my mental and physical health further, I cannot complete it.
If I had a family or SO, I would much rather be spending time with them. I basically disappear from the lives of friends and family members during job searches. If a candidate has a child with a disability that demands intensive care, you're just going to assume they "[don't] really want to work here?" If I volunteer as an EMT and have to fit 6-24 hour shifts into my life, you're going to assume similar?
I understand your intention wasn't to discriminate with a code test, but that's a possible outcome. There needs to be reasonable limits to a code test. I would suggest timeboxing it to 4 hours or less, providing a boilerplate solution so candidates can spend time actually solving a problem instead of setting up Webpack, and accepting open-source contributions in lieu of an assignment.
Even with those extremes, things can still get out of hand. I had an employer ask me to design an API + front-end using idiomatic Go code in 3 hours that demonstrated secure coding principles. I've never used Go before and the employer knew that. This was for a security-related position, so I assumed honesty would be one of the most key factors behind my evaluation, so I turned in an incomplete solution, but gave a list of probably 50 bullet points of secure decisions that I could add to the site that I didn't have a chance to get to. I could tell that they were disappointed (almost to the point of bewilderment) when I admitted I didn't have a running solution because I would have to lie about how long I worked on it to accept it under the conditions I had been given.
They immediately ceased contact with me. By the way, what's up with companies being too lazy to tell candidates they are no longer under consideration? A canned e-mail is better than limbo. Obviously, personally calling the candidate and giving them constructive criticism would be the most ideal scenario, but it's not always possible. I remember the companies that ghost me and it forms a strong opinion about the company in my mind.
And regarding requirements, you better make sure your requirements are damn clear if you are going to take divergence from requirements as a strike against the candidate. What if you give terrible requirements? I always assume a company wants me to make decisions of my own and get something workable rather than bothering them, since the recruiter won't have a clue and you probably want your programmers working instead of having to conduct a requirements clarification session with every candidate.
Also, you said that non-OOP code is "not taking it seriously." What if I prefer functional programming and think that OOP is often a massively flawed solution? Apparently you're going to just make a snap judgment against me and not take me seriously because we have differing opinions on programming styles.
I have also noticed an inverse relationship between length of coding assignment and salary. Companies with ridiculously long coding assessments will almost always pay 15-20% less than companies with shorter assessments. Keep in mind your choice of assessment speaks about your company as well, and you can send loud and clear messages that you don't care about people (ghosting candidates), you disrespect their time or expect wage slaves (ridiculous assessment) or other messages by how you treat candidates and the expectations you set.
If you can't come up with a coding challenge that can accurately assess candidates without requiring them to waste many hours of their precious time, maybe your company doesn't view employees as people with diverse lifestyles outside of their job, lacks the technical knowledge to create a challenging assessment with reasonable completion time, is run by narcissists, has a workaholic culture, lacks empathy...you get the point.
tl;drCoding tests are a 2-way avenue. They say as much about you as they do about the candidate.
I don't disagree with much of what you said. Let me first get a couple of things out of the way, just to clarify:
DealerOn doesn't "ghost" candidates. We always review the code tests as soon as is reasonably possible (usually within 24-48 hours) and always get right back with them letting them know the result. While I frequently provide some detailed feedback to our recruiter to pass along, in one recent case the candidate send an email asking what they could have done better, as they were looking to improve. I gave a long response, as I really liked that attitude and felt an obligation to be as clear and transparent with the candidate, given their stated desire to learn from their mistakes on the test.
We have removed the time limitation from our code test for a couple of reasons. #1, it's not enforceable without having the person in our office while they're taking it. #2, things come up in your personal life and we didn't want people to feel that they needed to sacrifice just to get us a code test on time. The fact that we're almost always hiring makes this easier for us.
I actually have a special needs child, and am fully aware of the time commitments associated with that. That being said, I took DealerOn's code test prior to being hired here. It wasn't too bad, and can be done in a few hours if you're efficient.
Our code test is very basic, and the requirements are clear, except in areas where we specifically want to see how you can fill in the gaps. What we're usually looking for are basic attention to detail. For example, we provide a set of sample inputs and the expected outputs. When we receive a submission that doesn't work with the given sample inputs, it's a pretty bad sign. Also, not only do we not require installing webpack for our code test, we specifically state that we only want you to use what comes by default with .Net or Java, depending on what you use for your test. We don't want to have to spend hours setting up and environment just to evaluate your code test either. :-)
Bottom line: Do we miss out on candidates from time to time due to our code test? Almost certainly. But we've also been extremely successful in hiring great candidates and our turnover rate has been very low. The purpose of this article wasn't to defend DealerOn's use of a code test, but to recognize that code tests are a reality in the software industry at this time and there are a few things you can do to improve your chances of doing well on them. It isn't targeted at very senior engineers, as they likely know most of this anyway.
Thanks for the comments.
Many very good points.
Funny you should mention these, because they're all items that we either have currently, or are in discussions on integrating into our process. Our current style of coding tests are typically expected to take between 2-4 hours on average to complete based on your skill level. You're welcome to spend more time if you want to (no time limit to turn it in), but generally what you can accomplish in about 4 hours is enough of a gauge for what we're looking for.
Open-source contributions are a little harder to gauge, since they can have much wider variety than the relatively well defined box the code test fits inside of, but they can often be a more accurate portrayal of the applicant's skill set and coding styles.
Absolutely true. This is one of the main reasons that all of our current coding tests (we have 3 the applicant can select from) include sample input and expected output, as well as text explanations of the expected usage and requirements for the test. "Divergence from requirements" in this case means that an explicit requirement was ignored, or the stock provided test cases don't produce the provided expected output. We'll test a few extra cases as well and some edge cases to see how you interpreted the assignment, but the base cases are what cover the central logic that you're expected to implement, and if that works properly, everything else is just bonus. You're able to document your assumptions and can reach out to our team (not only the recruiter) if you have questions/misunderstandings of the assignment as well.
At the end of the day, we're looking for people that think and ask questions, not just folks to come in and blindly hammer away at a keyboard for hours. If a ticket or feature request comes through that doesn't make sense (granted, it likely would be caught and marked invalid long before reaching one of our engineers, but every one here's human, so it's not impossible for a ticket to slip through the filters eventually), we want that engineer to ask questions, get clarification, verify, and if necessary, push back on the ticket when necessary. If something doesn't seem right, you're empowered to raise the question.
All in all, your assessment of code tests was entirely valid and eerily spot on (we even have a volunteer EMT on our team as well).
From your profile, I see that you're currently looking for new opportunities. If that's still the case, based on your profile and LinkedIn, I think your skill sets and experience align pretty well with the tools and technologies we use here at DealerOn. Apparently you even have a recommendation from a former classmate of yours and current coworker of mine here at DealerOn.
We have a few full time remote folks now, and I believe we're open to expanding that for candidates with prior experience being fully remote (I see your location and most of your prior experience is in George, but you worked for a company in Maryland for a time, though no indication if that was remote or on site). If you're interested, I would encourage you to apply. I think your skill set, experience, and penchant for asking questions and calling out where you see ways to improve something would fit in well here.
I'm going to go ahead and apply! I came very close to applying right before I left Maryland/DC and decided to go back to Atlanta instead. If I can get through the process quickly, it's definitely a position I'd love to consider.