All the tests have their place, if done appropriately. I'm very much in the "I'm not jumping through hoops" camp. But recently, I found myself on the opposite side of the table, knowing my employer didn't have the best reputation in the market. So I did something about it.
I effectively took complete control over the hiring process, and moved it from CV, pen & paper test (who even does that?!?) and formal in-person interview.
The new process is much more modular, all CVs come to me for filtering personally. At that point, I basically sort no/junior/mid/senior/rockstar. Since we're looking to build a team, "no" and "rockstar" are treated the same - I'm simply not in a position to have an uber brilliant coder that can't communicate.
From there, things get a little different. Basically:
juniors get a Skype call, during it, we give them two code examples with bugs, the second of which asks them to optimise. I share my screen & they talk me through what they would do. If they can't fix the basic bugs (syntax errors), they fail. Anything better than that & we can talk more. If they don't like my IDE, or the variable naming, I can use another IDE or refactor things. After the Skype call, it's decision time, no more hoops.
mid-senior & senior get given a set of requirements and a week long deadline. There's a total of 5 requirements (and one of those is the language & minimum version). I can crank out a solution in an hour to it, and other developers that I've given it to take a couple of days. But people have families (and other interviews to do), so we give more time just to be nice. After the test, we move on to a Skype call.
At the more junior end of the scale, I admit, I'm a little evil. I don't expect anyone to understand how to verify that a number is a Perfect Prime, so I ask them to do exactly that, live on the Skype call. But what I'm really looking at there, is how quickly a junior will throw up a flag & ask for help. Fail-fast & you look much better than if you bluff your way through it.
At the more senior end of the scale, things are a bit more relaxed. Submit code that doesn't even compile for all I care. I'm only looking to see how you think, to give me talking points during the Skype call, where I'll be asking why you did things the way you did. I don't want us to be in an echo chamber, so I want candidates to think differently to me, but if you can't explain your reasons for your choices, then we have problems.
On the point of feedback, juniors effectively only really get "isn't quick enough to admit they don't know things" - since that's all I'm looking for in a junior. Seniors, if rejected due to the test submission, I've averaged about 2 pages of A4 (what they did wrong, and a couple of options for what could have been better, with reasons why it's better). It takes me longer to write up the feedback for Seniors than it would do for me to do the test, and I think that's the fair way to be.
We've had some failures:
One used jHipster to generate most of their Senior test submission, including a whole front end (for a CRUD API that we asked for). Way off piste, over delivering trying to impress, and using auto-gen tools. Sure, do that after you've got the job, but before hand, all that does is hide the way you think.
Some simply submitted too late, and another candidate beat them to the seat. I class those as failures because I wasted someones time, when I should have reached out to them to tell them to stop because we'd hired someone.
We've had some great success:
One applied as a junior, but I think they're a unicorn after spending an hour with them on Skype. So we're paying international relocation, a couple of months accommodation, UDemy subscription & they're getting the "Mid" position & appropriate salary.
I was put in the position of having to choose between a wet-behind-the-ears never had a dev job, and someone with 6 months industry experience. Man was that a weekend full of stress for me. I've gone for the more inexperienced guy - partly because he showed more passion on Skype, but also with a slightly lower salary, I can put more into the training budget.
Co-incidentally, as a result of all of this, I've moved from Senior Developer to Architect, and the new starters will be reporting to me.
Great points! And it sounds like you have a very reasoned and fair process for evaluating candidates. That's quite an accomplishment - because few do.
At the risk of oversimplifying what you've written, I think it really comes down to this:
There aren't any "wrong" questions or "wrong" tests if you (the interviewer / hiring manager) have the right (meaning: reasonable) expectations and have a professional, mature way to parse the submissions.
As you pointed out, you can ask candidates to answer questions that they can't reasonably answer - as long as you know that and you're fair about parsing the answers you do receive. There's value in almost any kind of hiring process - but too many times, the hiring managers don't understand the value and expect the wrong things.
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
Oh man, great article!
All the tests have their place, if done appropriately. I'm very much in the "I'm not jumping through hoops" camp. But recently, I found myself on the opposite side of the table, knowing my employer didn't have the best reputation in the market. So I did something about it.
I effectively took complete control over the hiring process, and moved it from CV, pen & paper test (who even does that?!?) and formal in-person interview.
The new process is much more modular, all CVs come to me for filtering personally. At that point, I basically sort no/junior/mid/senior/rockstar. Since we're looking to build a team, "no" and "rockstar" are treated the same - I'm simply not in a position to have an uber brilliant coder that can't communicate.
From there, things get a little different. Basically:
At the more junior end of the scale, I admit, I'm a little evil. I don't expect anyone to understand how to verify that a number is a Perfect Prime, so I ask them to do exactly that, live on the Skype call. But what I'm really looking at there, is how quickly a junior will throw up a flag & ask for help. Fail-fast & you look much better than if you bluff your way through it.
At the more senior end of the scale, things are a bit more relaxed. Submit code that doesn't even compile for all I care. I'm only looking to see how you think, to give me talking points during the Skype call, where I'll be asking why you did things the way you did. I don't want us to be in an echo chamber, so I want candidates to think differently to me, but if you can't explain your reasons for your choices, then we have problems.
On the point of feedback, juniors effectively only really get "isn't quick enough to admit they don't know things" - since that's all I'm looking for in a junior. Seniors, if rejected due to the test submission, I've averaged about 2 pages of A4 (what they did wrong, and a couple of options for what could have been better, with reasons why it's better). It takes me longer to write up the feedback for Seniors than it would do for me to do the test, and I think that's the fair way to be.
We've had some failures:
We've had some great success:
Co-incidentally, as a result of all of this, I've moved from Senior Developer to Architect, and the new starters will be reporting to me.
Great points! And it sounds like you have a very reasoned and fair process for evaluating candidates. That's quite an accomplishment - because few do.
At the risk of oversimplifying what you've written, I think it really comes down to this:
There aren't any "wrong" questions or "wrong" tests if you (the interviewer / hiring manager) have the right (meaning: reasonable) expectations and have a professional, mature way to parse the submissions.
As you pointed out, you can ask candidates to answer questions that they can't reasonably answer - as long as you know that and you're fair about parsing the answers you do receive. There's value in almost any kind of hiring process - but too many times, the hiring managers don't understand the value and expect the wrong things.