Originally published at deepu.tech.
This topic has been discussed and debated for a long time and there have been countless articles written about it and even websites dedicated to it. But still, the problem remains.
I wanted to write about these for a while but was procrastinating on it. Today I decided to finally pick it up after seeing the below tweet
Antonia Forster@antoniarforster
"Whiteboard technical interviews appear to favor men over women... no women successfully solved the problem in the public setting, whereas all women solved it correctly in the private setting".
Interesting findings from NCSU and Microsoft 🤔
theregister.com/AMP/2020/07/15…07:55 AM - 16 Jul 2020
Broken process
So I guess you got an idea of what I'm talking about. Yes, the technical interview process in the IT industry is broken. It is biased, it's based on the old mentality of how a software developer has to be and based on unrealistic expectations. I guess we are all partly to blame as we didn't push back against this enough. Below are the most common practices during a technical interview these days.
Whiteboarding
Solving a problem on a whiteboard including writing pseudocode or actual code on a whiteboard or drawing process/technical diagrams.
Most of the IT companies, including most of the big names out there follow this outdated process when evaluating someone's technical credibility.
Live coding
Coding on the spot while someone watches over you or via a remote session when someone monitors what you do. Most of the time, these assignments are asked to be done in an IDE or environment that you are not familiar with.
Coding challenges
Solving coding challenges on a platform like HackerRank or Codility. This normally requires you to complete some short coding problems within a given time and most of these platforms can track your step-by-step input and mostly will have automated tests that may not be visible to you. These often require you to work in environments that are not familiar to you. Coding challenges are not great as it can also cause anxiety and doesn't have a lot of scopes to ask questions or to understand problems at hand but at least they are not as bad as whiteboarding and live coding, as you still have the ability to do this at your convenience and use external resources to look up.
Take home assignment
You are given a small application or a problem to be solved at your own convenience, often with a max time limit. You might have back and forth with the company and you might have to explain your solution in an interview. While take-home assignments are better than other options in this list, it still is time-intensive and if you are applying to multiple jobs that require this then you will spend a lot of time doing assignments.
Why is Whiteboarding & Live coding bad
While coding challenges and take-home assignments are not particularly great, they are not as bad and biased as whiteboarding and live coding. These require you to work in an unfamiliar environment without resources like an IDE or code editor. And most importantly you don't have the best tools in a developers toolbelt, Google and Stack overflow :)
How many of us can honestly say that we can perform our best without our favorite IDE, tools, and Google. Like it or not, this is the fact, so why should a technical interview be any different.
Give your future employees the same comfort and tools that your current employees have access to when interviewing them as well. The results could be amazing.
Anxiety & pressure
These methods are designed to cause anxiety and pressure. Most people will not be comfortable performing complex tasks in front of strangers who are evaluating your every move. Of course, there will be few who can ace these but what about the remaining? Being able to perform under pressure is not a quality of a great programmer. You are not working as a hacker for an intelligence agency (Remember the popular NSFW scene from the movie Swordfish?).
You are not Hugh Jackman from Swordfish or a hacker working for an intelligence agency
Most of us never work under such pressure and performance anxiety in real life (I'm only talking about the case of someone watching over your shoulder and judging you) you might be under unrelated anxiety and pressure and those are the times we won't be able to do our work properly and we might take sick days and stuff, so what is the point of expecting someone to perform these tasks under pressure for a job.
An engineer is a problem solver not a fact memorizer. This is why I like coding assignments rather then whiteboarding. dev.to/kkemple/should…19:14 PM - 31 Aug 2019
I generally do well in interviews and don't have any issues solving coding challenges and stuff, once during a tech interview for a Developer Advocate position for a well-known company, the interviewer suddenly asked me to fix a factorial code snippet on the spot over zoom call without ever hinting beforehand that this will be part of the process. I have solved factorials a thousand times, hell; I use them as examples for recursion all the time. Despite that, I completely froze and my brain just shut down, I couldn't even read the four lines of code and make any sense of it as my brain was just frozen and I was feeling so much anxiety and I started behaving like a 1st grader, and was blurting nonsense for the next few minutes. The anxiety was so unbearable that I asked them to end the interview. I would have never accepted the interview if I had known that this was part of it, as I don't think putting myself through such pressure is worth the job.
Imposter syndrome
Imposter syndrome is very common in the software industry and people who struggle with it are the worst hit from these outdated hiring practices. Having to do whiteboarding or live coding will only make the situation worse and if you happen to fail in those, then it will make things even worse. You will lose confidence and will start to believe that you are not good enough. Often times this goes hand in hand with the above points about anxiety.
Bias
A recent research from North Carolina State University (NCSU) and Microsoft suggests that whiteboard interviews favor men over women psychologically and are 'anti-women psychological stress examinations'. These interview processes test for stage fright rather than coding competency. Apparently, these processes resemble 'Trier Social Stress Test' a gold standard procedure used by psychologists to induce stress. So the process is not just biased toward women, but also biased towards people susceptible to social or performance anxiety, imposter syndrome, and so on.
Conclusion
In the beginning, I said we are partly to blame. Why? Because in most companies, still doing these interview techniques, it's not something mandated by the company management or some standards, it's mostly people who are in charge of hiring (hiring managers, developers, team leads) who decide the process. So if you are in a position to influence hiring practice and didn't try to change this, then you are part of the problem as well. So what can we do to change this?
- Talk to hiring managers and people in charge of hiring in your company, explain to them issues of whiteboarding and live coding interviews, pressure them to change the process. If you are a team lead, make sure your hiring process doesn't have these.
- Turn down interviews that have whiteboarding or live coding in their process and explain to them why you turned it down. I have done this many times in the past. Of course, this may not be practical if you are desperate for a job. In that case, ask them if they are willing to do something like a coding challenge or take-home assignment instead and explain why you are asking for that. If they still insist, provide honest feedback on how it felt after the final hiring decision is made.
- If you are in some position of power or influence in your company, please voice out against these practices
A company that treats an interviewee well is a company that will treat its employees better.
Resources
- http://they.whiteboarded.me/
- https://www.theregister.com/AMP/2020/07/15/it_hiring_whiteboard/
- https://news.ncsu.edu/2020/07/tech-job-interviews-anxiety/
If you like this article, please leave a like. I would love to hear your thoughts and experiences in the comment.
You can follow me on Twitter and LinkedIn.
Cover image credit: Photo by Scott Graham on Unsplash
Top comments (64)
I wish it didn't matter but I'll preface this comment with the fact I'm a woman, just in case. I don't mind whiteboard, in fact I prefer them over take home assignments. Live coding I've never done so I don't have an opinion.
Of course I don't enjoy stress but HR has to see if you have the skill needed to fit in the team and the mentality. How you handle the stress of a whiteboard (or an oral examination) that's also a good skill to assess. It's not true that you won't ever code with someone looking over your shoulder, pair coding for example has someone literally next to you, watching you code. You need a certain level of stress resilience, especially if you'll have sprints and releases to make. If you have to explain to a senior what you did you need to be able to explain your logic as you go along, or code along. Not having Google or your IDE means that you rely on your logic and reasoning capacity, which is a hard skill, without the crutches of your IDE.
I see it a bit like an IQ versus a math test. You can use memory when it comes to math evaluations, so IQ test try to filter this out when measuring the logical-mathematical type of intelligence.
Combine hard skills with a solid soft skill set, such as being able to explain while you reason, humour if you get stuck, finding workarounds such as writing something in pseudocode and saying you'll google the syntax later, all those are good pointers of how you'll be when faced with hardship and less than optimal conditions.
Personally I've had brain-blanks on interviews where I had no idea what to do anymore. When that happened, after trying to fumble through it at first, I came clear to the recruiter about what was happening and explained the situation, had a discussion about impostor syndrome, expectations etc. It gave me feedback on how to better prepare, even if it was a disagreeable experience, it was a "good" one in the sense that it helped ace my future interviews.
Thanks for the response. I guess you are mistaking general stress with performance anxiety. If you are a good developer you are not gonna feel stressed when pair programming coz you know that you are working with your peer, a fellow human with flaws like you, who might make mistakes like you. Its not the same when an interviewer is looking over your shoulder and you know that your mistakes might cost you the job, so you start getting anxious ending up making even more mistakes. So I'm talking about people who will perform well during peer programming but cannot do anything during a live coding (I fit into that category and dare I say, I'm a pretty good developer according to my peers) so no I don't agree that evaluation of stress is necessary in anyway. Also lets be realistic you won't face situations that induce performance anxiety in your day today work and if you do then I would consider that a toxic work environment and I would run far away from there.
I'm not trying to discredit your experiences. They are of course valid for you but doesn't mean you should expect them to be valid for everyone and the process shouldn't be unfair to others who cannot cope with these practices. A fair process should cater to the lowest denominator.
I think there's a major difference in how we view the issue. The issue is performing poorly under performance anxiety. You seem to say that interviews shouldn't put you in situations where you experience performance anxiety. I say you should find ways to deal with performance anxiety.
Well you are entitled to your opinion but I don't agree. My point is there is no need to put someone through that in an interview as it doesn't measure anything that is required for you to be a developer. Having even extreme performance anxiety doesn't affect your development quality in real world jobs so what is the need to do it. If you just accept the status quo and move on nothing will change and I for one would like to see change and want to attack it at the root
Being able to successfully manage your emotions (whether that's anger, anxiety, feeling down) I think is a very important soft skill - in your job and private life. If you realise you struggle with performance anxiety, it's a good idea to look at why that is and where that comes from. It might liberate you in other areas, or make it clearer for you in which environments you thrive or not.
Of course if you can work on thats fine but doesn't mean others get to incite that. Same for other emotions, you can try to control them but wouldn't you prefer that no one makes you angry or sad especially at work. Also unlike anger, anxiety is not easy to control and often requires medications with powerful side effects or phychotherapy
You suggested I read the other posts, so I am doing.
Fundamentally, this is an absolutely horrendous way for any company to behave, in an interview situation or otherwise.
I don't agree. I have worked with such companies and it works pretty well. We never hired anyone who wasn't qualified for the job. When I say lowest denominator I'm not saying dumping down the process, I'm talking about making it fair for people with less social skills and anxiety as others who have social skills and no anxiety shouldn't have a problem anyway
On that subject, we'll agree to disagree.
Two of my children are on the autistic spectrum, and while it's not everyone's choice, our choice has been to raise them in the way that we expect them to put in the appropriate effort required to be successful in society. Unfortunately for them, that means probably putting in more effort than the average person, just to get a level playing field. I've reminded them a few times that if they use their condition as an excuse for anything, they really won't like the way I handle whatever situation it is, but if they put too much effort in and pander to the whims of others, they'll wish they made an excuse and didn't bother trying at all.
One, for example, simply could not hold a pen/pencil. Should I have let him never learn handwriting, maybe skipping straight to a keyboard? Or should I have put the work in with him, enrolled him in extra (specialist) classes, and bought a 3D printer to custom make a few grips that he could hold to use pens/pencils?
This is the point I was trying to make before in our discussion - in my opinion - there must be some middle ground that's achievable. I don't know what it is, and I don't know how to completely eliminate things like performance anxiety from the interview, I don't know if I'll ever know. But that won't stop me trying to figure it out.
Lets say in some hypothetical situation, a candidate (or their recruiter) tells me that they have severe anxiety issues. I think (and like to hope) that my response there would be "ok, tell me what you need, to minimise that issue." Somewhere in the ensuing discussion, there would be a point that either they'd agree to the flexibility that I was willing to put in, or I'd thank them for their time & wish them luck. As a starting point, I'd probably lay out the kind of things we'd be talking about, and tell them that I would be putting code up on the screen and they'd have to talk me through a couple of minor bug fixes etc. I'd probably advise that they spoke to a medical professional (eg., their regular doctor), and suggest that if they wanted, they'd be welcome to bring someone into the Skype call to help relax their nerves etc.
Finally, as some of the other commentors have mentioned, I think probably the article could be worded better. Yes, some interviews are horrendous (a friend is about to do a paired programming exercise with someone for stage 3 of 7 at a company). Sometimes take home tests demand too much time etc. But much like mental health issues are incredibly specific to the person, the interview process is incredibly specific to the hiring manager or company, and they exist on a spectrum.
Well its ok that we don't agree on everything. I'm glad we agree on somethings and I'm even more glad the post generated this conversations.
Based on my "personality test" - you sound like someone I'd hire.
But I'm curious, why do you prefer whiteboard to take home? Maybe I just do the take home test differently to everyone else... but maybe I don't see the problem with it.
I'll preface (or is this post now?). In my take home test, the requirements are deliberately vague, and the only way to do it wrong, is to not turn it in. When talking to candidates about it, I'm careful not to critique it, I'm more interested in WHY choices were made and what else is in your head, rather than WHAT you did.
Finally, I get the gender thing, but please don't feel like you have to. I'll admit my bias - I'm more likely to hire women than men, for the same reason I'm more likely to hire a Russian than a Brit, or (shock horror) a Black guy over a White guy. All else being equal, I'll swing in the direction of what we don't have in the office to try to minimise the echo chamber effect.
I often find instructions confusing - knowing what is expected of me is important (I might have ASD though so that could be it). With a whiteboard you can easily turn around and ask, and if they are stiff-lipped I just say something "oh right so this is about how I make my decisions, gotcha", then go on - at least I've been able to announce what it is I'll be doing and why I might cut corners here and there. It's also often much shorter. A whiteboard is a short burst of energy which is closer to how I function.
The take-homes I made were often several hour long assignments, and timed (countdown the moment you open it), so I couldn't decide how I worked on it or when, I wasn't sure what kind of solution they were looking for. If it's too short of a question it's basically leetcode / algorithm grinding and you could have done that during the interview irl, if it's a big codebase that you need to dig through, it can take a lot of time just understanding what they want.
Must say I'm not sure about the last part of your comment or whether I'll ruin my hypothetical job chances with you, but I'd like to get in because I was the best, not because I was a woman. I know it often says "all else being equal" but no two candidates are ever carbon copies of one another. At some point gender or race becomes a notch in the plus or minus column, along with their other points. I wish that didn't happen.
While I agree with many of your points, as I've said on now dozens of these posts, there's one problem you're overlooking: we have to be able to verify your skills!
Live coding is the only way we can get to know how you approach coding! It's not just about getting code running; it all comes down to how.
We owe it to all our employees, and to you if we hire you, to actually make sure all hirees can code. Believe it or not, there are applicants who are actually just clever disguised outsource brokers, and others who cannot code but apply for jobs whose minimum requirements are many miles beyond their abilities.
Rather, a good interview involves live coding something practical, and the interviewer should know how to account for nerves.
Personally, I like pairing the take-home and live coding together, for a few reasons.
I actually appreciate take-homes, even when I'm doing a lot of applying myself, because it gives me a taste of what that job's coding is like. I can self-eliminate myself in some cases; in the least, I learn something I can use later.
If the live coding builds on the take-home, it's less "surprising". The applicant knows the code; they wrote it. (And if the applicant outsourced the code, it becomes immediately apparent.)
A real job will have naturally occurring high-pressure situations, not by conspiracy of an evil boss, but because stuff happens. Prod goes down. The product is late. A bug has shipped. Whatever the situation, it's helpful to see if someone can still think critically if they're nervous. No, their performance won't be at its peak, but a good interviewer can account for that.
In eight years, I've learned that if someone's bravado is so strong, they can march into a final interview and go through the whole thing without any nerves, they've got an ego the size of Ontario, and will not be an asset to the team. Nerves tell me there's a real human there. (Seriously, I second-guessed this for years, only to discover it was entirely accurate.)
By the way, in the near decade I've been doing this, very few female applicants (maybe one in eight years) or minority applicants (less than a quarter, same as for white males) actually struggle with the live coding. There are plenty of candidates, in roughly equal proportions for the different demographics, who self-eliminated from the process, because the take-home made it clear to them they weren't ready for this yet, so it saved us both time.
But, all that aside, yes, nuke the whiteboarding and fizz-buzz. Take-homes and live coding are good, I'd even argue, utterly indispensable for hiring well, but only if they're based on an original, practical, job-related premise.
Unfortunately, if you were to tell developers to just walk out on any company that uses live coding, you can almost guarantee that the job they'll get will be with an unvetted team, which they will almost certainly wind up having to carry.
No you don't. You need to be able to measure the interest somebody has in the job. You have to measure how your candidate can adapt. The knowledge of today will be different from the one tomorrow, so what the point to measure something which will change?
Another question: how do the candidate evaluate the company? 'cause you know, it doesn't only go in one sense.
Even if the candidate is a liar, as you say, you can still ask him some questions he won't be able to answer. Example: what's an interface, or whatever other construct in the language of your choice.
Even better: there is the trial period. Or a trial day. The candidate can see how the company work and see if he or she likes it; you can see how he or she adapts to your real work. Win win. You have to pay him or her, of course.
The go home stuff: I'm sick of them. Spending hours, not payed, on tests? Not my thing, thanks. Especially if, on the other hand, people who interview the candidate learn some stuff from his or her test. It's called working for free.
I'm not even going to take the time to respond to your points. While I will not make any assumptions or comments about you as a person or your intelligence, as to the ideas outlined here, there is nothing even approaching a rational proposal in your entire post. I would strongly encourage you to get some practical experience hiring; after a few years of seeing things from the other side of the table, you'll understand why your proposals are altogether irrational from any stance: practical, interpersonal, or logistical.
I have some practical experience hiring, and it works quite well.
Well, I'm glad you've apparently been lucky to only ever get qualified, honest candidates and work for companies with boundless time and money, who can take on the significant costs and legal risks of "test hiring" all these candidates. Count your blessings; very few (if any) people in hiring get to work under such perfect circumstances.
Meanwhile, I've never gotten a complaint from anyone, including from the diverse set of employees who later got to participate in and give feedback ON the hiring process, nor from anyone in the career services departments of the universities I work with. Seeing as the way I'm doing things was developed over years, with exponentially more care, compassion, and attention to the interpersonal components than you imagine (read "accuse"), based on what worked and what profoundly did not, I'll stick to what actually works.
For everyone else reading: I urge everyone NOT to take Matthieu's advice about what is acceptable in hiring into account when job hunting. From someone who fights long and hard for compassion, communication, and diversity in tech, I assure you, the hypothesis he forth is based in utopic ideals, not the daily realities of business. Burning a bridge at one company can close many good doors elsewhere. Don't burn opportunities just because you don't want to put in a couple hours of unpaid effort into the process (the company is already spending a lot of time and money just to consider you), or because you're paranoid they'll "steal" your take-home work (baseless: we already have a solution long since written for it, and you're probably not Donald Knuth). It's reasonable to set boundaries for yourself, but the hard reality is skills must be verified somehow, out of respect for you AND the rest of the team!
Job searching is hard on both interviewers and interviewees. Hiring managers are not automatically malicious bad-actors. We don't know you from Adam. Expect to prove yourself, to the same degree you expect us to prove ourselves. That's how this works in any skills-based industry, and tech is no different.
For somebody who speaks about compassion, your post seems a bit... aggressive. I'm sorry if you misinterpreted my opinion, I was just saying what I was thinking.
I'm an idealist for sure, but, as John Lennon was saying, I'm not the only one..
Compassion doesn't mean that I'm not going to be passionate.
As someone who has been in charge of hiring for the same company for many years, I've seen some excellent candidates...as well as some absolutely horrific displays of behavior from some candidates. Both ends of the spectrum are only consistent with what other hiring managers I'm acquainted with have seen, I later learned.
I've also watched a pervasive attitude of sheer entitlement creep into the job-hunting process, whereby applicants are beginning to feel that they should never have to prove anything, should be paid for their time (not accounting for the money and time the employer already spends on the process), and should be somehow given a sort of "consolation prize" if they're not hired. And after venting that attitude freely, they can't figure out why no one, not even the gentlest interviewers, will give them a job.
So yes, I do get passionate about this topic. Job searching a difficult process on both parties. Typically, when people start representing employers who use, say, live coding and take-homes as somehow malicious bad actors, that's because they are failing to account for the reasons those things were introduced into the process in the first place.
You can say what you think to a point, but you absolutely must think about how a young developer is going to take it, and what the worst possible outcome is. Just venting your opinion in a public place where it will mislead someone with less experience than you is socially irresponsible at best.
To put it frankly, no one has ever died (or, I'd wager, even suffered long term harm) from live coding; I'd even defy anyone to provide a single provable instance of a reputable employer "stealing" the "free work" from a take-home assignment.
In the end, it's because of my compassion that I get angry about this. I'm thinking of all the people who may read your post and be misguided into a treacherous and career-stalling path because "that one guy says employers shouldn't be vetting my skills, so I guess I'll just walk if they do!"
P.S. I've learned that 98% of situations where an opinion was "misunderstood" or "misinterpreted", the correct word is actually "miscommunicated". This is precisely why I seldom use the former two words when I feel like my point isn't understood; I instead take responsibility to clarify. It's not my job to read your mind, it's your job to be clear.
You look very sure about yourself, and you conclude many things I've never said.
I'm not as sure to be right as you are, that's why I experimented a few things in the hiring process, and I will continue to do so if I have the occasion. Because if I stay convinced in a set of ideas and I'm ready to fight for it blindly, I will never experiment, learn and bring new knowledge to improve the situation.
If you think I'm dangerous, I'm sorry you feel that way.
I have experimented, too. As I've said before, the process I follow is based on years of trial-and-error, with a strong emphasis on compassion and communication. I've experimented extensively, and only adopted live coding and take-home assignments after discovering the necessity for them. I'm only more confident in my stance by nature of those years of experience, experimentation, and learning.
I'm sorry if you actually believe I meant "you are dangerous." I absolutely believe that how you stated your opinions is dangerous, but I can't find where I said you were. Just in case that separation between person and idea wasn't clear, I am only debating the validity and wisdom of your idea.
I'm a hiring manager.
Yes, I need to know candidates skills - to the level of "do you know the core language?"
I don't care for most frameworks, they're transient, but if you can't spot a compile error glowing bright red at you in an IDE, please don't waste my time.
I agree with you that I have to measure how a candidate can adapt, but as the author of the article talks about performance anxiety - how do we measure adaptability without risking performance anxiety?
Trial periods, even a day, are terrible, in my opinion. I don't care about paying the candidate for it, I care about the costs to the rest of the team, starting onboarding for someone who simply may not be there tomorrow because they don't like it, or they had a better offer etc.
Your point on take home tests - I use one (for senior candidates). It's very clearly spelled out that we won't be using it for anything other than the recruitment process, and that a passable solution can be done in an hour, but feel free to take a week if you like.
Every submission of it that I've had, I've not even run a build against it - I literally don't care if it fulfills the requirements given, it's just too give talking points in the interview.
I would argue that you can see if a candidate "can spot a compile error" when the IDE turns to red by only asking question. I'm always mesmerized of the crazy answers I have to "what's an interface? What's the purpose of it"? It does already A LOT of sorting.
For the trial days, it depends on your on boarding. I know what you mean though; once, we invited a candidate for a trial day. Only running our project on his laptop took 4 hours. Not because trial days are terrible, but because our documentation was bad and our set up too complicated. We fixed that, and it was not a problem anymore. The good side of the experience: it showed us a big problem we had.
This is the only kind of on boarding you need, I think. You need to explain quickly the goal of the application as well, and then let he candidate implementing little stuff or fix small bugs. Some open source projects are very good at that, to have more contributors. I take the inspiration from them.
For the home tests, you can't guarantee that it will only be used in the frame of the recruitment. If you read it and learn something from it, your candidate already gave his knowledge against nothing. We are knowledge worker, our worth is (partly) our knowledge.
That being said, your process seems focusing on what's important, which bring me to the eternal conclusion that the tools themselves are not bad, but the way you use them can be bad. I saw too many recruiters / hiring managers being so picky on the way to do stuff (if it's not their way, it's the wrong way), or on the knowledge somebody should have on a precise framework, or not judging enough the soft skills, that I prefer not spending time on tests and such. Especially since, in my experience, they precede any form of discussion.
Another thing: the problem to me as well is that the interview process is often going in one way, judging the candidate. The candidate should have an idea about the company as well, the culture, how people like it. It's very difficult to have most of the time and it's essential, because if your candidate is not motivated to work in your company, everybody will regret it.
Tests don't really show you how the candidate will be in your team either, and it's very important too. If you look at the research done on the subject, it's even more important than the skills of a single developer. So why focusing on that?
These are just thoughts. It's an interesting subject, which is unfortunately too often "space vs tab" kind of discussion. I think your process is quite good, but I think as well there is always room to improvement.
Oh don't get me wrong, we have lots of room for improvement - and to a large degree, that's why I can't even think about trial periods at the moment. Though, we had 2 juniors start recently that were pointed at a repository and nothing more, and they had 3 of our 27 components running within a day, and a day later were confident enough to start estimating tickets. One of those juniors is fresh out of university, never worked as a developer.
Ultimately, I only really care for juniors "do you know the language, and how quickly will you tell me that you're stuck on a problem?" The latter being a very fine line, obviously. For seniors, all I care about is "can you learn quickly, and can you communicate well both up & down the ladder."
I get your point on giving free knowledge away, but really, we're talking about things that you could search online & find out. Case in point here, I spoke to a Senior candidate yesterday, and during his code-review stage, I mentioned he had a lot of getters & setters, so I asked if he'd used Lombok before (we are a Java "shop"), with a view to branching the discussion towards AOP vs readability. He made a quick note on some paper he had at the side, and admitted he hasn't. There were a number of times this happened, with various technologies - and he scored better with me because not only did he admitted his gaps quickly, he made a note and genuinely seemed interested to check it out regardless of the outcome of the interview. I don't know yet if I'll hire him or not, but if not, he's learnt some things for free and I really don't mind sharing them.
I know, which is why the tests that I give (other than "spot the missing semi-colon" stuff for juniors - which is done in an IDE and the IDE is screaming at you to tell you where it should be) - are geared towards discussion points. I'd rather try to work out how a candidate thinks rather than what they currently know/think.
I couldn't agree more - I've seen my fair share of poor recruitment processes. Ours used to be horrendous, and I had to answer maybe 20 or so Java "riddles" using pen & paper for my interview here! So when I moved into a more managerial position, I immediately moved the process to one that I'd like to go through as a candidate. Juniors get one Skype call and a decision, Seniors get a take-home test, one Skype call, and a decision.
I highly doubt I have it right for every candidate, but I'll tweak as I go, and my whole point for reading all the comments on this article, was to see if others had better ideas that I could "steal" (and in some cases, there have been, and I have).
Interesting. I like your approach. Thanks for that!
Thanks for the response. No offense but I call bullshit on some of these (for the lack of a better expression I could think of)
There are multiple different ways to verify someone's skills. Some I could think of right now
I agree there might be some people who can't code but they will be very easy to catch once you have a discussion with them. You don't have to put everyone through unnecessary stress or some bad actors
I agree with the take-home stuff but not with live coding. Even if live coding is built on top of take-home it still induces performance anxiety and you can't expect every interviewer to understand and evaluate accordingly. So this is not realistic or fair IMO
I'm talking about performance anxiety and imposter syndrome, the real-life high-pressure situations are quite different. any decent developer will be able to handle that. I have experienced such situations many times in my career (I once accidentally deleted a production DB on the night of a critical cutover for a huge airline, so imagine the pressure to restore and go live before morning) and was able to do my job quite well but I still suffer from performance anxiety. I also know a lot of developers with performance anxiety who have never broken down during a high-pressure work situation. So high-pressure != performance anxiety and I really don't believe you need to evaluate for that unless you are doing rocket launch and stuff like that maybe
I get your POV though, you are saying that even when you do live coding and such you are aware of the issue and you take them into consideration, but my experience is that not everyone does that. But still why put someone through that pressure. It's quite emotionally and psychologically nerve-wracking. The person applying for the job might already be having many issues (Jobless, desperate, family issues, money issues, and so on) why add more stress to that?
It's possible that your didn't encounter people who struggled or many of them did struggle but got through it without showing it (which also happens a lot) especially if the problem given was easier and the interviewer was very friendly but that doesn't mean the problem doesn't exist. Look at the research paper I linked chrisparnin.me/pdf/stress_FSE_20.pdf
I disagree with your last point about unvetted team, you are assuming live coding is the only way to evaluate. I have had 3 jobs in a decade and all three didn't involve any live coding and these were amazing teams I worked with and you can find 1000s of similar testimony. I have interviewed for some amazing companies with no live coding involved in the process, one of them was my dream team but unfortunately, I didn't make it
Again thank you for sharing your POV
Well, we will definitely never agree on this point. You didn't "call B.S." on anything validly, but I understand your views are based on your personal experience.
Just please recognize that, as you (from what you've said) have little to no experience managing or hiring for a team, your ideas are based largely on conjecture and largely unevaluated theories. I'm speaking out of having done this for many years.
Just please be aware of where your expertise stops; it is all too easy to misguide young developers into making career-devastating decisions.
Yes lets agree to disagree. And I'm not expecting everyone to agree anyway. Btw I never said that I haven't done hiring or managed teams. I have done both. I was part of the company wide hiring team in 2 out of 3 companies I worked for and in a decade of being in IT I have been a project manager and team lead on multiple occasions so I'm not talking without experience here.
And I believe I'm not misguiding young developers here, I'm specifically asking people who have a say to speak up and others to honestly tell the interviews how they feel. We need to be more realistic and straightforward in this Industry. You are entitled to your opinion the same way I'm entitled to mine so lets stop accusing each other of anything. Cheers
You know, I'm stealing some of your approach - I actually did it be accident (before reading your comment).
I have a take home test, designed so that there is no wrong answer to it. The only way to fail it is to write a horrible mess that makes my brain hurt - but you'd never know that, because failing it doesn't necessarily mean a rejection.
The part I'm stealing, is adding requirements to it as live coding on Skype.
As an offer of exchange - I use whiteboard during the interview. But it's not really a test. It's a "draw me the architecture of the current system that you're employer has right now" followed by "ok, talk me through a problem you've had to solve with it."
I figure, if they can't do that, they've not been paying attention in their current role.
Nice, 😄, But what is correct process to evaluate Software Engineers then ? Also what do say about Algorithms and Data Structure Questions in a interview, is it really required ?
IMO the focus should be on evaluating ones logical skills and reasoning along with ability to learn. So short coding assignments (which doesn't take more than few hours) would be fine along with a discussion with the candidate where you just talk about technology architecture and so on. Algorithms and data structure can be part of the discussion if those are in scope of the work
Totally agree on that. I would try to determine as well some soft skills, like the ability to speak about ideas he doesn't agree on.
As a second step, I like trial day / morning trial day (payed, please), where the candidate:
The company can see:
Tried it in 3 different companies, it works very well.
Another thing: if the candidate as enough code online, skipping any coding challenge should be possible
Yes, the onsite day works well as long as it doesn't contain supervised coding. I enjoyed such sessions when the focus was on discussions.
Totally agree. How the team works together (and how the members communicate) is really important, maybe even more than the technical skills. I believe adaptability and willing to learn is important too.
It depends of the company of course, but I mean if you're maintaining a CRUD application, I don't see the point to have highly skilled engineers capable of writing a Fibonacci recursion in 12 languages.
I'm interested in this too.
I'm in a "position of power" - our old process was using pen & paper to answer pre-set riddles. So I changed it to a take-home code sample (use your own setup, fulfill these vague 7 requirements to write a CRUD application with 3 data entities, acceptable answers can be done in an hour, but take a week if you like).
That's just for seniors, juniors get "tell me how to fix these two bugs, and optimise this one algorithm." The bugs are just missing colons, and the algorithm is designed so that you CAN'T get it right - because for a junior, I only care "do you know the language, and how quickly do you ask for help?"
So, how can my hiring process be improved? Genuine answers will be IMMEDIATELY implemented (and I'll give the next few days of candidates as much notice as I can).
@deepu105 , any thoughts?
I like the first part (for senior as you said) maybe you can add a step where you discuss the solution with the candidate to understand the approach s/he took if you are not doing already.
For the second part, it depends on how you do it. If you are doing this on the spot or if you are asking them to do this in front of you or someone else, I still think it will just create performance anxiety, especially for juniors. You wouldn't supervise someone working for you when they are solving a problem right. But if you are sending the question and ask them to work on it unsupervised I guess it would be fine.
For the seniors, the whole point of the test is to talk it through with them afterwards, even if based on their submission, I'm rejecting them. They took the time to write it, so I take the time to talk to them about it.
Mostly, I'm querying them about why they made decisions, what else they thought of etc. If I know it's a rejection, I'll also tell them how, in my opinion, it could have been better.
For juniors, I'm mostly testing their communication skills. The code samples are on my screen and they don't have control of the mouse or keyboard. They have to talk me through their thoughts, and discuss the code (and solution) with me. Kind of like a code review/merge request.
Absolutely anything they want to change, we can change. I've talked to people who didn't like intelliJ, so I opened Eclipse. I've talked to people that didn't like the variable names, so I asked them for better names & refactored on the spot. I've talked to people who didn't like my dark theme, so I switched it.
I recently hired 2 juniors, using this process. The deciding factor over & above their communication, was that they asked if we could change the requirements of Test 2 (the impossible answer test). So on the spot, I changed the requirements for them. Everything about the junior test can be changed on the fly, including how much help I give, all they have to do, is ask.
Less than a week after employment, both the new guys are fixing bugs in our production code, and I very nearly rejected one of the bugs based on infrequency Vs effort to fix. But the junior I hired wanted to look at it from a different angle... and found an easy fix.
Maybe you can ask them first if they are comfortable to do the test that way and give an option to either do it together with you or look at the sample alone first and then talk together with you it will ensure you are not scaring away a potential candidate who might just be afraid of being wrong
If they are afraid of being wrong, to the point that they can't ask questions, I don't think I'd be hiring them.
There comes a point, in all jobs, when you have to do as the boss asks, just because the boss asks.
In our office, there are times of pressure, from lots of different sources, there are times when we get asked to do the impossible, or to shortcut process to get a win for the business. I would be doing candidates a disservice if I couldn't work out in an interview/application how they might handle those times.
If, under pressure, a junior developer can at least say "it's not my role to sign off on that, who approved it? Or better "should we do it that way? Isn't there some compromise that works for everyone?" - they're more likely to get hired.
If they sit at their keyboard being afraid to say what's on their mind, the whole team suffers in many ways.
Like many others you are confusing work pressure with performance anxiety. Please read the other comments on the blog. I'm honestly tired of answering the same thing again.
I'm not confusing the terms at all. I'm trying to find a way that works for everyone in the process, genuinely.
I need to, at least hazard a guess, at how a candidate (at any level) might cope with work pressure. How am I to do that, without risking some performance anxiety?
Would you suggest, instead, that I focus on removing work pressure? (We already have very little of it)
Further, if they have performance anxiety in an interview, are they not likely to have the same (and suffer imposter syndrome etc) while working? Unfortunately, even with juniors, I don't have the budget to help improve their confidence over the next (as an example) 6-12 months.
I completely understand that some people suffer panic attacks etc, and I wish them well with their health & their career - but I'd much rather know about that up front, and take it into account. Who knows, for the PERFECT candidate in every other way, I could probably spend the time to shield them from sources of anxiety in the office. I've just never talked to the PERFECT candidate.
As another point of topic, I worked with a senior once who didn't realise what he was feeling was imposter syndrome. He could do the job perfectly well, but group meetings made him anxious. So we changed things up for him, and made more of a point of showing him his successes. The point being, he communicated, and essentially asked questions. Apparently, him talking to me, putting a label on it, and him researching about it helped him in his personal life, and he's spotted the same in other people.
If people cannot ask questions, for whatever reason, then we're going to have problems.
Asking questions have nothing to do with live coding or whiteboarding. I don't think people suffering with anxiety or imposter syndrome have problems asking questions. They will ask questions if they have any. Problem is unable to code or solve a problem (which they will be able to do perfectly fine in a normal work situation, even with time based pressure or some other work pressure) when they know that there is a person watching their every move and evaluating it (doesn't matter how nice or comforting you sound, its hard to control how a brain works) from what you said it seems like you care about people you hire, so just switch to take home assignments or worst case, something like codility, then have a review session to talk through what they did. It will give you more insight into the candidate and they will feel much more at ease and perform much better.
Specifically whiteboarding, I probably don't do the way that most people do. I use a whiteboard for the very purpose of them drawing something they know (or should know, if they've been paying attention in their previous/last role), in order to generate the questions I ask.
I think that probably differentiates on that key principle, other than the candidates history, I don't come into an interview with a pre-set list of questions. It's much more akin to a conversation, and if it's not a two way conversation, with the candidate asking me (sometimes very tricky) questions, they probably won't get the job.
That's probably why I (probably incorrectly) jumped from "performance anxiety" to "debilitating anxiety" - while they have commonalities, it's most definitely not the same thing.
Back to whiteboards, the best one I had, was an interviewer asked me to "draw the testing triangle" - nothing more, just wanted to know if I knew it. It's the kind of thing that as a developer, we never really bother thinking about, but should definitely be aware of.
In any situation, interview, daily work, shopping - if someone can't tell me there's a problem, there's nothing I can do. However, tell me there's a problem, and we'll move mountains (hopefully together) to put it right.
Seniors get a take home assignment, juniors don't. For a little better context with what I do with juniors, check my post (and please feel free to give feedback, knowing the context better): dev.to/190245/we-re-hiring-new-jun...
Off-topic (for this conversation) - I didn't see your job title until today, when I decided to read the thread fully on my laptop, instead of my phone. It was actually a junior developer (in the job, not in an ineterview) that introduced me to jHipster, and while it doesn't quite work well for us, we do make use of it where sensible. So kudos on working on something that other dev teams use (our work is all internal only, very specific domain knowledge etc).
That's a bit flawed logic, isn't it?
I'm a hired worker, I've been coding professionally for a couple of years. I've never had anyone sitting behind my shoulder watching me coding at work (unless we agreed on pair programming).
Based on that it seems that I am able to provide certain value to my employer without having to let people sit next to me and watch me write code.
So why the ability to code with someone sitting behind my shoulder should determine whether I am suited to be a hired worker?
For now, I write mostly on my blog about foundations and soft skills, but sometimes I just think going full blown into this problem.
I'm coding professionally for 10 years, and for 10 years I had to endure these interviews. People were globally happy to work with me. Nobody really complained about my skills. Some companies would love me to come back.
I'm not the best, but I like what I do. It pushes me to learn more, to try to be better, to try to help my peers. But I'm so useless in interviews. I'm so tired to code the usual fizzbuzz or to balance binary tree. To me, it only indicates that the team I will work with is composed of people who were hired based on random tests, and that the company has no idea how a developer can bring value.
The last company I applied for, they asked me to pass a test for 4 to 5 hours. Not payed, of course. Another one: coding test of one hour on Hackerrank, about algorithms they would never use. I have hundred of examples like that.
This was my rant. I wrote an article more or less related some time ago, if you're curious.
That was a great article. thank you for sharing
I'll check it out. thanks
No, I cannot when someone is looking over my shoulder on a whiteboard. And I'm not a junior developer, look up my profile. This has nothing to do with seniority its about performance anxiety. I can however can write even complex code a notepad when I'm at my own comfort. Also release deadline is not same as having to do something in a live coding session.
I've been a hiring manager as well as a developer forced to do these whiteboard sessions which I like to refer to as coding performance art, I find them generally useless and absolutely stress inducing. I think they are perhaps useful if your hiring some one what will be presenting technical designs to an audience but really they don't have much place in an engineer hiring decision. I have seen good developers struggle with this practice and hired them anyway and their performance has proven that this method is invalid.
Great article, I Think you are right that as a group developers need to push back on this sort of practice and pass on the interviews or request another form of evaluation. I for one no longer accept this sort of interview either and I have a number of peers that decline them as well. It's up to us to make the change!
Great to hear that. This is the kind of change I'm talking about. Btw even to present technical design and stuff to audience you don't really need this evaluation. I have been in pre sales presenting tech design and solutions to clients for many years and I'm also a public speaker who have done around two dozen conference talks many of which included live coding. But I'll still tank a whiteboarding session.
Well, where I live its still an employee market. Also, that doesn't matter even if its an employers market we can still try for change.
you are oversimplifying and overgeneralizing the issue IMO. First of all I'm not talking about the outcome of these interviews, I'm talking about putting someone through unwanted anxiety. Not all hiring managers are like that and even if they are it gives them no right to put someone through unwanted pressure. The person applying for the job might already be having many issues (Jobless, desperate, family issues, money issues, and so on) why add more stress to that?
some times you need a time machine to meet the 12 years K8S experience. Or you need to have the 100% skills for the unicorn role.
I have been offended by the offer of a contract role instead of permanent because they were not too sure about the role. They arent hearing from me anymore and i will be sharing my thoughts on Glassdoor!
I wasn't over generalizing by saying everyone goes to through Anxiety and pressure. I clearly said it only affects some people (and I would assume those are at least half of all devs from what I have seen and heard) but changing the process can't be done conditionally so yes for that I was generalizing. And your comments about if I know how the job works and about hurting feelings are bit ridiculous and I'm not gonna respond to that. If you want to know you can look up my profile and judge if I know how "jobs" work
I agree with some of those points the interview hiring process is quite broken and convoluted to the point of driving a sane person crazy. It needs an almost complete rework because let's face it there are dozens of highly skilled unemployed developers out there who are more than capable of working. And I find it ridiculous that many companies have job descriptions listing many skills that are not even required on the job.
All it does is scare people away from applying for the role leading to imposter syndrome when you think you are not good enough when really they are looking for just 1 or 2 skills. And then these companies complain that they can't find the right candidate or that hardly anyone applies for their roles.
So everybody should work long and hard to be able to succeed in interviews, like you did, because... ?
If people look over your shoulders and judge you constantly everyday at work, like managers do during interviews, asking you questions every 5 minutes "to see how you think" and breaking your flow, on tests which has nothing to do with the real job, you should definitely find another company.
It's not how it works. People support each others at work. They learn from each others. They build something real and (hopefully) useful. Interviews are completely different.
@deepu105 : Nice article. I could relate to this very well unfortunately. Sometimes like 4 or 5 interviews or even more each of which is around 1 and a half hours long and all of them live coding of some sort. I am thinking part of the reason for this kinda of interview process in the industry is that it gives a process; a deterministic mechanism whereby given 200+ applicants you could choose the one (or maybe few) and probably works well and quickly compared to other methods (such as take home).
Also I feel I am spending more time on interview preparation (reading books like "Cracking the Coding Interview") rather than actual coding which is sad. Imagine if it's 5 interviews with the same company I do this preparation 5 times; it's anxiety before interview; at the interview and also after the interview if you know you are progressing to next round. Although having gone though this over and over again I feel I am getting used to them. However I also feel I have gotten better at "interviewing" rather than "coding" which is sad 😄
I don't hate this kinda interviews but I wish it would change to something better in the future but for now it is what it is I guess. 😄
Yes exactly developers are being evaluated for anything other than development skills
Nice. Exactly when someone is staring at my screen I cannot write code. Another problem with Coding Platform problem statements is that most of them are pseudo-problem statements. Home assignments are good as we can finish those without any pressure.
From interviewer's perspective, I enjoy doing live coding. It gives me an opportunity to not only see the candidate's coding skills, but also to understand their thought process, and to see their communication skills - can they explain me how they plan to solve the problem? If they get stuck, can they explain me what is the issue?
In order to make this process as comfortable for candidates as possible, I asked them to either bring their own laptop, or tell me they preferred language and IDE so that I could prepare it for them; we used a laptop + large TV, so that I didn't have to sit behind the candidate, they were looking on the laptop and I could see the code on the 2nd screen; candidates could use any online tools available when writing code.
It worked for some candidates, and it worked quite well. I must admit though - there were cases where people I knew people failed because they were stressed. I had one candidate literally look at me scared and say "oh, you're going to sit here and watch me coding?!". So I agree with you that this is not a great way to test candidates. Being good at live coding is not the same as being good at software development.
Eventually I think the take-home exercise (+ a follow-up discussion during live interview as next step) was a good balance between giving candidates environment that allows them to show their best skills, and my need to evaluate their skills and to determine whether we should be working together. You're right that these exercises might take some time, but it's an investment on both sides - my team prepares the exercises, we review the code, talk to the candidate, finally if we reject the candidate we provide written feedback on what went wrong. I think it's a fair deal - both sides invest some time (and I always check with candidates how much time they took to complete the task, so that I can limit the scope for the next candidates if I underestimated the required effort).
Great to hear that. I hope you switch to sort take-home assignments for future candidates.