I find it distressing that more and more companies using personality tests in their interview process. My colleagues mentioned that they have seen this trend increase as well. They note that it is a convenient way to get around anti-discrimination laws because the results are hidden. I’m going to give a benefit of a doubt and not assume this practice is due to some sinister motive. I think a much simpler explanation is that hiring managers are trying to make their jobs easier.
Photo by Glenn Carstens-Peters
A personality test is easy to conduct. You get a candidate to answer some standardized questions. The answers are compiled into a set of scores for various traits. Anyone can pick the traits they think they want and pick a candidate that fits the mold. It is a lot more difficult to make this determination in the actual interview process so there is a huge temptation to simply rely on a test that has some authority behind it.
There is a much better way to determine a candidate's qualifications however. Let's look at some of the questions we would want a personality test to answer for us:
- Will a candidate get along with people?
- How does a candidate communicate?
- How does a candidate handle disagreements?
- How does a candidate handle new situations?
- Does a candidate learn from their mistakes?
All of these questions can be answered better with an open ended software design or architecture problem. I prefer to base these questions on a system the candidate will actually work with. Some examples are: running a data migration from legacy systems, implementing a new feature across the stack (frontend components, api endpoints, datastore used and/or database schema, etc.), or managing a pipeline of asynchronous jobs that had dependencies on each other.
The beauty of these types of questions is that they will inevitably result in a candidate having questions. The types of questions they ask will be an indicator for their technical ability. Plus, these questions will result in discussions between the interviewer and the candidate. That discussion will resemble the types of discussions development teams will have when building these systems for real. While the candidate's behavior is modified because they are in an interview situation, this is as realistic a scenario we can get with what it will be like to actually work with this person. This answers how a candidate gets along with the specific people on our team and how well we can communicate with them. A standardized personality test can provide some insights on abstract traits, but not how well a candidate will mesh with the team members we have.
In addition, we can add things to our open ended problem to answer other questions about a candidate. Since an open ended problem is rarely a problem that has an answer which is 100% correct, we are free to disagree with a candidate regardless of their answer. The interviewer can propose a different approach to solving the problem and ask for the candidate's opinion. Will they answer defensively or with arrogance? Or will they provide a neutral analysis of the trade offs between the two approaches? I don't know a single development team where everyone agreed on everything 100% of the time. That makes this another realistic scenario we are evaluating a candidate with and seeing how they interact with our team.
And speaking of realistic scenarios, we can also add requirement changes to our open ended problem. How does a candidate respond here? Do they freeze? Do they focus on solving the new problem? Are they open about some of the regrets they have about their original design? Can they find a way to deliver on the new requirements with their existing design or do they think they need to rebuild everything? How a candidate handles this situation says more about their personality than any standardized test of abstract questions.
I've learned the hard way that abstract theoretical questions aren't the best judge of engineering candidates. It's why I never use standard algorithm questions in interviews anymore. Personality tests may test for something different, but they share the same fundamental problem with coding on a whiteboard. Neither are realistic representations of a candidate's true ability. There are better alternatives. The open ended architecture/design problem is one, but there are others. And as with everything else, we should always strive to improve our interview techniques with even better alternatives.
Oldest comments (10)
I've experienced the personality test phenomenon myself, but it was very clearly an IQ test. They just can't call it an IQ test, because that's illegal, at least in the United States.
That's awful. I can only hope good candidates refuse to work for that company.
I keep seeing these posts complaining about the interview process and I'm surprised I haven't yet seen one suggestion for an obvious approach to determining a candidate's technical AND inter-personal skills: i.e. code review.
Why not present a candidate with a PR with some code/style errors etc. and ask them to do a code review. You'll establish their ability to spot issues in code and you'll get an impression of how they communicate with their peers. You'll also get some sense of their professional experience: e.g. if they clearly understand the code-review process; if they can spot things that could cause long-term problems etc.
I wouldn't see lack of code-review experience as being a blocker to hiring someone - they may simply not yet have worked somewhere with established good practice - but I do think it's a way of presenting code 'challenges' that are much closer to real-world scenarios.
I agree about the need to test technical and inter-personal skills, which is why I described using an open ended architecture/design problem to do so.
I disagree that a code review is as good. Discussing how code should be structured and the trade offs of various decisions is more representative of technical skill than catching style errors.
I don't agree that code review is simply about catching style errors: that's what linters and tools like Prettier automate for you. Amongst other things code review should definitely be looking at how coding decisions conform to architecture/design goals.
We at least both agree that using real-world scenarios is a better approach than personality tests and abstract code problems :)
I interviewed at Slack recently and literally part of the interview was doing a code review. I don't have any insight into how they reviewed it and how it factored into a hiring decision, but I thought it was incredibly smart.
The ironic thing is that one goal of a personality test would be to filter out sociopaths. But a sociopath is going to be capable of manipulating their answers of a personality test to "pass".
I'm disappointed to hear that about ThoughtWorks. Up until now I have only heard good things about them. I'm glad you were in a position to cancel an interview. Those of us in that position are the most able in rejecting poor hiring practices.
I've worked with a few people who got along with everyone, except for the women on the team. I'm not sure how you test for this in an interview. While I don't like personality tests (and in fact have dropped out of interviewing in response to being asked to take one), a more subjective approach has its own flaws. Whether that's hiring someone who doesn't get along with women, or not hiring someone of a different background because you don't click with them as fast as you do someone of your own background.
I do like your approach of testing for personality in a more technical setting. Rather than just trying to get along with someone, you see how they respond to technical problems with the assumption that their ability to handle technical conflict will carry over to their ability to handle social conflict.
You bring up an excellent point. People will definitely act differently in different contexts, such as talking to women vs talking to men. There are so many to try out though, different races, different age groups, or even different regions of the same country. It is incredibly difficult to test out all those contexts without having them work at the company for months. Even then, you may not have someone on the existing team with a certain background. And then you'll only find out someone you've hired has conflicts later on.
I don't have a perfect answer here. I don't think a personality test will solve this problem because what would the abstract question be? "Do you hate people of X?" It would probably be more subtle, but most people would understand what the question would try to get at and provide an answer they think is acceptable rather than an honest one.
Part of me has wishful thinking in knowing that software development is a pragmatic endeavor. I'd like to think that the best developers focus on that pragmatism and know logically that they should focus on what a person can do rather than that person's background.
Yeah, it's definitely impractical to have the candidate sit down with someone from every background. Not to mention that puts an undue burden on minorities (if 2% of your company is black and you want a black engineer on every interview slate, you're asking them to spend significantly more of their time interviewing than anyone else). I don't have an answer.
Some comments may only be visible to logged-in visitors. Sign in to view all comments.