DEV Community: Kaitlyn Tierney

Your Hiring Process Is Probably Biased

Kaitlyn Tierney — Mon, 04 Jan 2021 16:44:21 +0000

This post is cross-posted from my blog, just like my last dev.to post in 2017!

Hiring is hard and important

Hiring is hard. This is true in any industry, but in the tech industry we're in the habit of handing out life-changing salaries to people who have bootstrapped their own careers. With our hiring choices, we have the power to reshape the financial reality for people from marginalised backgrounds or impoverished communities.

It's not uncommon for skilled developers to be neurodivergent, or to struggle with mental illness. For many people, mental health problems have been exacerbated by years of abusive treatment within the industry, or by burnout. As a hiring manager, you have a duty to care for the people you interact with. At a minimum, your hiring process should not actively hinder people's well-being and mental health. When successful, your hiring process is a critical step to building a high-performing engineering team and a great product. When done badly, you'll burn your reputation and you'll struggle to recruit, or you'll find yourself only recruiting people who resemble your current (likely homogeneous) team.

You are not entitled to good candidates

This post is for engineering managers, technical founders, engineers who aspire to become managers one day—for anyone with the responsibility of hiring software developers at a small to mid-sized company. It contains strong opinions, anecdotal examples of actual hiring processes I have endured, and quite a lot of advice for ways you might be able to improve your hiring.

Of course, my idea of a better hiring process might not be the same as yours. If you don't agree that diverse teams build better products, you've found yourself in the wrong place. Please move along. I'm not here to sell you on the importance of diversity, and you have bigger problems to sort out than your broken hiring process. You can try to build a diverse team within an organisation that doesn't value diversity, but it won't be sustainable and you'll risk recruiting wonderful engineers and then giving them a bad experience. Please don't do that. I've made that mistake, and I can assure you, it feels terrible. If anyone from HR or your executive team says the words, "It's a pipeline problem," to you, just run as quickly as you can into the arms of a new and better employer.

The goal is simple

Healthy hiring has a simple goal: to enable candidates to show you their best work, so that you can try to assess whether they'll be able to perform the job responsibilities, and so that they can assess whether they'd like to work with you.

Brittle == biased

Hiring can be conducted on a rolling basis—you have many openings to fill and a good runway to pay them—or what I like to think of as "high-stakes hiring"—you have one or two immediate openings, and must select the best candidate(s) from a pool of people who have applied during a fixed time period. The process is basically the same for both, but your feedback loop is longer for a high-stakes hiring event, since you won't be able to enact improvements to your process until your next hiring round. With rolling hiring, your process can evolve and improve over time as you learn from the feedback candidates provide to you.

Building a good hiring process follows the same cycle as building a good product: Build, measure, learn. But how do you make sure you're building the right thing? Who is your hiring process optimised for? One size doesn't fit all—not for socks, not for t-shirts, and especially not for assessing the diverse range of skills and capabilities that make for a good software engineer.

Most hiring processes for software engineers are divided up into a few rounds, or phases. There's some sort of intake procedure, a sifting or screening process, often performed by an algorithm or a low-level HR employee or support staff. There's likely to be one or more technical assessment rounds. There will almost certainly be some sort of interview delving into a candidate's past experience, and often an additional interview to evaluate "culture fit". These stages can drag on for weeks, even months, or can be pushed through in less than a week. Sometimes the timing is flexible, while the steps followed are not. Bias creeps in at each stage of the process, and it's easy to lose sight of the goal.

If your hiring process isn't flexible, you're defaulting to optimising for a certain type of candidate. Is that really what's best for your business? Is this a choice you've made, or a trap you've fallen into? Consider the wide range of people you might be excluding, and the valuable perspectives they could be bringing to your product development:

People with mental illness (anxiety, depression)
Neurodivergent people (autism spectrum, ADHD, etc.)
People from nontraditional backgrounds, who may have job-relevant skills but, if they don't job-hop or interview a lot, may not yet have gained the specific interview skills you're expecting them to exercise
People of lower socioeconomic status, for whom performing unpaid timely labour (completing your hiring process) may not be feasible
People with carer responsibilities, more than full-time work, or other time and availability constraints
People with disabilities, and if you haven't offered suitable accommodation, this may be illegal in your country

Sifting and screening

You probably don't have time to interview every candidate who applies for a job at your organisation. And you might not want to, even if you have time—copy & paste cover letters are common, and some folks clearly take a firehose approach to job applications, especially if your application software offers a 1-click "import my data from LinkedIn to apply" button. This initial screening, to hand-pick the candidates who might be a good fit based on their CV and cover letter, is frequently performed by people far removed from the rest of the hiring process. I can't tell you how many stories I've heard from hiring managers who received a stack of 40 CVs for an open role, filtered by their HR department, only to discover that everyone who wasn't a white man had been filtered out.

It's also a measurable, proven source of bias and discrimination in hiring. It's totally unnecessary, unhelpful, and you should skip it and do something better. And I say this as someone who almost always, 99 times out of 100, will get through this initial screening due to my penchant for writing a persuasive cover letter. This process is clearly biased in my favour and I'm still telling you straight-up that it is bad and toxic and you shouldn't do it.

This is not an advertisement for Applied, but I had the opportunity recently to use the Applied platform as a candidate and see for myself what their sifting process is like. It's fantastic. Candidates are presented with a slate of questions designed to reveal competencies relevant to the role. Once they've submitted their answers to these questions, candidate responses are anonymised and shuffled into random order, presented alongside responses from other candidates for scoring and review. The aggregate of anonymous scores is used as a cut-off to determine which candidates have scored well enough to proceed. During this initial sift, no one responsible for hiring is seeing candidates' names, educational history, or employment history. Candidates are evaluated solely on their responses to job-relevant questions, which (unsurprisingly) turns out is a far more accurate predictor for success.

In my experience as a candidate with Applied, I was asked questions about web performance, code reviewing broken Javascript, using elasticsearch for analytics and reporting, working in teams, and prioritising work. The questions were thoughtfully designed to reveal a lot about my thought proccess and approach in each of these areas, and I'm confident their team learned a lot more about my skills and knowledge than they would have learned from looking at my CV.

Writing skills are crucial to working well in teams remotely. Writing a cover letter is a skill unto itself, and not one which is generally useful during day-to-day work as an engineer. By asking candidates to provide thoughtful written responses to questions relevant to their work, you'll be able to evaluate competence at written communication; by doing this anonymously and in randomised order, you'll ensure that your evaluation is based on salient information instead of the irrelevant background noise to which you've been conditioned to pay attention.

Interviewing remotely

Once you've identified a few promising candidates from your pool, you'll contact them to schedule an interview. It's 2021 and the world is a hellscape, so unless you're lucky enough to be in New Zealand, South Korea, Taiwan, Australia, or a few other countries with competent governance, you'll be interviewing remotely over Zoom or Skype or some WebRTC-based video chat tool.

Interviewing remotely sucks. I love remote working, but if given the choice, I'd always choose an in-person interview. For many people, the technology for remote meetings just doesn’t yet compare with the fluidity of IRL conversation. In particular, unreliable upstream internet service can have huge impacts on how a candidate will perform in an interview.

So it sucks, but we have to do it, so we may as well make the best of it. Luckily, with some slight adjustments to your interviewing etiquette you can set your candidates at ease and give everyone a fair chance at success.

Start by making sure everyone can hear and see each other. This can feel a bit awkward, but normalise starting your call with a verbal confirmation that all parties can see and hear one another.
Outline the agenda in advance. You should include a detailed agenda, with the questions you plan to ask, in the calendar invitation for the interview. At the start of the call, review the agenda and outline how the interview will proceed. For example, "The purpose of this call is to get to know you a bit and to allow you to ask questions of us. We'll start by introducing ourselves, then we'll talk through the questions we shared in advance. We expect that to take around 20 minutes, which should leave 10 minutes for us to answer any questions you have."
Score responses to fixed questions. Ask the same questions of each candidate, as much as possible. Define your scoring system and your questions in advance. If you're hiring on a rolling basis, these questions will change over time. That's fine. But aim to maintain a consistent approach and ensure that it's well-documented.
Watch closely for visible signs of packet loss. You're a technical person. You can see and identify signs of poor connection, high latency, and packet loss. You need to adjust your communication style accordingly, by pausing and repeating yourself as-needed to counteract technical challenges.

Bad connections are inevitable, often outside of our control, and make video meetings an incredibly ineffective and inefficient way to communicate important information. We still do them, because it's nice to meet someone face-to-face and see how you interact with one another. But the limitations are substantial. Use video calls to augment your knowledge, but do not rely on them as the only means of collecting certain information about a candidate.

One of the best interviews I had was with a company who build a WebRTC-based video chat system. I spoke with two of their engineers, who were on it when it came to noticing packet loss and adjusting the conversation accordingly. Even though the connection wasn't great, we were able to have a delightful, comfortable conversation by working around the technical difficulties.

One of the worst interviews I had started with, "I'm XXXXX and this is XXXXX. Let's start with the back-end task. Can you share your screen?" For this interview, I requested an agenda and more information about the nature of the interview in advance, but was told my questions wouldn't be answered because it "wouldn't be fair." This was not a high-stakes hiring situation; the role had been open for months and many recruiters were working to fill it. If you're relying on your interviews to weed out candidates, or to pit them against one another, you've lost sight of the goal. To enable candidates to show you their best work, help them to come prepared, just as they would for a day at work. And for goodness sake, at least start your call with, "Hello, how are you?" and "Can you see and hear me okay?"

Assessing technical competence

Tech tests, coding challenges, technical assessment. Whatever you call it, this is where a lot of places fuck up.

Many companies use a take-home coding challenge as a second-round screening, to eliminate candidates from the funnel who don't show a particular set of skills in their solution. Congratulations, you're optimising for people with free time and comfortable socioeconomic status, and you're placing the burden of evaluation solely on the engineers on your team who are willing to score a coding challenge.

Some hiring managers look at side projects and open-source contributions as a way of assessing technical competence. Congratulations, you're optimising for people with free time, comfortable socioeconomic status, and who have not been victims of harrassment, abuse, or stalking.

Maybe you prefer to do a technical interview or some pair programming? Congratulations, you're optimising for people who have taken the time to hone their technical interviewing skills. These skills are not actually very related to the job you'll be asking them do. You're effectively eliminating candidates with anxiety, from nontraditional backgrounds, or with some forms of disability. Pair programming is particularly insidious, as it shares a name with a common practice you might engage in at work, but in an interview format it bears no resemblance to the other kind of pair programming you might be familiar with.

There are a lot of different ways to be a good programmer. A person usually only has to be good at one or two of them. Odds are good that you’re testing for skills you don't need. This may also be preventing you from assessing far more valuable skills that you’re not seeing. To truly assess someone's technical competence, don't lose sight of the goal. How can you enable a candidate to show you their best technical work? I don't know! And you don't either. The only person who will know how they work best is that specific candidate. This is where a brittle process will fall flat, losing you valuable candidates and poisoning your name if you torment people unnecessarily. Define the skills you're looking for in advance, and offer your candidates a slate of choices for demonstrating those skills.

Options for technical assessments

Take-home tests
- 👍 Working at home and independently is similar to an actual remote working environment. A well-designed take-home test can allow anxious candidates a good opportunity to show their best work.
- 💰 Can be inaccessible for candidates with care duties and full-time jobs, so your take-home test shouldn't be a firm requirement. If you must have a candidate complete your take-home test, you need to pay them for their time.
- 🤖 The challenge should be as closely related to your business domain as possible. I have seen challenges to build a vending machine (for an AI-type chat application), to program a robot to move in a certain way (for a fintech company), and to make a 3-card Monty game (for a company doing film analytics). Just.... why. Don't do this. Don't do any of this.
- 🛠 Unless you're hiring a developer for a greenfield project, you're probably looking for someone who can work well in your existing codebase. So make sure your take-home test includes starter files! Bonus points if you use your actual codebase, or some sanitised portion of it.
- 🔍 Requirements must be clearly defined. No, more clearly than that.
- ⏰ Be mindful of your timing! Particularly in high-stakes hiring, if you're giving all of your candidates the same take-home test to do over the same time period, you're a real asshole if you make that time period "the weekend of Rosh Hashanah". This did happen to me. I don't think they have any Jews working for them. Probably not too coincidentally, there are also no women on the team this was hiring for.
- 💞 The feedback you provide to candidates should be of comparable quality to what's shared in an internal code review. This means it should be kind, empathetic, and constructive. If you find yourself providing the same piece of feedback to everyone, that's a great indicator that your challenge requirements aren't clear enough.
Technical interviews
- ⚖️ More respectful of candidates' time, but require skills that are interview-specific, not very related to job duties
- 🧩 Can be a good way to demonstrate technical competence in lieu of a take-home test, or to follow up on knowledge gaps potentially evidenced by take-home tests
- 🧠 To get the best performance from candidates, allow them to prepare in advance. A good developer doesn’t go into a meeting unprepared. Why are you screening for people who excel at doing that? It’s far more interesting to see how a candidate can approach a problem if they’ve been given some time in advance to think it over and understand the context, particularly for design and architecture problems.
- 💣 I've often heard the conventional wisdom that developers should practice their interview skills regularly by going on interviews for jobs we have no intention of taking, or applying for jobs even when we're happily employed elsewhere. While it's true that often these interviews, particularly systems design and other technical interviews, generally require practice to do well at, this advice is, frankly, absurd. Going to practice interviews is a waste of everyone's time, and privileges people who can afford to take time out of their jobs to do this regularly. And bad interview experiences can be psychologically damaging! Practice for the sake of practice doesn't come without risks. Moreover, the reason these types of interviews require practice is because they don't exercise skills actually used regularly on the job.
Code review
- 🔑 A trend I've been seeing more of lately is to include a code review phase of technical assessment, often as part of a take-home test or interview.
- 🧶 I like this, but it relies on clear requirements to work well. If your take-home code review assessment says something like, "You're reviewing this code for a colleague," that raises more questions than it answers. Determining the appropriate tone and content for a code review is highly context-dependent, so you need to make sure candidates have that context.
Trial work day
- ⏳ One place I interviewed used a "practice onboarding day” in lieu of other technical assessment. It’s a good option if you’re in a high-risk hiring situation, where you need to give a fixed number of candidates a deep assessment over a short time period.
- 📅 Sometimes this can be extended; you can offer up to a full week's worth of trial work if you're very nervous or unsure.
- ⚡️ You must offer options; this option won't be accessible to people with full-time jobs if you're expecting them to do this during regular working hours.
- 💸 Of course, you must pay them.
Side projects or open-source contributions
- 🐙 This is fine if the candidate offers up some code to look at, but bear in mind that it might not tell you what you need to know.
- 🔒 It's reasonable to use this as a positive measure, but it's not acceptable to count an absence of side projects or OSS contributions against a candidate.

Feedback and fallout

A bad experience with your hiring process can ruin a person's week, or even month. You don't want that kind of ill-will towards your company floating around out there if you can avoid it. (And you can! It's not that hard to not be awful!) What feels like a good experience to one person might be anxiety-inducing or worse for others. People with mental illness, disabilities, or other constraints have developed strategies to be successful at work, but these strategies don't necessarily map to the different set of skills required to interview and apply for jobs. By making your hiring process as close to real-life work as possible, you'll gather more relevant information and stop excluding those candidates who may provide the most value to your organisation.

Over-communicate, and communicate in multiple formats, to align expectations and overcome mismatches in communication style or technical issues.
Provide valuable, constructive feedback to all of your candidates, and listen to their feedback for you.
Offer options and accomodations during each of your hiring stages to enable candidates to show you their best.
Share questions and problems in advance so that candidates can come prepared to dazzle you.

With a bit of effort, you can craft a more inclusive hiring process. Effort expended up-front to make your process more pleasant for everyone will pay dividends in diverse teams of people who feel valued and supported, and who are empowered to do their best work for you. Good luck!

My First Year as a Developer, Post-Bootcamp

Kaitlyn Tierney — Wed, 28 Jun 2017 09:53:55 +0000

This post was originally published on my blog, but after it gained some traction on Twitter, I realised it might be suitable for a wider audience...

Today marks one year since I walked into General Assembly's fresh new Aldgate East campus for the first time, bright-eyed and bushy-tailed and ready to throw all of my money and three months of my time behind one singular goal: Get a decent job in London.

In as much as there is a typical profile (there probably isn't), I never really thought I'd be a likely coding bootcamp alum. I've been making websites most of my life, teaching myself HTML and CSS by snooping code from other sites before I really knew what "code" even was. Much later on, I took courses covering basic HTML, CSS, XML, SQL, all for my graduate degree in library & information science. I knew as much code as I needed to do my job well, and was comfortable enough with it to poke around and build things (or modify existing things to suit my tastes) in Wordpress, Joomla, Drupal, and whatever else people were using.

And yet, jumping from that level of technical ability to actually being a professional developer seemed impossibly daunting. Somewhere down the road, perhaps while I was spending my early 20s becoming a terrific but completely unemployable librarian (inasmuch as nearly any librarian is unemployable these days, thanks to the economics of supporting social services and libraries in a capitalist society), the inner workings of the internet just passed me by. I don't know when exactly building websites became synonymous with knowing a lot of JavaScript and understanding what, exactly, is meant by that mysterious phrase "back-end" that I had tossed around so casually—as in, "Oh, if you want this to be connected to a database, we'll need someone to handle the 'back-end' bit." I don't know when it happened, but I knew that I was sick of mentally punting that responsibility to other people. I hated that there was an entire domain of technology that I was almost entirely ignorant of, and I hated to see it invading my precious internet and closing me off from playing around and understanding how things work.

I also hated putting up with sexist bar patrons and trying to fight my way through the male-dominated craft beer industry, which I had sort of fallen into after failing to find library work, or copywriting work, or editing work, or any other sort of gainful employment after relocating to London. So naturally, becoming a web developer seemed like a solid career move. If I had to put up with sexist pigs, at least I could get paid a bit more handsomely for it, and go back to working regular daytime hours.

The lessons I took away from GA are not necessarily the ones I thought I was paying for. The Web Development Immersive course teaches students how to read documentation—this is a very important skill for an aspiring developer. It teaches the basics of how to use Git + GitHub, and makes sure everyone is able to put together a CV and portfolio showcasing the projects built over the course of the program. The course assignments start out pretty interesting, as students learn the basics of programming and thinking logically to build simple games like tic-tac-toe or roshambo in JavaScript, and the curriculum builds students up to a level where they're able to create standalone webapps that run locally using whatever framework is in vogue and being taught by the course—we learned to build boilerplate applications with Express, with Sinatra, with Rails; we learned the basics of jQuery syntax and a bit about SASS. I learned that the instructional team and advisors at General Assembly thought that my Twitter account was too aggressively feminist and that it would prevent me from finding jobs. I learned that if the white men who pay for the course think that there's a pipeline problem in tech that it's not my place to correct them, and that if there are fewer female developers it's because women just aren't as interested in computers. I learned that junior developers don't write tests in their jobs; that task is reserved for senior developers. The point is, I learned a lot, and much of it was good and useful, and some of it was not. As I saw more examples of the type of developer I didn't want to be, I was able to form a pretty decent impression of the type of developer I'd like to become.

I've always been comfortable enough hacking away at things to make them work—in my view, that's not the mark of a professional. If I'm to make this my vocation, I want to do it well, like, really well. I want to be able to write software that other developers can look at and say, "Yes, this makes sense." I want to write code that's maintainable, performative, self-documenting, well-tested, that does what it's meant to do. I also want to really, deeply understand the tools I use, which is a process I understand will take rather a lot of time.

It's hard not to despair at times, feeling like I'm at least a decade behind the curve and missing so much foundational knowledge. It's even harder to measure my progress—how can I quantify knowledge gained, or deeper understanding slowly percolating in my mind? I can't throw together a Rails application any faster now than I could when I built one for one of my GA projects, but my process for doing so is completely different now (and it includes a hell of a lot more test writing).

My favourite part of being a professional developer so far is, surprisingly, something that's a bit of a divisive topic: Code review. After years of hacking away at things and making them work, it's so refreshing to have the confidence to share my code with a mentor and get feedback (even when the feedback is basically, "Tear it all down and start over, but here's where to start and what to think about"). I've been really fortunate to work alongside great developers who give thoughtful, considered feedback and are exactly as pedantic as I'd like them to be. I've gotten great feedback about design decisions, architectural patterns, structuring database queries, and how to optimise for performance, and I've learned that naming things probably isn't as hard as everyone thinks it is—at least, not if you've got a decade of writing and editing experience under your belt.

I'm extremely grateful for everyone who took a chance on me and helped me get this far over the past year—pretty sure you know who you are already, and naming names feels a bit gauche. I plan to repay that debt by putting in the effort to grow into a contributing member of the community, continually fighting against discrimination and cultural hegemony, helping to encourage aspiring developers to boldly pursue their dreams, and calling out bullshit and bad takes whenever I see them. In short, I'll continue to just be myself, but I'll probably get further and bring more people along with me thanks to your help and support.

xx,
Kaitlyn

Hi, I'm Kaitlyn Tierney

Kaitlyn Tierney — Wed, 28 Jun 2017 08:56:25 +0000

I have been coding (professionally, full-time) for one year.

You can find me on GitHub as krtierney and on Twitter @krtierney

I live in London.

I work for Happy Bear Software. We're a small Ruby on Rails consultancy, with a focus on improving security in Rails applications.

I mostly program in Ruby, Python, and Javascript.

I am currently learning more about Elm, shell scripting, cryptography, networking performance, and trying to become proficient with vim after getting annoyed at Atom crashing one too many times.

Nice to meet you.