One of our most recent seed investments recently posed a question to me. They had designed a technical hiring process to try and eliminate gender bias but were losing all the women midway through their funnel when a take-home project was introduced. What was going wrong?
I reviewed their current approaches alongside Homebrew’s Head of Talent Beth Scheer and we made a few recommendations, but there was nothing glaringly broken. Attention then turned to the take-home project since that’s where the breakage seemed to be most acute. Was there something about this step that was unintentionally sending the wrong signal to female engineers? This wasn’t my area of expertise but I’m fortunate to have smart friends so I asked Joelle Emerson, founder of Paradigm, a leading consultancy providing training and strategy around inclusion. Her response was, not surprisingly, really great and she agreed to let me publish it below in its entirety. Hopefully this will help other startups too. Thanks Joelle!
Email From Me to Joelle
hi! QQ for your expertise. Have you seen any research or have any POV on how giving a take-home coding project during interview process impacts genders differently in the funnel? ie increases/decreases candidate or employer decision to move forward in process, and does it differ based on when in the process it’s introduced?
Hi hi! Great question. Take home exercises can be a really helpful complement to interviews. When they’re designed thoughtfully, they can give great signal on how someone would actually approach their work, and can reduce subjectivity and bias in hiring. But whether a take home project is effective or not depends entirely on how it’s designed and framed. Here are a few things, in particular, I’d be on the lookout for given the circumstances you’re describing (where women are less likely to participate in the exercise):
- How the project is framed: Your instinct on this is spot on. The language around how both how this is introduced, and how the exercise itself is framed, can have a big impact. A few things to look out for:
- Growth v. fixed mindset language: Is this described as an assessment of ability or approach? It seems like it’s not being framed as a “test,” which is great, but there can be other language that makes this feel like you’re trying to get at someone’s ability as an engineer, which can lead candidates from underrepresented backgrounds to experience stereotype threat, and select out.
- Vague vs. clear language: Are you specific about exactly what this will entail, and what you want to see from people?
- Masculine/homogenous vs. inclusive language: I’m guessing you’ve already looked at this, but language that skews masculine (words or phrases like hard core, dominant, competitive, intense etc.) would be unhelpful 🙂
- What the project actually is: I’m not an engineer, so I probably can’t be super helpful here, but I’d be curious about what you’re actually asking them to do in this project. Does it simulate what real work would look like? (It sounds like that’s the goal, which is great!) Does it bias towards a specific type of engineering style or approach? Is there anything else that might make the project itself less exciting to some people?
- How long the project takes: People from underrepresented backgrounds in this industry are less likely to be able to invest a ton of time in a something, unpaid. How big a time commitment are you asking people to make?
Now, the postscript on my unnamed startup is that they’re a small team which is prioritizing diversity and reaching back out to the female candidates who decided not to go forward in order to offer a path without the take-home project if they’d like to continue discussions. I’m proud of this CEO for being so proactive in iterating and improving!
Update: Textio’s CEO sent me some helpful tips regarding Take Home engineering