When I was a fresh university computer science graduate a company I applied for set me a problem, and got me to write some code. This made so much sense to me that I’ve used this approach throughout my career. In fact, as a candidate, I don’t like it when a company doesn’t get me to write code. If they don’t assess my code, then they haven’t assessed the code of my (possible) future colleagues.

After all, as they say in Peopleware, you wouldn’t hire a juggler without seeing them juggle first.

While hiring recently, I had a candidate object to the take home test. He wanted to be paid to do it. I don’t believe that it is ethically sound to pay someone, or take payment, during an interview process. But, I could see his point. It’s more time, on top of our interviews and any other interviews he’s currently going through. What is the right way to assess someone’s technical skills during an interview process?

Firstly, why use a coding test at all?

1. I want to see the code you write. I want you to write the code in an environment that is most comfortable to you, with the tools you are most familiar with, using all the resources you would normally use. Writing code on a whiteboard is so artificial that all it demonstrates is that you can write code on a whiteboard during an interview.
2. I want to read the code you write when I understand what it’s supposed to be doing. Code is read much more than it’s written, after all.
3. I want to work with you on the code you write. I want to see what you’re like to sit next to and work with.
4. I want to see the code you produce now, with everything you’ve learned. I’m not hiring you five years ago, I’m hoping to hire you now. Who are you now?
5. I want to be sure that the code you give me is code you wrote.

Setting a problem and giving the candidate a few days to submit a solution provides all of that. But there are downsides. It privileges those people who have the time to work on problems like this in their spare time. It’s hard to construct a problem that is not trivial, but doesn’t take too long to implement. And because the problem is so clearly a toy, it’s hard to get a sense of the code someone would write as part of a real problem at a real job.

A common message over the last few years has been that Github is now your resume. As others have written, this is not acceptable. This privileges those people who have the time and economic support to build extensive open source portfolios — who are overwhelmingly people with plenty of privilege already. Plus, these contributions might be old. Plus, you’ll probably follow the style of the project, which may not be yours. Plus, I can’t be sure that the code is yours.

Privilege is the central problem though. If we are to build the best teams, and if we’re to improve our industry we have a responsibility to identify pre-existing advantages that are helping one group over another and do what we can to level these advantages.

Prompted by this candidate, I had a long discussion with friends, former colleagues and peers about how to compromise. One friend asked me a very good question:

Are you looking for a one-size fits all solution?

Now I give candidates choice. Either solve a problem, or point me at some code you’ve already written. I explain why we’re asking for code, and what properties the code should have. Mostly, so far, candidates have been choosing to solve the set problem.

Here are the instructions I send to candidates. This is after they have already passed a technical phone screen with an engineer.

Thank you for your time so far. We believe that it is very important to see code you write, and to work with you on writing code, as part of the hiring process. This should also give you an opportunity to see what we value, and how we work. There are two options:

1. Please implement a solution to the attached problem. Feel free to choose the language. It’s small, but please demonstrate (at a small scale) the things you care about when writing code. We believe that a useful sample for the remainder of the hiring process can be created with two hours work. We’d also love to hear how long you spend on the problem as that will help us improve. We typically spend two hours reviewing each submission.
2. We recognise that some candidates have an existing portfolio of code that speaks to their skill as a developer. If this is you, please submit some code you have already written. This must be under a suitable open source license, such as GPL, Apache, MIT, Eclipse or BSD. It must also be small enough for someone who is not familiar with the code base to understand and assess in two hours – a good guideline is code that took you two hours to write. If you are submitting a Github (or similar) repository, then please point us towards the portions you’d like us to read.

For the next stage in the process we will pair with you to extend and refactor the code you submit, so please bear this in mind if you choose to submit something of your own.

Why Two Options?

Anthemis is building a technology team that reflects the customers of the companies we launch. To enable this, we want a hiring process that is as free from bias as possible. Accepted hiring norms and the constraints of the tech industry make this hard work, but you wouldn’t hire a juggler without seeing them juggle and you shouldn’t hire a developer without seeing them write code. If we ask you to submit open source code on Github, then we’re giving an advantage to developers who have the time and resources to work on open source. If we ask you to solve our problem, then we’re making it harder for people who have busy lives right now. So we’re offering two options, and we’d welcome your feedback on how we further improve our hiring process.