I believe that a live tech test is still part of the most effective way to establish a candidate’s ability, a feeling that has only been strengthened by the growing use of AI in interviews and screening processes. I also believe, however, that the code a candidate writes is the least important part of a live coding interview. I also feel that the most important function of the live tech test is not to establish the candidate’s ability to code - though this has become a more important side effect as AI has allowed candidates to fake their experience more than ever before (anecdotally I’ve seen evidence of this more than once first-hand). Instead, the main purpose of this interview should be to establish the candidate’s ability in the other arguably more important parts of our role.
The most important aspects that I care to see from a candidate in this type of interview are:
With this in mind, lets try to imagine the best possible case for how a tech test might proceed:
Ideally a candidate will, once they’ve arrived and introductions are done, begin by reading through the task in front of them. This may seem simple, but in my experience its often one of the best indicators for how the rest of the task will proceed. The candidate should read and comprehend the entirety of the task, including follow-on questions, and briefly discuss their initial thoughts on the problem. They may ask questions, but should at least be able to immediately grasp the complexity and general approach necessary. With this being the best case scenario, I consider it ideal if the candidate either verbally discusses their initial plan to solve the problem, and even better if they draw out the problem or solution visually to make it clearer for both themselves and the interviewers (where appropriate).
Next, I would expect to see the candidate begin coding. This being best-case, they should immediately start to work on a sensible solution, which they can explain as they go. We should see them build it in such a way that later sections are taken into account; reusable util functions, sensible return types, and implementations that break the solution into easily modifiable sections are ideal. I expect all candidates to have to adapt their solution to varying degrees as they go, but in the best case I expect them to be able to explain the change and their reasoning behind it.
I’ve seen excellent performances without looking anything up, but for myself its a big plus to see the candidate googling things as they go. An encyclopedic knowledge of the standard library is impressive, but doesn’t represent the day-to-day reality of the job, where research is constant and vital. I like to see the candidate search effectively, choose strong sources, and use that knowledge in their solution. Finally, I want to see the candidate make a mistake. The way they react to a mistake is possibly the most important part of the exercise, and they should be able to identify the bug, explain it, and resolve it quickly under pressure. I’ve never seen a candidate make no mistakes at all, but I would consider it an unfortunate turn of events if they did as we’d miss out on important context about the way they work.
Finally, the candidate should be challenged on some aspect of their approach. If they haven’t already discussed an alternative solution at any point with the interviewers, they should be prompted with one now to see how they respond. Its entirely acceptable, and even likely, that they will defend their own approach, but being able to take other viewpoints on board and consider their potential advantages over their own is a vital part of working within a team.
Finally, with this in mind lets envision the kind of test might make space for this type of performance. First, the complexity. Tests should be completed at a prior time by the interviewers to establish how long it takes (and to give context during the actual interview) - they will always take longer during the interview due to the pressure involved and the expectation to explain the approach as they go. Therefore the test should be tuned so that it can be completed out-of-context in around 60-70% of the time (using the same tools that the candidate will be afforded).
It should require as little external knowledge as possible, but also provide the opportunity to use standard library features like JS’ array.sort() or string.replace() who’s syntax might need googling. Obviously if pre-existing knowledge of a library is considered mandatory for a role then that can be included, but I’ve always considered this secondary for most roles.
A test should ideally consist of at least two parts, where the later ones build on or partially reuse the solution to the earlier ones. This gives the candidate the opportunity to display forward thinking, and to show how they consider resusability and adaptability when writing code. And finally, a great test should include a pitfall; not necessarily a trap, but an easy mistake that they’re likely to make that they can then identify and debug. The purpose of including this is to give the interviewers prior knowledge - removing the need to try and debug an issue yourself gives you the chance to watch the candidate arrive at a known conclusion, and observe their process in as much detail as possible. Examples of these pitfalls could be data or tasks that break common conventions, requiring the candidate to rethink their usual approach; for example arrays with custom indexes that start at 1.
I’ve taken a crack at adapting a tech test to fit my own guidelines - its adapted from an example from Code Your Future’s bank of mock interview questions, and I’ve posted it as its own article here.