Tips from an SVP of Engineering: How to interview engineers

Growing your engineering team is exciting: It means your organization has reached an important milestone. But hiring engineers comes with challenges. And while it is common to hear about recruiting difficulties as it pertains to external competition, compensation, and employer branding, the internal challenge of finding the right person for your team doesn’t seem to be discussed as often.

I’ve seen a pattern play out countless times over the years. It usually starts with a small team doing great work and needing additional engineers to tackle a growing backlog of tasks. The hiring manager puts together a job description they think defines their needs and brings in a recruiter to build their pipeline. 

The team starts doing phone screens and the first signs of failure start to show: lots of people don’t seem to pass. The ones that do pass come onsite and for some reason the team can’t seem to agree on whether the applicants are fit for the role. The team churns through candidates and gets burnt out doing interviews while balancing day-to-day work, and the recruiter starts to feel dejected having to turn down so many seemingly great prospects.

I’ve observed this same pattern happen on teams with varying degrees of existing talent and varying degrees of support from recruiters. I won’t claim to have all the answers to avoid the situation all together, but there are a few key strategies that I have seen teams adopt that improved their chances of overcoming this hurdle and progressing towards their hiring goals.

---

Define your required skills

Before diving into interviewing, teams need to step back and define what fundamentally makes a solid team member. This starts with basic questions like, “How important is it that a candidate knows this programming language?” This might seem like too obvious a question to require a discussion. But if you don’t align on it beforehand, you could get into a situation where one interviewer turns down a candidate purely based off of their lack of experience with a language, while another’s view is that this candidate is an excellent problem solver and would likely pick up the language syntax on the job with just a bit of support. I tend to lean towards the latter opinion and believe that, using Android for example, an engineer that can understand and explain why the ListView/RecyclerView concept exists, what problem it is trying to solve and when it is appropriate to use, is a lot more valuable than someone who knows how to recite the code. Where does your team stand, and have you had this discussion?

Beyond technical capabilities, it is important for your team to define a set of soft skills you all agree are valuable to the success of your team. This exercise will help the team think a bit more critically about what things don’t matter as much. For example, is a lack of experience in an agile environment as detrimental to your team as an inability to receive feedback? The goal is to align your team on four to six soft and technical skills that you all believe are critical to success, and to communicate that information to everyone involved in the hiring process. Your recruiter needs to be part of this discussion so they can screen for these skills early in the process and tailor their search.

Give your interviewers distinct roles

The great thing about defining skill requirements before diving into the interviews is that it helps you plan how the interviews themselves will be conducted. There are two common themes that I have observed in poorly planned interview panels:

• There is a lot of overlap in the technical skills assessed in the interviews
• Every interviewer is assessing everything, and by doing so, is actually assessing nothing in true depth

Too often interviewers focus interviews on finding the flaws in candidates, rather than trying to find and highlight the strengths they can bring to the team. By repeating the same types of questions again and again, you don’t give the candidate a chance to demonstrate another area of strength, and can potentially keep digging into one of their weaknesses. If you try to assess every soft skill in every interview, you become focused on sussing out a single deficiency to raise as a concern. It will distract you from seeking a deeper view of the person’s experience, potential for progress, and risks in a specific skill set, and you’ll end up with a superficial view of the candidate.

Solving this problem can be as simple as segmenting your interviewers’ roles. Give each interviewer a specific skill from your list to assess. Their goal is to dive deep in that area and report back on their findings. If your panel is large enough, ideally you can have each interviewer overlap one skill with another interviewer, allowing for at least two opinions on every skill. It is quite interesting what happens when interviewers shift their focus from “Is there anything that concerns me about this person?” to “Can this person really demonstrate this skill to me?” Oftentimes this results in much more fruitful discussions and actionable feedback.

Tailor your interviews to the skill level of your candidates

Leaders often overlook how to interview junior and senior engineers differently. The common approach of an interviewer is to ask the same questions in every interview so that they have a running calibration and know what to look for. Does that still hold when you interview a person with one year of experience followed by someone with ten? If your interview is purely algorithmic (which brings up a whole separate topic of discussion), you’ll likely find that a more junior candidate fresh out of a college computer science exam will outperform a seasoned engineer with industry experience. Is that a fair outcome that leads to hiring the right person for your team? Think about how your team can change the interview process and the expectations for candidates depending on their experience level.

The flip side is that you can’t just change the interview if the role requires a specific set of skills. Your interview should be aligned to what the role demands, and it doesn’t make sense to change the bar based on who you are talking to. The team, starting with the recruiter, should be aligned on how senior of a person you really need for the role. Sometimes teams aim really high because they want the best and bring in people with ten years of experience expecting them to take on a role that a fresh graduate could easily do. You’ll struggle to excite and retain this hire, and more surprisingly, you actually might find you’ll be rejecting many of those candidates because you think they are “unqualified” simply because you are asking them to do work that isn’t aligned with their current skill set.

Encourage discussions about candidates

The final point I want to touch on is feedback systems. A common practice is to have interviewers submit their feedback into an online portal that hiring managers and recruiters will read to make the final decision. Hours of conversations are boiled down into a few paragraphs of text, all to inform one person’s ultimate decision. While this may seem efficient, you take out all the feedback loops out of the process. Interviewers don’t get a chance to hear challenges to their opinions of the candidate. They don’t get to hear other evidence that might make them rethink their impressions in a new context. You also often fall into a trap where engineers with limited time will simply write brief feedback that glosses over many nuances from their conversation. Those missing details could end up swaying a decision in one way or another.

One way to address this problem is to have an interview debrief panel after every interview. Gather everyone in a room and go around briefly discussing any key highlights or lowlights from the interview. You’ll be surprised at the discussions and self reflections that can come out of this process, and you’ll have a lot more visibility into where your interview process might be breaking down. It is a small time investment to make, and it could end up saving you much more time down the road.

One other thing that can be helpful here is to have the hiring manager be the final person to formally interview the candidate. This allows them to hear how the candidate thinks the interviews went and to get a sense of how well the team is selling the opportunity throughout the day. Candidates will be more honest than you’d expect about the process, especially if things are going poorly. Your recruiter is also a great partner in collecting this feedback, as they’ll often be the last person that talks to the candidate on their way out at the end of the day.

Set your team – and your candidates – up for success

Recruiting isn’t easy, but a few simple steps and conversations could save growing teams from wasting time and energy. Try to avoid that first urge to just dive into resumes and profiles. Instead, step back for a few minutes, define your process, and align on what the right candidate means for your company. You’ll save yourself a lot of grief, but arguably even better, you’ll learn a lot about your own team.