How My Favorite Interview Questions Became My Day Job
Julien shares how his favorite interview questions became the real challenges of his day-to-day job.
Interviewing AppSec Engineers
I’ve worked in large software companies for a significant part of my career as a security engineer, and I’ve had to interview hundreds of potential engineers for Citrix and Amazon. Usually, I was in charge of evaluating their AppSec approach and skillset. A typical interview was about an hour and was split into the following sections:
Intro and warmup: 5 mins
Tech and tooling questions: 15 mins
Role play section: 30 minutes
Reverse interview: 10 mins
The Interview Structure and Philosophy
First, let me explain the last section, the reverse interview. This is an opportunity to exchange roles and let the candidate ask any question about the role, company, or anything in tech.
The first section, intro and warmup, contained one important point for this interview: there is no good or bad answer. As the interviewer, I will not spend too much time digging into what the candidate knows but will try to explore as many topics as possible to be able to paint a clear picture of the candidate’s skillset. Therefore, it is expected that I will ask questions about stuff he/she doesn’t know, and it’s fine to say, “I don’t know” as much as it’s fine to say, “I’ve never been in this situation, but this is what I would do.”
The tech and tooling section questions usually revolved around the candidate’s experience with AppSec process, how they ensure that code produced by dev teams is actually secure and how to measure this. This is used to create a comfort zone, understand what the candidate is good at, what are the technologies or languages he/she is familiar with, and where I can take real world examples.
The Role-Play Scenario and What It Reveals
The turning point of the interview happens with the role-play section. I usually start, with the following scenario:
You have just been recruited as the first security engineer in a tech startup in the early days. The company will launch their new product in three months, and they are currently wrapping up the development phase. There is no process, no tooling, and as the first security engineer, you realize that security skillset is mixed among the staff. Take some time to think about it, ask me more context questions if needed, and tell me what do you do in these three months to ensure the product is secure and launches successfully?
I then proceed to setup a product context with a mix of technologies he/she is familiar with, but also some unusual design choices and shapes: ex. ”Backend API is old and written in PHP/perl/Pascal”, “product has a mobile app deployed to a million users”, “product is deployed on untrusted devices”. At this point, I’m listening to the candidate’s proposals, looking for signals of the following:
What is the technical AppSec approach? Is it sound?
What is the human strategy? I may have to create some non-player-characters for the company stakeholders.
What about buy-in from leadership?
What are the business related questions, compliance goals?
What is their security response strategy? How much is needed now?
Pressure Testing and Adaptability
After 10 minutes, I usually drop a surprise card in order to talk about the subjects the candidate did not account for like: “The CEO calls you at 6pm and asks if we can achieve SOC2 compliance by the end of the week”, or “An engineer points you to a reddit post about how some student across the globe who has built a free tool that seems to be using our product backend”, or “Instead of 3 months of runway, we need to ship in four weeks”. Then I ask, “How do you adapt your plan?”
From Interview Exercise to Real Life
Well, now the tables have turned on me, and guess what? I’m excited to announce that I just joined a startup company as their first security engineer! I now have the real world experience of this scenario, that I ran more than a hundred times with candidates. And even more exciting, our product will help companies that have limited or no security expertise to answer these questions. So stay tuned, more on that, is coming soon 🙂
