SRE interview questions: scenarios beat trivia
SRE interview questions built as scenarios beat trivia every time. Real prompts, what strong and weak answers look like, and how to score them fairly.
Most SRE interview questions are tests of vocabulary wearing a pager costume. "What's an SLO?" "Explain the difference between an SLI and an SLA." "What is toil?" A candidate can ace all of them by reading the first three chapters of the Google SRE book on the train to your office - and none of those answers tells you what happens when that person is alone with a degraded system and a timer.
The job is scenarios. The interview should be too. Here's a set of scenario prompts that map to actual on-call reality, with what strong and weak answers sound like, and a scoring approach that doesn't collapse into gut feel.
Scenario-based SRE interview questions that map to on-call
Each of these works as a conversation. No lab required - though we'll get to why a lab is the endgame.
1. "You're paged: p99 latency tripled, error rate is flat. Go."
Strong answer: starts by scoping - when did it start, what changed (deploys, config, traffic), is it one endpoint or everything, one region or all. Distinguishes latency-without-errors as a saturation smell: queues, thread pools, connection pools, GC, a slow downstream dependency. Names concrete places to look (dashboards, a slow-query log, upstream timeouts) and states what evidence would confirm each hypothesis before acting.
Weak answer: "I'd roll back the last deploy." Maybe! But as a reflex, with no check that a deploy even happened in the window, it's a superstition. Also weak: reciting the incident-command framework at length without ever saying what they'd actually look at.
2. "Load average is 40 on a 4-core box, but CPU is 20% idle and users say the app is slow."
This one filters people who've memorized "load average = CPU busy-ness."
Strong answer: knows Linux load average counts runnable and uninterruptible (D-state)
tasks, so high load with idle CPU points at I/O or a lock - reaches for vmstat, iostat,
and a look at process states in ps for the D column. If they've read something like our
high load average field guide, they'll rebuild that
reasoning live - that's fine, that's the point.
Weak answer: "Add more CPU." Scaling the resource that's demonstrably idle is the kind of fix that costs money and changes nothing.
3. "Your ingress is serving 502s for one service. Every other service is fine. Walk me through the layers."
Strong answer: traces the request path out loud - load balancer → ingress controller → Service → endpoints → pod - and identifies what a 502 means at each hop (usually: the upstream connection failed, so look at whether endpoints exist and whether the pod is actually listening on the targetPort). Proposes the discriminating test: curl the pod IP directly from inside the cluster. Our guide to ingress 502/503/504s walks the same path - the question tests whether the candidate carries that map in their head.
Weak answer: restarts the ingress controller "to see if it helps." Restarting a shared component to debug one service is also a blast-radius red flag.
4. "Mid-incident, your first hypothesis just got disproven. What now - literally, what do you do next?"
The most predictive question on this list, because incidents are mostly wrong hypotheses.
Strong answer: treats the disproof as information - it eliminated a branch. Goes back to the evidence, states the next-most-likely hypothesis, and mentions keeping notes so they don't re-test the same idea twice. Calm, ordered, unashamed of having been wrong.
Weak answer: vague ("I'd keep digging"), or the tell you should never ignore: they get defensive about the hypothetical being wrong. People who can't be wrong gracefully in an interview are worse at it at 3 AM.
5. "You can fix this outage in 30 seconds with a command you're not sure is safe. It's 3 AM. Nobody's watching."
Strong answer: wants to know the blast radius before running anything - what does the command touch, is it reversible, what's the rollback if it makes things worse. Weighs outage cost against risk-of-worse honestly; sometimes the fast risky fix is right, and a strong candidate can articulate when. Mentions writing down what they did for the postmortem, watched or not.
Weak answer: "Run it, it's an outage." Or the opposite failure: paralysis dressed up as prudence, escalating everything and owning nothing.
How to score scenario answers fairly
Trivia is easy to grade and worthless. Scenarios are valuable and easy to grade unfairly - unless you fix the rubric before the interview. Score each answer on four dimensions, with written anchors for what 1, 3, and 5 look like:
| Dimension | What you're listening for |
|---|---|
| Diagnosis quality | Evidence before hypotheses; discriminating tests, not vibes |
| Efficiency | Cheapest/most-likely checks first; no repeated dead ends |
| Blast radius | Awareness of what a fix could break; reversibility instinct |
| Verification | "Here's how I'd confirm it's actually fixed" - unprompted |
Note what's missing: "got the right answer." Correctness matters, but a candidate who reaches a wrong conclusion through sound process on ambiguous evidence is a better hire than one who guesses right. Grade the process - the same reason good incident reviews are blameless and focus on decisions, not outcomes.
Two logistics rules that keep this fair: ask every candidate the same scenarios, and write your anchors down before the first interview, not after the third.
The limit of talking about incidents
Here's the honest caveat about everything above: a verbal scenario grades the candidate's narration of debugging, not the debugging. The failure modes are real in both directions. Polished talkers describe a beautiful process they don't follow under pressure. Excellent quiet engineers - often your best hires - interview badly in a format that's ultimately a storytelling contest.
The upgrade path is a live work sample: a real broken system, a time box, and a recording.
Watching someone actually run kubectl describe, actually mis-guess, actually notice and
recover, is a different class of evidence than hearing them say they would. It's the same
argument we made when we said the take-home test is dead:
test the job, not the proxy. Verbal scenarios are the best proxy available in a
conference room. A broken cluster isn't a proxy at all.
FAQ
What are the most important SRE interview questions to ask?
Scenario prompts that mirror on-call: a latency spike with no errors, a high load average with idle CPU, a 502 from one service, and at least one judgment question about a risky fix. The differentiator isn't which failure you pick - it's that you grade the diagnostic process against written anchors instead of listening for keywords.
Should SRE interviews include coding questions?
Some scripting fluency matters - SREs automate things - but a full algorithms round measures the wrong skill. A short, practical task (parse a log, write a small tool) plus debugging scenarios covers the real job far better than dynamic programming does.
How do I evaluate SRE candidates without asking trivia?
Give them a situation, not a term to define. Then push for specifics: which command, what output would you expect, what would you do if that comes back clean? Depth of follow-up is what separates a rehearsed answer from a real method.
Are verbal scenarios enough, or do I need a hands-on assessment?
Verbal scenarios are a big step up from trivia and cost nothing to adopt. But they still reward articulate description over actual operation. For final rounds, a hands-on broken environment with consistent grading is the strongest signal available - and it's fairer to candidates whose skill outruns their storytelling.
Faultybox runs your candidates through a real broken cluster and shows you exactly how they fixed it - replay included. Free pilot in beta → join