AI Infra Interviews logo
Free · No account needed · About 15 minutes

How ready are you for an AI infra interview?

17 scenarios from real AI infrastructure engineer loops. You will find out which of the eleven areas these interviews test you have actually worked with, which gap is costing you most, and exactly what to do about it. Every answer tells you what that choice signals to an interviewer, so the fifteen minutes are worth it even if you stop halfway.

About you0%

Which role are you actually going for?

The titles overlap, and the loops behind them do not weight the same rounds.

The eleven areas an AI infra loop tests

AI infrastructure engineer interviews do not test what a standard software loop tests. They weight the rounds below, and they weight them differently depending on whether you are aiming at a frontier lab, a chip maker, a GPU cloud or an ML platform team. The assessment above works out which of these you have covered.

Napkin math & capacitydecides loops
The estimation round almost every AI infra loop includes. Memory per parameter, bytes per token of KV cache, 6ND and MFU decide whether the design you draw next is even physically possible, and interviewers grade the assumptions you say out loud.
AI infra system designdecides loops
The whiteboard round that ties the rest together. An inference platform, a training cluster, a scheduler with preemption: the grade comes from the questions you ask before drawing and the numbers you attach to each box.
Distributed trainingdecides loops
Owning a run at cluster scale. Which parallelism, why that layout on this fabric, what the checkpoint cadence costs, and what you do at 3am when step time doubles. The centre of every frontier lab training loop.
Inference & servingdecides loops
Hosting open-weight models at a latency SLO and a cost you can defend. Prefill against decode, KV cache occupancy and continuous batching are the levers, and most loops now have a round built on them.
CUDA & kernel engineeringdecides loops
The live-coding and take-home round at chip makers and performance teams. Profiler first, then fusion, tiling and pipelining against the stall reason it names. Rarely asked elsewhere, decisive where it is.
GPU architectureexpected
The vocabulary every other round assumes. You are rarely asked to derive anything, but being unable to place a kernel on the roofline or say what FP8 costs in quality loses credibility in every technical conversation.
Open weights & enginesexpected
Standing up a model the team read about this morning. Sizing it from config.json, knowing whether an engine supports its attention and quantization, and configuring vLLM or SGLang against a latency target rather than against defaults.
Hardware & build-outexpected
The physical layer every other round assumes exists. Which accelerator for which workload, what an NVLink domain buys, what the cables and optics cost in power, and what a rack needs in kilowatts and litres per minute before anything runs.
Behavioral & ownershipexpected
Graded in every loop whether or not it has its own stage. Pushing back on a launch with data, the on-call story, and working with researchers who do not report to you decide between two technically equal candidates.
Coding for infraexpected
The screen that happens before anyone sees your infrastructure judgment. A rate limiter, a batching queue, merging idle intervals from a fleet log: graded on edge cases and tests, not on speed.
Networking & storageexpected
The fabric and the disks that decide whether ten thousand GPUs act like one. Weighted heavily at GPU clouds and hyperscalers, and it is where a training slowdown usually turns out to live.
Fleet reliabilityexpected
The on-call reality most prep skips. DCGM, the XID taxonomy, stragglers, health checks before a job starts. The platform and neocloud loops are built on it, and every lab wants to hear one real incident.
Kubernetes & GPU schedulingsituational
Matters enormously at GPU clouds and ML platform teams, and barely at all in a kernel loop. Gang scheduling, MIG against time-slicing, quotas and topology-aware placement. Worth knowing which loop you are walking into.

Profiles this tends to find

The result names the pattern in your answers rather than ranking you. These are the shapes that come up most, and each one has a different fastest path.

  • The Kernel Engineer Without a Fleet

    You read a profile the way most people read prose, and your answers on the memory hierarchy and the roofline are the answers of someone who has made a kernel faster. Then the fleet questions land differently: what a node does when a GPU falls off the bus, what the scheduler needs, what you look at first when a rank stalls.

  • The Platform Engineer Meeting the Accelerator

    You run systems. Scheduling, on-call, capacity and the incident craft are already in your answers, and they transfer directly.

  • The Researcher Who Runs Jobs

    You have trained at scale and you know the parallelism trade-offs from the inside. What the infra loop tests, and what your answers do not yet show, is the side of the run that the researcher does not own: the estimate you make before the cluster exists, the scheduler and the fleet under the job, and the practical build in 45 minutes.

  • The Serving Engineer Who Has Not Trained

    Your serving answers are strong and specific: the KV cache, batching, the design round with numbers. Distributed training and the fabric under it are where you go general.

  • The One Who Does the Math First

    You reached for the numbers and the assumptions without being prompted, which is the strongest single signal in an estimation-heavy loop and rarer than it should be. Your remaining risk is depth: this profile often carries the design round on arithmetic and then meets a specialist round (a kernel, a fabric, an incident) where arithmetic is not enough..

  • The Strong Generalist

    There is no obvious hole here, which is a good position and a slightly awkward one to prepare from: with no glaring gap, most people default to revising what they already know. Your risk is arriving broadly competent and not distinctly strong in the rounds that decide the offer..

Where the plan sends you

Whatever your result, it is built from the same library. You can also just start reading.

You could ask a model instead. Here is the difference.

Genuinely, ask one. It will give you a good general answer about AI infrastructure engineer interviews, and if you are early it may be all you need. Four things it will not do.

It will agree with you

Say you would add a reranker and a model will tell you that is a solid instinct. Here it is scored as premature, with the reason: reranking helps when the right passage comes back at rank eight and does nothing when it never comes back at all. Every option in this assessment was weighted by what it signals in a real loop, which means it is built to disagree with you when disagreeing is the useful thing.

It answers the question you asked

You cannot ask about the gap you do not know you have, and the gaps that fail people in these loops are almost always the ones they never thought to raise. This works the other way round: it probes all eleven areas whether or not you would have brought them up, which is the entire point of a diagnostic.

It can describe a plan, not give you one

A model can tell you to study retrieval evaluation. It cannot hand you the concept, the four questions that test it and the lesson that covers it, because it does not have them. Your plan resolves to real material that exists here and is maintained: 413 questions with worked answers, 150 concepts and 55 course lessons.

It does not remember

Come back in six weeks and a chat is gone. Your result is a stable artifact: the same diagnosis, rebuilt from your answers, with links that still point at content that exists. Sign in and it is waiting for you.

None of this is a claim that models are bad at explaining things. They are very good at it. It is a claim that explaining and diagnosing are different jobs, and that the second one needs a fixed instrument, calibrated answers and a library behind it.

Why this measures coverage and not ability

A multiple-choice question cannot tell you whether someone can run a discovery conversation, and pretending otherwise would be the fastest way to lose the engineers this is built for. So nothing here produces a score out of a hundred. It reports which areas your answers show you have worked with, which is something a fixed set of scenarios can honestly support, and it is the more useful half anyway: you can act on a gap, and there is nothing to do with a number.