xAI AI Infrastructure Engineer interview questions

Sixteen bytes per parameter says 1.6 TB of state against a 640 GB node, so the model spans nodes before the first token. The arithmetic that sizes the fleet, the layout that puts each axis on the right link, and the two numbers that decide FSDP against pipeline across nodes.

A loss that stops moving has five different causes and one of them is not a training problem at all. The checks in order: is the step counter moving, what are the learning rate and the grad norm doing, is the data repeating, has the loss scaler collapsed, and is one rank sending zeros.

Two restarts a day at fifteen minutes each is the 2%. The loss decomposed into detection, rescheduling, reload and rewound work, the lever on each, the in-memory checkpoint that makes the interval a minute, the square-root pareto of interval against write cost, and the residual that only fewer failures can remove.

A 400B dense model on 10 trillion tokens is 2.4 × 10²⁵ FLOPs, 43 days of pure compute on 16,384 H100s, and about 380 interruptions along the way. The order in which to derive every number, the layout and the data rate, the checkpoint and failure budgets, and the schedule that survives its own arithmetic.

The fleet equation solved for GPU count: about 6,200 H100s at 40% MFU, why it rounds up to a power of two, and the reasons a good answer adds 15% before naming a number.

Both carry RDMA at the same line rate. One arrives lossless because the transport was designed that way, the other becomes lossless only if a set of switch settings is correct on every port. What that difference costs in operations, what it saves in money and hiring, and the fleet size where the answer flips.

Thirty-two senders aimed at one port fill a switch buffer in microseconds, and what happens next depends on which of two mechanisms fires first. The buffer arithmetic, why the slow mechanism is the safe one and the fast one is the dangerous one, and the counters that tell you which is happening.

The number comes from one requirement: the gradient reduction has to finish inside the backward pass that produces it. Working that requirement backward gives a bandwidth per GPU, and the answer lands near the port speed the industry ships, which is not a coincidence.

The signature is unusual: many jobs degrade together, throughput collapses, and the drop counters stay at zero. Reading pause counters backward along the paths to find the origin, the three conditions that produce one, and the configuration change that stops it happening again.

Ten thousand GPUs is 1,280 nodes, ten pods, about 700 switches, a filesystem that swallows a 5 TB checkpoint in under a minute and a 14 MW feed. The bill of materials with a reason on every line, from the FLOPs the model needs to the spare pool the failure rate demands.

The first question is whether 1T is total or active, because storage follows one and compute follows the other. The state budget, the layout that falls out of it, why expert parallelism belongs inside NVLink, and the two failure modes a dense-model plan does not have: router imbalance and all-to-all congestion.

The answer that fails proposes changes in week one. The answer that lands reads the postmortems first, draws the map from the job's point of view, ships one small fix, and ends the month with a list of questions nobody on the team could answer.

One forward pass reads every weight. Whether that read is the bottleneck depends on how many tokens ride along with it, and the answer is different for the two halves of a request.

Every token a model has seen leaves a key and a value in every layer. Multiply that out for a 70B model and you will see why memory, not compute, caps how many users a GPU can hold.

A batch of eight requests finishes when the longest one does, and seven slots compute padding until then. The fix is to make the scheduling unit one decode step, and the throughput math shows why that was worth several times the hardware.

Before vLLM, engines reserved the maximum context for every request up front and wasted most of it. The fix borrowed virtual memory from the operating system, and the numbers show how much capacity it returned.

Tokens per second flatters a system that is failing its users. The metrics that matter are two latencies at a percentile and the fraction of requests that meet both.

Weights take a fixed slice, the KV cache takes the rest, and the rest divided by context is your user count. Work it in bf16 and fp8, then find out why the answer also depends on how you split the model.

One 20k-token prompt can freeze every active stream for most of a second. The fix slices it into per-step budgets, and the budget number is a trade between two SLOs you can compute.

A small model guesses four tokens and the big one checks them in a single step. That converts idle bandwidth into tokens, and the arithmetic tells you exactly which batch size stops it working.