easyNewOpenAIAnthropic

Implement sampling from a model's logits with temperature, top-k and top-p. What are the numerical traps?

Four lines of arithmetic with three ways to get it wrong: dividing by a temperature of zero, exponentiating without subtracting the maximum, and applying the filters in an order that changes the result. The implementation, the verification against the analytic distribution, and the overflow that is one logit away.

Updated Sep 2026 · Grounded in real AI infrastructure interview loops and written to a senior-engineer editorial bar, with every number worked and every diagram hand-built.

more free answers with an account · no card

The concepts behind this question

Ranked by how closely each one overlaps this question's topic, so the first card is the thing to read if the answer above moved too fast.

Foundational

⚡ Kernels & Compilers

Kernel FusionAn elementwise or reduction kernel does a few FLOPs per byte and runs at HBM speed, so a chain of five of them costs five trips through HBM for work that needs one. Fusion collapses the chain into a single kernel that keeps intermediates in registers. It is the first lever for anything memory-bound, and knowing what it cannot fix (weight reads in decode, the GEMMs themselves) is what the interview is really testing.

Advanced

💻 Coding for Infra🔒 Premium

Batching Queues and BackpressureWrite a request batcher is the coding round's version of the serving engine's scheduler: requests arrive one at a time, the GPU wants them in groups, and the batcher decides when a group is full enough to send without holding anyone too long or accepting more than it can hold. The two knobs are the maximum batch size and the maximum wait, the invariant is a bounded queue, and the follow-ups (priorities, cost-aware batching, cancellation, bounded in-flight batches) are the ideas the real engines carry. This page implements the batcher in asyncio, derives what each knob buys, and walks the follow-ups.

Advanced

💻 Coding for Infra🔒 Premium

Interval Merging and Utilization LogsGiven busy intervals per GPU, when was the whole cluster idle? What was the utilization per hour from a log of start and stop events? Which jobs overlapped? These are the interval problems of the infrastructure coding screen, and they share one tool: sort the endpoints and sweep. The sweep line turns every variant into a single pass with a counter, the sort is the only thing that costs more than linear time, and the edge cases (touching intervals, zero-length events, an unterminated start) are where candidates lose the round. This page works the standard problem and its relatives with code, tests and the complexity derivation.

Foundational

💻 Coding for Infra

The GPU Credit Scheduler PatternThe most widely reported coding problem in AI infrastructure loops is a small scheduler: accounts hold credits, jobs arrive with a cost and a priority, and you must decide which jobs run, in what order, without letting any account overspend, then extend it under follow-ups (refunds, reservations, concurrency limits, fairness). It is not a trick question; it is a test of whether you can model state cleanly, pick the right data structures, keep invariants under mutation, and talk about complexity while typing. This page works the problem from the first line to the fourth follow-up, with the code, the invariants, and the derivations.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackCompute per-tenant GPU-hours for a billing period from a log of allocation start and stop events.Next in this trackImplement a circuit breaker in front of a model endpoint. What are the states, and what does the half-open state buy?Next in this trackGiven per-rank heartbeats from a training job, detect which rank has stopped and when. What produces false positives?

FEDITOR'S NOTE

Scored on the max subtraction for stability, on temperature zero handled as greedy rather than by division, on the order of temperature then top-k then softmax then top-p, and on verifying empirically against the analytic softmax.

DISCUSSION · 0

No comments yet — be the first to share your approach.