← Coding for Infra / 26

medium★ EssentialNewOpenAIAnthropicCoreWeave

Design the API for a GPU job scheduler in 45 minutes. What are the resources, states and semantics?

Four resources, one state machine and three semantics that separate a working API from one that corrupts state under retries. The idempotency key that makes a duplicate POST safe, why cancel returns 202 rather than 200, and the cursor that survives concurrent inserts.

Updated Sep 2026 · Grounded in real AI infrastructure interview loops and written to a senior-engineer editorial bar, with every number worked and every diagram hand-built.

Four resources, one state machine and three semantics that separate a working API from one that corrupts state under retries. The idempotency key that makes a duplicate POST safe, why cancel returns 202 rather than 200, and the cursor that survives concurrent inserts.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 283 remaining answers · ₹2,000 / $25

The concepts behind this question

Ranked by how closely each one overlaps this question's topic, so the first card is the thing to read if the answer above moved too fast.

📐 AI Systems Design

Control Plane and API Design for GPU PlatformsEvery GPU platform has a control plane, and its API is what the rest of the organization experiences as the platform. Three semantics decide whether it survives contact with a network: idempotent creation so a retried request does not launch a second job on sixty-four GPUs, cancellation modelled as intent because only the node agent can stop a running process, and cursor pagination that does not skip rows when work is created during a listing.

💻 Coding for Infra🔒 Premium

Producer-Consumer PipelinesA data loader, a log shipper, a batch inference job and a checkpoint writer are the same program: stages connected by bounded buffers, each running at its own pace, the slowest setting the throughput and the buffers absorbing the jitter between them. The coding screen asks you to build one (read, decode, batch, feed a consumer) and then pushes on the production questions: buffer sizes, clean stops, failure propagation, and why it runs at a third of the expected speed. This page derives throughput from stage times, implements the pipeline in threads and asyncio, and works the stop and failure semantics.

💻 Coding for Infra

The Practical Coding Screen PlaybookThe AI infrastructure coding screen is 45 to 60 minutes of building a small, realistic piece of systems code (a scheduler, a rate limiter, a batcher, a log parser, a cache) in the language you choose, with an interviewer who extends the problem twice and watches how you handle it. It is not a puzzle round: the score comes from working code early, tests that name the invariants, complexity said out loud, and calm follow-ups. Some companies allow an AI assistant and some ban it, and each policy changes what is measured. This page gives the minute-by-minute plan, the habits that score, and the mistakes that end the screen.

💻 Coding for InfraSign in

Retry, Backoff and IdempotencyA retry is a second request that the system did not budget for, and a thousand clients retrying at the same moment is a second outage that the first one caused. The craft is small and specific: retry only what is safe to retry, wait an exponentially growing random interval so the retries spread out, cap the total retries with a budget, and make every retried operation idempotent so a duplicate does not double-charge or double-train. This page derives why synchronized retries double the load, works the jitter arithmetic, implements the client correctly, and covers idempotency keys for the operations an AI platform exposes.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackImplement dominant resource fairness for a cluster where jobs need different mixes of GPU, CPU and memory.Next in this trackWrite a tiled matrix multiply and explain the cache effect. Then say honestly what your benchmark actually measured.Next in this trackImplement a log-structured event store for fleet metrics with range queries and compaction.

FEDITOR'S NOTE

Scored on the idempotency key on create, on cancel being a request rather than a state, and on cursor pagination that does not skip or repeat rows.

DISCUSSION · 0

No comments yet — be the first to share your approach.