← 💻 Coding for Infra
Core
Retry, Backoff and Idempotency
A retry is a second request that the system did not budget for, and a thousand clients retrying at the same moment is a second outage that the first one caused. The craft is small and specific: retry only what is safe to retry, wait an exponentially growing random interval so the retries spread out, cap the total retries with a budget, and make every retried operation idempotent so a duplicate does not double-charge or double-train. This page derives why synchronized retries double the load, works the jitter arithmetic, implements the client correctly, and covers idempotency keys for the operations an AI platform exposes.
a free account unlocks the core curriculum tier · no card
RELATED CONCEPTS
PRACTICE THIS IN REAL QUESTIONS
Coding for InfraImplement retry with exponential backoff and jitter. Why is the jitter the part that matters, and what must never be retried?→LLM Inference & ServingYou stream tokens to clients over HTTP. What can go wrong, and how does the server handle each case?→CUDA, Triton & Kernel EngineeringGiven the addresses each thread in a warp touched, classify the access pattern: coalesced, strided or random. Write the classifier.→Coding for InfraDesign the API for a GPU job scheduler in 45 minutes. What are the resources, states and semantics?→AI Infrastructure System DesignDesign token-level billing and metering for an LLM API: count at the engine, aggregate, reconcile, and survive the failure modes.→Coding for InfraPlace GPU jobs onto nodes. Compare first fit, best fit and worst fit, and say which one a training cluster wants.→