The cluster dashboard says 90% allocated and 30% utilized. What is happening, how do you prove it, and what policy fixes it?

Allocated means a scheduler handed the GPU out; utilized means it did work. A 60-point gap is jobs holding GPUs they do not use: idle notebooks, placeholder jobs, a loader-bound run at 20% tensor-active. The three metrics that separate the causes, the per-tenant table that names them, and the policies that close it.

Updated Sep 2026 · Grounded in real AI infrastructure interview loops and written to a senior-engineer editorial bar, with every number worked and every diagram hand-built.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 283 remaining answers · ₹2,000 / $25

The concepts behind this question

Ranked by how closely each one overlaps this question's topic, so the first card is the thing to read if the answer above moved too fast.

Foundational

🩺 Fleet Reliability & Observability

Alert Design and On-Call LoadAn alert exists to change what a human does, so any alert that fires without a decision attached is a false alarm regardless of whether its condition was true. GPU fleets generate a specific set of noisy signals that look serious and are not, and separating those from the ones that need a person at three in the morning is what keeps a rotation sustainable. The measure of an alerting system is the fraction of pages that led to an action.

Advanced

🗂️ Scheduling & Orchestration🔒 Premium

Multi-Tenancy, Quotas and Fair ShareA shared GPU pool is cheaper than ten private ones because ten teams' demand is smoother than one team's, and it only works if the sharing is enforced. Quotas say what each team is guaranteed, borrowing lets idle guarantees be used by others, fair share decides who waits when everyone wants more, and preemption reclaims borrowed capacity. This page works the arithmetic that makes pooling worth it, the layers of isolation a tenant needs, and the incentive problems (hoarding, gaming, the research-versus-product tension) that any policy has to survive.

Advanced

🗂️ Scheduling & Orchestration🔒 Premium

Gang Scheduling with Kueue and VolcanoA distributed training job is 64 pods that start together or not at all: if 40 are running and 24 are Pending, the 40 hold their GPUs idle at a collective barrier waiting for ranks that may never come, and two such jobs can deadlock a whole cluster. Gang scheduling makes the job the unit of admission. Kueue and Volcano add queues, quotas, priorities and preemption on top, which is what turns a pile of GPUs into a platform several teams can share without starving each other.

Core

🗂️ Scheduling & OrchestrationSign in

MIG, MPS and Time-SlicingA whole H100 is far more than a notebook, a small inference service or a CI job needs, and giving each of them a card leaves most of the fleet idle. Three mechanisms share a GPU, and they differ in what they isolate: MIG partitions the hardware into up to seven slices with their own memory and compute, MPS lets several processes share one GPU's SMs concurrently with no memory isolation, and time-slicing context-switches between processes with no isolation at all. The choice is the isolation the workload needs against the utilization the platform wants.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackWe have GPUs in two clouds. Can we run one training job across both, and what would you do instead?Next in this trackDesign a notebook platform for 300 researchers on 64 GPUs. How do you share, reclaim and account for the GPUs?Next in this trackHow do you give a Kubernetes pod RDMA access to the fabric for NCCL, and what goes wrong when it is done naively?

FEDITOR'S NOTE

Scored on distinguishing allocated, active and useful with the right DCGM fields, on attributing the gap per tenant and per job class before proposing policy, and on policies that change incentives rather than lecture users.

DISCUSSION · 0

No comments yet — be the first to share your approach.