Kubernetes device plugins versus Dynamic Resource Allocation: what changes for GPU scheduling, and what would you adopt in 2026?

Device plugins count GPUs and know nothing else; Dynamic Resource Allocation lets a pod ask for devices by attribute and share them by claim. What the two models can and cannot express, what DRA changes for MIG, NVLink domains and multi-node gangs, and a migration stance for a fleet that runs both.

Updated Sep 2026 · Grounded in real AI infrastructure interview loops and written to a senior-engineer editorial bar, with every number worked and every diagram hand-built.

more free answers with an account · no card

The concepts behind this question

Ranked by how closely each one overlaps this question's topic, so the first card is the thing to read if the answer above moved too fast.

Foundational

🗂️ Scheduling & Orchestration

Kubernetes GPU SchedulingKubernetes knows nothing about GPUs until something tells it. The NVIDIA device plugin advertises each node's GPUs as a countable resource, the scheduler matches a pod's request to a node with enough of them, and the container runtime wires the device in. That model is enough for one job per GPU and breaks the moment you need sharing, topology or multi-node placement, which is where Dynamic Resource Allocation, the GPU Operator and the batch schedulers come in. Knowing which layer does what is the platform interview's opening question.

Core

🗂️ Scheduling & OrchestrationSign in

MIG, MPS and Time-SlicingA whole H100 is far more than a notebook, a small inference service or a CI job needs, and giving each of them a card leaves most of the fleet idle. Three mechanisms share a GPU, and they differ in what they isolate: MIG partitions the hardware into up to seven slices with their own memory and compute, MPS lets several processes share one GPU's SMs concurrently with no memory isolation, and time-slicing context-switches between processes with no isolation at all. The choice is the isolation the workload needs against the utilization the platform wants.

Advanced

🗂️ Scheduling & Orchestration🔒 Premium

Topology-Aware SchedulingTwo placements of the same 64-GPU job can differ by 2x in step time: one keeps every tensor-parallel group on a single NVSwitch node and every data-parallel ring on a single rail, the other scatters ranks across racks and pushes per-layer traffic through the spine. The scheduler is the only thing that can prevent the second placement, because the framework maps ranks to whatever GPUs it is handed. Topology-aware scheduling means the scheduler knows the hierarchy (NVLink domain, rail, rack, spine block) and places gangs to keep traffic low in it.

Core

🚀 Inference & ServingSign in

Continuous BatchingContinuous batching schedules at the granularity of a single decode step instead of a whole request, so a finished sequence's slot is refilled on the next iteration rather than when the longest request in the batch ends. It is the scheduling idea that turned LLM serving from a padded, half-idle GPU into one that stays full, and it decides how the engine's scheduler, memory manager and latency SLOs interact.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackWhat do you run on a GPU node before you let a job land on it, how long does it take, and what happens on failure?Next in this trackA GPU in a running 512-GPU training job is throwing errors. How do you get it out of the job without losing the run?Next in this trackExplain how a Slurm job actually gets its GPUs: GRES, partitions, cgroups, and the sbatch that lands 8 GPUs on one node.

FEDITOR'S NOTE

Scored on the counting-versus-claims distinction, on concrete things DRA expresses that a plugin cannot (attributes, sharing, structured parameters), and on an adoption stance that keeps the plugin for whole-GPU training pods while DRA earns trust.

DISCUSSION · 0

No comments yet — be the first to share your approach.