TL;DR: MIG carves an A100 or H100 into up to seven hardware partitions with their own memory, cache and SMs, so tenants get memory and fault isolation and predictable performance, at the cost of profile waste (a 3g.40gb plus a 4g.40gb uses all 80 GB but two 3g.40gb leave one seventh of the compute unusable). Time-slicing runs several containers on one whole GPU in turns, with no memory isolation and a context switch each turn, so it suits bursty notebooks and nothing latency-bound. MPS lets processes run kernels concurrently on shared SMs with one address space, which is the highest utilization and the least isolation, so it stays inside one tenant. The tenant rule: across trust boundaries, MIG or whole GPUs, never MPS or time-slicing.
How to approach it
Ask who the sharers are: one team's processes, or different tenants. Then describe each mode by what it isolates (memory, faults, performance) rather than by its name. Do the MIG slice arithmetic for the interviewer's example so waste is visible, then say which mode you would pick for the stated workload and the condition that changes it. The MIG, MPS and Time-Slicing concept page carries the profile tables; the answer spends its words on the decision.
A strong answer
A typical situation: a platform team has 32 H100s serving small models that each use 12 GB and 15% of the GPU, and the finance report shows 85% of the fleet's compute idle. Sharing is the fix; the question is which kind.
MIG (Multi-Instance GPU) is a hardware partition. The GPU's memory, L2 slices and SMs are divided into instances, each of which appears to the driver as a separate device with its own memory bus and fault domain. A crash or an out-of-memory in one instance does not touch another, and a neighbor's kernels cannot slow yours because they run on different SMs. The profiles are fixed shapes:
H100 80 GB MIG profiles (compute fraction in sevenths, memory in eighths of 80 GB)
1g.10gb = 1/7 compute, 10 GB (up to 7 per GPU)
2g.20gb = 2/7 compute, 20 GB (up to 3)
3g.40gb = 3/7 compute, 40 GB (up to 2)
4g.40gb = 4/7 compute, 40 GB (1, combinable with a 3g.40gb)
7g.80gb = the whole GPU
The waste comes from the shapes. Work the serving case:
demand per model: 12 GB memory, ~15% of an H100's compute
smallest profile that fits 12 GB: 2g.20gb (a 1g.10gb is 2 GB short)
per GPU: 3 × 2g.20gb = 6/7 of compute, 60 GB of memory carved; 1/7 compute and 20 GB unused
models per GPU: 3, each with 2/7 = 29% of compute against a 15% need
fleet: 32 GPUs × 3 = 96 models, compute idle ≈ 1 − (96 × 15%) ÷ (32 × 100%) = 55%
sanity: 55% idle beats 85% idle, but the 2 GB overshoot on memory cost a whole profile step;
a 10 GB model would have fit 7 per GPU by memory, but 7 × 15% = 105% of the compute,
so six per GPU at 90% busy is the honest fit and the seventh slice stays empty
The profile is chosen once per GPU (mixed profiles need a reconfiguration, which empties the GPU), so a fleet of MIG GPUs is a fleet of fixed shapes and the scheduler bin-packs jobs into them.
Time-slicing runs several containers on one whole GPU with the driver switching between their contexts. Each container sees the full 80 GB and can use it, so there is no memory isolation: one container allocating 70 GB starves the rest, and one crash can reset the device for all. Performance is shared in turns, so a latency-sensitive model next to a busy one sees its p99 move with the neighbor's load. It is the right tool for notebooks and development pods that are idle most of the time, where the alternative is a dedicated GPU at 5% use.
time-slicing, four notebooks on one GPU, each active 10% of the time
expected simultaneous demand = 4 × 0.10 = 0.4 of a GPU; contention is rare
when two are active, each gets ~50% and a context switch per time slice
memory: no limit, so one 60 GB allocation leaves 20 GB for three others
sanity: fine for four people editing code; wrong for one serving container with an SLO
MPS (Multi-Process Service) is different in kind. The processes share one CUDA context and their kernels run concurrently on the same SMs, so a GPU that a single process leaves half idle can be filled by a second. Memory limits per client exist (CUDA_MPS_PINNED_DEVICE_MEM_LIMIT), but the address space is shared, and a fatal fault in one client takes down the server and all clients. That makes MPS a tool for one team's own processes (several inference replicas of the same model, an MPI job with several ranks per GPU), never for two tenants.
| Mode | Memory isolation | Fault isolation | Performance isolation | Best for |
|---|---|---|---|---|
| MIG | yes, hardware | yes | yes | multi-tenant serving of small models; anything with an SLO on a shared card |
| time-slicing | none | none | none, turns | notebooks and dev pods, low duty cycle |
| MPS | soft limits | none | none, concurrent | one tenant's many processes filling a GPU |
The decision for the serving fleet above: MIG at 2g.20gb, because the models have SLOs and belong to different internal customers. The condition that reverses it is a fleet of one team's identical replicas, where MPS packs more of them per GPU than MIG's seventh-granularity allows and the shared fault domain is that team's own risk.
The reversal condition: a single team sharing one node, where a noisy neighbour is a conversation rather than a contract breach. MPS then recovers capacity that MIG's fixed slices would strand, and the isolation given up was isolation from yourself. MIG, MPS and Time-Slicing has the three modes side by side. Capacity Planning and Utilization is where the stranded capacity gets counted.
What interviewers probe next
- "Can you mix a 3g.40gb and a 4g.40gb on one H100?" Yes; that pairing uses all 80 GB and all seven compute slices, which is why it is the common two-tenant layout.
- "Does MIG support NVLink between instances?" No; a MIG instance is single-GPU by construction, so tensor parallelism across slices is out.
- "Why not time-slice serving replicas to save GPUs?" No memory isolation and shared turns: one replica's warmup allocation can OOM the others, and the p99 depends on neighbors.
- "How does the scheduler see a MIG slice?" As its own extended resource (
nvidia.com/mig-2g.20gb) with the device plugin, or as a device with a profile attribute under Dynamic Resource Allocation.
Common mistakes
- Presenting time-slicing as "MIG for older GPUs." It isolates nothing; the resemblance is only that several pods land on one GPU.
- Choosing a MIG profile by memory alone and then discovering the compute fraction is too small, or the reverse.
- Running MPS across tenants because it gave the best utilization in a test with friendly workloads.
- Forgetting that changing MIG profiles drains the GPU, so a "flexible" MIG fleet is a fleet of scheduled reconfigurations.
Key takeaways
- MIG: hardware partitions, up to seven per GPU, memory and fault isolation, fixed shapes that waste compute when demand does not match a profile.
- Time-slicing: whole GPU shared in turns, no isolation, right for idle-mostly dev pods.
- MPS: concurrent kernels in one address space, best fill and least isolation, one tenant only.
- Across tenants: MIG or whole GPUs. The 3g.40gb plus 4g.40gb pair is the way to use an H100 fully with two tenants.