NVIDIA AI Infrastructure Engineer interview questions

A grid is a request, a block is a residency unit, a warp is what the scheduler issues. Which of those pins to an SM, why 32 matters, and how a GPU hides a 600 ns memory latency with no branch predictor and a cache that is tiny per thread.

Registers, shared memory, L1, L2, HBM, host memory: sizes, bandwidths and latencies for an H100, derived rather than recited, and the habit of asking 'which level am I hitting' before touching a kernel.

Not 'more cores.' The real answers are a 10x memory system, a 16x matrix datapath, and a design that spends transistors on lanes and registers instead of on making one thread wait less. With the numbers for a two-socket server against one H100.

A tensor core multiplies small matrix tiles in one instruction and is 16x faster than the regular lanes, but only for dense matmul at the right precision, with the right shapes and layouts. What the instruction looks like, what it refuses, and how to tell from a profile whether you are on it.

Peak FLOPS divided by bandwidth is one number per chip, and it decides whether any kernel can ever reach peak. How to compute it, how to compute a kernel's intensity from its bytes and FLOPs, and how to read the answer before writing a line of CUDA.

Every format is a trade between how big a number can be and how finely it is spaced. The bit layouts, the largest and smallest values derived from them, why fp16 needed loss scaling and bf16 did not, and why fp8 comes in two flavors with a scale factor attached.

The H200 has the same compute die as the H100 and costs more per hour. The datasheet gives two ratios, 1.43x bandwidth and 1.76x memory, and those two numbers decide exactly which workloads pay back the premium and which ones lose money on it.

An H100 SXM moves 900 GB/s over NVLink; a PCIe card moves about 64 GB/s per direction over its slot. Whether that 7x to 14x gap costs you anything depends on how many bytes your parallelism strategy sends per step, and the answer is very different for tensor and data parallelism.

A GPU issues one instruction to 32 threads at once. When those threads want different instructions, the hardware runs each path in turn with the others masked off, and the time is the sum of the paths. The derivation of the cost, the metric that exposes it, and the three fixes that work.

Occupancy is a means, and the end is enough bytes in flight to cover memory latency. Little's law gives the number of loads an SM needs outstanding; the register file gives the warps you can afford; the kernels that win on H100 usually run at 25% to 50% occupancy with big tiles and no spills.

HBM stacks DRAM dies on top of each other and wires them to the GPU through a silicon interposer with a 1,024-bit bus per stack. That design sets how much bandwidth and capacity a card can have, why the two scale together, and why they have grown more slowly than FLOPS across three generations.

Three things change at once: dense compute more than doubles with an fp4 tensor core, HBM3e reaches 8 TB/s, and NVLink stops at 72 GPUs instead of 8. Work through what each does to tensor-parallel degree, per-step weight reads and expert placement for a 671B-parameter mixture of experts.

MIG carves an H100 into up to seven hardware slices with their own SMs, memory and bandwidth; MPS lets processes share one SM pool through a single context. One gives fault and performance isolation at a fixed slice size; the other gives flexibility and a shared failure domain. The numbers that decide it.

Four bits give eight magnitudes per sign, and the largest step between them is 50%. Block scaling rescues most of the tensor; a single outlier in a block of sixteen flushes its neighbors to zero. Where the precision goes, which layers fail first, and the measurements that separate acceptable from broken.

512 GPUs at 400 Gb/s each is 205 Tb/s of injection bandwidth a two-tier fabric must carry with no oversubscribed link. The port arithmetic that gives 16 leaves and 8 spines, the rail wiring that keeps NCCL traffic one hop away, and the line where 900 GB/s of NVLink becomes 50 GB/s of InfiniBand.

Without it, every byte a GPU sends over the network crosses PCIe twice and passes through host memory. With it, the NIC reads and writes GPU memory directly. The PCIe arithmetic that shows why a 400G NIC cannot be fed the slow way, the NCCL line that tells you which path you are on, and the checkpoint case.

Four percentages from a profiler summary are enough to place a kernel on the roofline, estimate its arithmetic intensity, and rule out half the optimizations people reach for. The reading, the arithmetic that backs it, and the fix list in the order it pays.

An H100 has 50 MB of L2, split into two partitions, with a bandwidth several times HBM's. That is enough to change the roofline for a kernel whose working set fits and to do nothing at all for the ones that stream. The arithmetic of what fits, the residency controls, and the two ways the L2 lies to a profile.

The workload asks for bytes and the chip delivers FLOPs: bytes per FLOP halved from A100 to H100 and held level on B200. A defensible thesis is more bandwidth and capacity per FLOP, the shoreline and power arithmetic for why it has not happened, and the counterargument that software already routes around the wall.

Half the expected bus bandwidth has four plausible causes needing different fixes: a link trained at reduced width, a fabric that never came up, a topology the job did not expect, or a test that measured latency. The sequence of commands that tells them apart, and what each output looks like on a healthy node.

Decode is bandwidth-bound, so the FP8 FLOPS number that dominates the marketing does not move it. Where the B300 pays is capacity, and capacity converts into throughput through batch size rather than directly. The arithmetic that decides it, and the case where the B200 wins.

Links that never come up and links that come up and flap are different faults with different causes. The reach change between 400G and 800G explains most first-time failures, and the counters separate the rest. The order to check, and the one physical measurement that settles it.

A rail-optimized fabric makes this fault findable because the eight rails are identical by construction, so any asymmetry is a defect. Bisecting by rail, then by switch, then by node, then by port narrows 512 links to one in four steps, and the counters say which of three physical causes it is.

Each layer depends on the ones below it and a mistake at the bottom shows up six layers higher as something else entirely. The order, the dependency almost everyone gets wrong on Blackwell systems, and the validation ladder that keeps a fault attributable to one node instead of five hundred.

The part with no NVLink can be the correct choice, and the reason is the workload shape rather than the specification. What fits on one card, what MIG partitioning buys a multi-tenant service, and the exact point where the missing scale-up link makes the decision flip.

The count follows from three inputs and one topology rule, and getting it wrong by the leaf-to-spine half is the classic error. The arithmetic, the radix check that says two tiers is enough, and the power the transceivers add that nobody put in the budget.

Single-node fine and two-node halved is a specific signature, and it points at one thing before anything else. Why the copy path costs exactly about half, the five causes that produce it, and the one log line that confirms which.

Most architecture diagrams show one network and a cluster has four. Each has a different traffic pattern, a different failure consequence and a different port count, and the one that is under-counted every time is the one you need when everything else is broken.

Switch radix decides it, and at 10,000 endpoints the answer sits right at the boundary for current high-radix switches. The arithmetic, what a third tier costs in switches, optics and power, and the two designs that avoid it entirely.

Same GPU count, same silicon, and a completely different machine. The domain size is the whole argument, it is worth an order of magnitude on specific traffic, and it costs a failure domain nine times larger plus a facility that most halls do not have.

It configures the NVSwitch fabric so eight GPUs present as one NVLink domain, and when it is absent nothing errors. Jobs run, results are correct, and everything multi-GPU is slower by a factor the team spends weeks not finding. The check that takes one command.

The rule is one sentence and the execution is thousands of connections where a single transposition creates a fault that is invisible until a collective runs. The mapping, the labelling scheme that makes errors findable, and the verification that has to happen before anyone accepts the cluster.

Modules do not fail randomly at three times a published rate, so the excess has a cause and it is usually thermal or a batch. The cohort cuts that localize it, the counters that separate a dying module from a dirty connector, and the environmental cause that looks like a hardware defect.

The kernel is four lines; the interview is about the other forty. How a global index comes out of block and thread ids, why the bounds check exists, what the launch configuration means for a 100M-element array, and the arithmetic that shows the host-to-device copies cost 50 times more than the add.

A warp issues one load instruction and the memory system turns it into some number of 32-byte sector requests; that number is the whole story. The arithmetic for contiguous, stride-2 and stride-32 access, the row-major matrix where a loop order change gives 8x, and the two Nsight Compute counters that show the waste.

Thirty-two addresses per warp instruction, thousands of instructions: say what the pattern is and how many sectors it cost. The address-delta test for contiguous and strided, the sector count that measures the damage, the code that does both, and the edge cases (misalignment, inactive lanes, mixed widths).

Shared memory has 32 banks, each 4 bytes wide, and a warp's access is as slow as the most-loaded bank. The bank of an address, why a column walk down a 32-wide tile puts all 32 lanes in one bank, the padding by one column that spreads them across all 32, and the profiler counter that confirms the fix.

The naive kernel reads two bytes per multiply-add; a 128x128 tile with register blocking reads a few hundredths of that. Each step (shared-memory tiles, register blocking, vectorized loads, double buffering, tensor cores) with its intensity arithmetic and speedup, and where a hand kernel stops and cuBLAS begins.

Occupancy is how many warps an SM holds against its maximum, and it is a means, not an end. The arithmetic from registers per thread to resident warps, the launch bound that caps registers and the spills that follow, the profiler's occupancy view, and the kernel where halving occupancy doubled speed.

Each version fixed a different bottleneck: version 1 fixed memory traffic, version 2 fixed non-matmul work and parallelism, version 3 fixed the fact that softmax and matmul were waiting for each other on Hopper. The arithmetic that shows why a softmax costing one percent of the FLOPs can cost sixty percent of the time.

Four sections, read in a fixed order, and one number that usually names the bottleneck outright. The report of a kernel at 12 percent of DRAM bandwidth while its memory pipeline reads 82 percent busy, what that gap means, the fix it implies, and the numbers the fixed kernel reports back.

A sum reads every byte once and does one add per element, so the only question is whether you reach the bandwidth ceiling. The ceiling in milliseconds, four versions from an atomic per element to warp shuffles with sixteen loads in flight, and the bytes-in-flight math behind the last jump.

A transpose moves every byte exactly once, so a device copy is the honest ceiling. Why the naive kernel pays eight times the write traffic, why staging through shared memory does nothing until you pad, and the counter that tells the two problems apart.

At batch 1 a decode step is a few milliseconds of GPU work behind several hundred kernel launches, and the CPU cannot issue them fast enough. The arithmetic showing the GPU starve, what replay changes, and the four constraints capture imposes on memory.

A library call gives you a tuned matmul and nothing around it. Four things a template library gives instead: an epilogue fused into the output stage, one launch for a list of ragged problems, a matmul whose inputs are different types, and a tile schedule that fills the SMs on awkward shapes.

Chunk the work, put copies and kernels on different streams, and the total stops being copy time plus compute time. The pipeline arithmetic that says how many chunks are enough, the three conditions without which the calls run in sequence anyway, and the two levels of asynchrony that are often confused.

A float format's resolution is relative, so the damage depends on how large a running total grows against the terms being added. The measured error for a 4,096-term dot product in each direction, the weight update that vanishes entirely, and why bf16 removed loss scaling but not master weights.

The matmul runs on tiny formats and the accumulator does not, so every low-precision GEMM is really a scaled integer or float product with a correction applied on the way out. Where the scales come from, the three granularities and what each costs, and why one of them needs a periodic promotion into fp32.

A row of logits is half a megabyte, so the kernel is not bandwidth-bound and the cost is in how many passes you make over it. Why a full sort does far more work than the question asks, the two shapes that fit small and large k, and the four-pass radix select with a reference that matches a sort exactly.

Registers are allocated in fixed steps out of a fixed budget per multiprocessor, so a small increase in live values can cost a whole resident block. The compiler output that shows it in two lines, the occupancy cliff arithmetic, and the two different failure modes that produce the same symptom.

Three ways to split a training job, one table, and the rule that places each of them: activations on NVLink, gradients on the fabric, stage boundaries in between. With the byte counts that justify the placement.

Two phases, n minus one steps each, one slice per step: the derivation that gives 2(n−1)/n times the buffer per rank, the argument that no algorithm can beat it on bytes, and the latency term that makes NCCL switch to trees at scale.

With p stages and m micro-batches the pipeline idles (p−1)/m of its compute time, and the obvious fix, more micro-batches, buys back throughput with activation memory. 1F1B caps that memory at p micro-batches instead of m. The derivation, the numbers for PP16, and the schedules that go further.

Four all-reduces per transformer block, on activations, on the critical path. Inside the NVLink domain they cost 8% of a step; across the NIC they cost more than the step itself. The derivation for a 70B, and the second reason TP stops that has nothing to do with the network.

Tensor parallel 8, pipeline 16, data parallel 128: the layout Meta trained Llama 3 405B on, rebuilt from the memory floor, the compute budget and the bandwidth of each link. Why the axes go in that order, what each one costs per step, and the bubble that the schedule has to hide.

fp16 loses gradients below 6e-8 and overflows above 65,504; bf16 has fp32's range and three digits of precision. The arithmetic that shows why an update of 3e-5 vanishes into a bf16 weight, where each of the 16 bytes per parameter comes from, and what fp8 block scaling adds on top.

Bucketed all-reduce in backward, FSDP's one-layer-ahead prefetch and split-GEMM tensor parallelism hide most of the bytes, and none of it works below a certain number of tokens per GPU. The ratio that decides it, the SMs the NCCL kernels take, and the tail that is never hidden.

Generation is a decode workload and the update is a training workload, and they want different software on different GPUs. The per-token arithmetic for each side, the 141 GB broadcast that has to happen every step, the ratio of actor to learner GPUs that follows, and the idle time that on-policy training builds in.

Sixteen samples per prompt, a verifier that runs code, no value model, responses to 16k tokens: GRPO moves the cost from the learner to rollouts and rewards. The token arithmetic for one step, the tail that holds a batch for the longest sample, and the three places a fleet idles.

Three mantissa bits and a range of 448 make fp8 unusable with one scale per tensor. The arithmetic of what an outlier channel destroys, the 128-element block scaling and fp32 promotion that the DeepSeek-V3 report used, the ops that stayed in bf16, and what the 2× peak bought in practice.

Prefill and decode fight over the same GPU and each ruins the other's latency. Putting them on separate pools ends the fight, at the price of shipping every request's cache across the network. The break-even is a number you can derive.

Three engines, one hardware roofline, and the difference between them is which part of the roofline each one reaches first on your workload. The decision is a table, and the table has reversal conditions.

A paused conversation holds gigabytes of cache that its next turn will need in thirty seconds. Recompute costs prefill; reload costs bandwidth. The comparison is one division per tier, and the answer flips with prompt length.

The bound assumes weights stream contiguously and nothing else costs time, and at batch one both assumptions fail badly. Four terms that make up the gap, why sparse models suffer most, and the two fixes that recover most of it.

Halving the weights halves the GPU count and moves the quality evaluation onto you. What the format actually buys in memory and in speed, why those are different questions, and the evaluation that has to run before it ships.

The conversion is the easy part and the evaluation is the project. What calibration data does and why yours should come from production, the per-category evaluation that catches what an average hides, and the parts of the model that should not be quantized at all.

Graphs remove the per-kernel launch cost, which is a large share of a decode step at low batch and a small one at high batch, so whether it matters is a question about your operating point. The arithmetic, the four reasons an engine turns them off, and the measurement that settles it.

Two divisions decide it: the kernel's FLOPs per byte against the card's ridge point. The worked case, the profiler counters that give the inputs, and what to change once you know which wall you hit.

The ring all-reduce formula, 2(n−1)/n times the buffer over the link bandwidth, applied to 141 GB of bf16 gradients on an NVLink node: 0.27 s by the formula, and the two reasons the measured number is different.

8 TB/s over the bytes per step: 57 tokens per second single-stream in bf16, 113 in fp8, 227 in fp4, and the batch curve on a single card that now holds the whole model. Where the curve bends and what caps it.

Activations are the term the 16-bytes-per-parameter rule leaves out, and at 8k context they are bigger than the weights: about 34 × sequence × hidden bytes per layer, 2.3 GB, 182 GB per micro-batch across 80 layers. The chain, what FlashAttention removes, and what checkpointing does to the number and the compute.

Before the first byte moves, the library has already discovered the machine's topology, built rings and trees over it, split the work into parallel channels and chosen a wire protocol by message size. What each of those four decisions is for, and the environment variables that let you see and override them.

Four operations cover almost everything a distributed training step sends. What each one does to the data, the bytes each rank moves for a message of size S across N ranks, and which parallelism strategy generates which, worked through for a 70B model so the numbers are concrete rather than symbolic.

Both carry RDMA at the same line rate. One arrives lossless because the transport was designed that way, the other becomes lossless only if a set of switch settings is correct on every port. What that difference costs in operations, what it saves in money and hiring, and the fleet size where the answer flips.

A single core moves a few gigabits per second of TCP once copies and interrupts are counted, so filling one 400 gigabit link would take most of a server's cores doing nothing but networking. What RDMA removes, what a queue pair actually is, and where the GPU fits when the data never belongs to the host at all.

Wire GPU number three of every node to the same switch and the collective that matters most never leaves it. The rank mapping that makes the layout pay, the switch and cable count for a thousand GPUs worked out, and what happens to a job whose ranks are assigned in the wrong order.

Thirty-two senders aimed at one port fill a switch buffer in microseconds, and what happens next depends on which of two mechanisms fires first. The buffer arithmetic, why the slow mechanism is the safe one and the fast one is the dangerous one, and the counters that tell you which is happening.

Reading straight from NVMe into GPU memory skips a copy through host memory, which is worth a lot for one workload and nothing for another. The bandwidth arithmetic for both paths, the two cases where the bounce is the bottleneck, and the far more common case where the CPU has to touch the data anyway.

Three parallelism dimensions, two link speeds eighteen times apart, and one mapping that makes the job fast. The traffic each dimension generates per step, why the loudest one must stay inside the node, and the arithmetic showing what a wrong rank order costs before anyone notices.

Four of them diagnose, four configure the machine correctly, and several popular ones override tuning that was already right. Which is which, the symptom each addresses, and why a cluster whose job scripts carry a dozen of these has a configuration problem rather than a tuning one.

The number comes from one requirement: the gradient reduction has to finish inside the backward pass that produces it. Working that requirement backward gives a bandwidth per GPU, and the answer lands near the port speed the industry ships, which is not a coincidence.

Two all-to-alls per layer across sixty layers is over a hundred collectives on the critical path of every decode step, each one small and latency-bound. The per-GPU volume worked out, the time on each link type, and why a single large NVLink domain changes the design rather than just improving it.

It fails open: the data still moves, the job still runs, and everything is about half as fast. The one log line that settles it, the four reasons the library declines to use it, and why the bandwidth signature is close enough to exactly half to be diagnostic on its own.

Hardware partitions, shared SMs, and context switching are three different promises about isolation and waste. The slice arithmetic for an H100, what each mode gives up, and the one rule about tenants that decides most of it.

Same job, same GPU count, half the speed: the slow one was split 4 and 4 across two nodes and its tensor-parallel all-reduces run at fabric speed instead of NVLink speed. The bandwidth arithmetic, the per-parallelism placement rule, and the scheduler policy that enforces it.

A GPU node needs a driver, a container toolkit, a device plugin, a metrics exporter and a validator, in that order, and each needs to know which nodes it belongs on. The operator's component chain, the labels NFD writes to trigger it, and the bring-up arithmetic that rules out doing it by hand.

Device plugins count GPUs and know nothing else; Dynamic Resource Allocation lets a pod ask for devices by attribute and share them by claim. What the two models can and cannot express, what DRA changes for MIG, NVLink domains and multi-node gangs, and a migration stance for a fleet that runs both.

The driver logs a numbered code when something goes wrong on a GPU, and the number tells you whether to retry the job, drain the node or file a hardware return. The codes worth memorizing in three groups, the action each implies, and the automation that turns a log line into a drained node.

The exporter offers hundreds of fields and about a dozen of them change decisions. The health fields that predict failure, the performance fields that tell you whether work is happening, the one everybody alerts on that means nothing, and the diagnostic levels with the time each takes.

A link that corrupts occasionally does not fail; it retransmits, and each retransmission costs bandwidth that shows up as a slow job with no error anywhere. The counters that reveal it, the arithmetic turning a replay rate into lost bandwidth, and the threshold where watching becomes draining.

Each rank keeps a ring buffer of its recent collectives, and on a timeout every rank dumps it. Comparing sequence numbers across a thousand of those buffers finds the missing participant in one pass, and the three patterns those comparisons produce point at three different causes.

A 70B model's KV cache is 320 KB per token, and a 30k-token conversation is 10 GB that GPU memory cannot keep between turns. The three tiers with their bandwidths, the break-even where reloading a prefix beats recomputing it, the eviction and lookup design, and the traffic shape where the tier is worth its complexity.

Prefill is compute-bound and decode is bandwidth-bound, and one GPU running both does each worse. The pool ratio from the traffic's prompt and output lengths, the KV transfer cost per request and the link it needs, the router that pairs a prefill worker with a decode worker, and the load at which it beats a monolith.

The first question is whether 1T is total or active, because storage follows one and compute follows the other. The state budget, the layout that falls out of it, why expert parallelism belongs inside NVLink, and the two failure modes a dense-model plan does not have: router imbalance and all-to-all congestion.

The classification is a small function; the parser around it is where the problem is. Generators rather than lists, aggregation bounded by the number of distinct instructions rather than by the file, a top-k that never sorts the whole thing, and the malformed lines that stop the job at hour three.

Two phases of N-1 steps each, a chunk index that rotates with the step, and a byte count that should come out to exactly the textbook formula. The implementation, the verification against a direct sum, and the measured traffic matching 2(N-1)/N times the message at every rank count tested.

One atomic increment is correct and does not scale, because every thread fights for the same cache line. The ordering that is sufficient and why the default is stronger than needed, the padding that removes contention entirely, and the arithmetic showing when sharding is worth it.

The standard forces one heap allocation per entry and a pointer dereference per lookup, and that is a specification requirement rather than a library weakness. The measured cost on this machine, the open-addressing layout that removes it, and the two cases where the standard container is still the right pick.

Two ends, two access patterns, and one design choice that removes almost all contention. Why the owner takes the newest task and thieves take the oldest, what that does to cache locality, and the measurement showing why this cannot be demonstrated in Python at all.

The tiling is ten lines and the theory is a working-set calculation. The interesting part is that the obvious benchmark does not measure what it appears to, because the library underneath is already tiled, and knowing which effect your numbers contain is the skill being tested.

The answer that fails is a compliment to the company. The answer that lands is a thesis about where the constraint sits, tied to something you have actually done, and it works whether you are coming from distributed systems, kernels, hardware or SRE.

A speedup number with no method behind it is worth nothing, and experienced interviewers stop listening at the number and start asking about the baseline. What makes a performance claim survive scrutiny, and the four ways a real one turns out to be smaller than it looked.

Naming three newsletters answers nothing. The answer that works has a filter, a source of ground truth that is not a blog post, and one thing you reproduced yourself, because that is the only evidence that you learn rather than accumulate.

Arguing is the losing move and so is capitulating. What moves a vendor case is a reproducer small enough for their engineer to run, counters that isolate their layer from yours, and a bisection that removes your configuration as a variable entirely.

This is a test of whether you have a position you can defend, not of which problem you pick. What makes a thesis defensible, two worked examples with the arithmetic behind them, and the counterargument you have to be able to state before the interviewer does.

Being right is a small part of this. The move that works is converting an opinion into a document with the tradeoff written out, and then committing fully once the decision is made. The two cases where escalating is correct, and how to do it without a fight.

Four claims that are defensible from arithmetic available today, each with the counterargument that could sink it, and what each one implies about the work. Dated to 2026, because a thesis with no date is not a prediction.

At batch 1 a decode step reads every weight byte to do two FLOPs with it. The intensity is the batch size, the ridge is about 295, and the KV cache puts a ceiling on how far batching can push you. The full chain for a 70B model on H100.

The roofline, the sharding arithmetic and the parallelism trade-offs transfer unchanged, and the numbers are in the same units. What changes is who writes the kernels, how shapes must behave, and the interconnect topology you shard against. The v6e and v7 numbers worked through against an H100.

An N × N grid of multiply-accumulate cells holds a weight tile still and pumps activations through it, one row per cycle, with no register file or instruction fetch in the inner loop. The cycle count, the fill and drain, and the formula that says why a 128-row tile wastes two thirds of the array.

A Trn2 instance carries 16 Trainium2 chips with 1.5 TB of HBM and more dense bf16 FLOPS than an 8-GPU H100 node. Whether the cheaper FLOPS reach your workload depends on the Neuron compiler, the kernels you do not have yet, and an MFU number you must measure rather than assume.

192 GB and 5.3 TB/s per card beat the H100 on both memory numbers at a lower hourly price. Whether that wins in production is a question about achieved bandwidth and ROCm kernel coverage, and there is a short list of numbers to run before you sign the order.

Put the weights in on-chip SRAM and the HBM wall disappears: tens of terabytes per second per chip, and a batch-1 step limited by the pipeline rather than the memory. The price is capacity: a 70B needs hundreds of chips per replica, and the cost per token depends on keeping every one of them busy.

A linear layer at batch 1 runs at a third of a percent of peak and at batch 4,096 near 70%. The curve between has two regions and a sawtooth: the memory-bound slope where intensity equals M, the plateau where the tensor cores are the limit, and the dips where the tile count does not divide the SM count.

On an HBM part ECC costs almost nothing; on a GDDR part about 6% of capacity and bandwidth. What it buys is the difference between a corrected bit and a silently flipped exponent that turns 1.0 into infinity across a 512-GPU all-reduce. The mechanisms, the signals, and the expected-loss arithmetic.