hardNewNVIDIAFireworksTogether AI

An Nsight summary reads DRAM 90%, SM 30%, occupancy 45%, L2 hit rate 20%. What is the bottleneck?

Four percentages from a profiler summary are enough to place a kernel on the roofline, estimate its arithmetic intensity, and rule out half the optimizations people reach for. The reading, the arithmetic that backs it, and the fix list in the order it pays.

Updated Sep 2026 · Grounded in real AI infrastructure interview loops and written to a senior-engineer editorial bar, with every number worked and every diagram hand-built.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 283 remaining answers · ₹2,000 / $25

The concepts behind this question

Ranked by how closely each one overlaps this question's topic, so the first card is the thing to read if the answer above moved too fast.

Advanced

⚡ Kernels & Compilers🔒 Premium

Profiling with NsightNsight Systems answers where wall-clock time goes across CPU, kernels and copies; Nsight Compute answers why one kernel is slow, from hardware counters. The skill interviewers test is the order: timeline first, then the Speed of Light section, then the two or three metrics that name the bottleneck, so that a memory-bound kernel is recognized from its profile in under a minute and the fix is bytes, not occupancy.

Foundational

🧩 GPU & Accelerator Architecture

Roofline ModelThe roofline plots a kernel's attainable throughput against its arithmetic intensity, FLOPs per byte moved from memory. Below the ridge point (peak FLOPS divided by memory bandwidth, about 295 on an H100 in bf16) a kernel is memory-bound and no amount of clever code reaches the peak; above it, compute is the limit. One picture explains why decode runs at under 1% of peak and why fusion and batching are the two levers that move it.

Advanced

🧩 GPU & Accelerator Architecture🔒 Premium

Memory-Bound vs Compute-Bound KernelsEvery kernel is limited by one of two walls: how fast bytes arrive from HBM, or how fast the tensor cores can multiply. Which wall applies is decided by arithmetic intensity against the ridge point, and the two regimes need opposite fixes. Decode, LayerNorm and softmax are memory-bound; prefill GEMMs are compute-bound; the interview question is which one you are looking at and what you would do about it.

Foundational

🚀 Inference & Serving

Prefill vs DecodeAn LLM request runs in two phases with opposite hardware profiles: prefill reads the whole prompt in one compute-bound pass and decides time to first token, decode emits one token per forward pass and is bound by memory bandwidth. Every serving decision, from batch size to which GPU to buy to whether to split the two phases across machines, follows from that split.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackWhy does the batch size change the TFLOPS a GEMM achieves? Draw the curve and explain each region.Next in this trackWhat can you rely on from the L2 cache on a GPU? When does it save a kernel, and when does it mislead you?Next in this trackWhat does ECC on a GPU cost you, and why do you keep it on across a fleet?

FEDITOR'S NOTE

Scored on reading the metrics as a system rather than a checklist: DRAM near peak with SM idle means memory-bound, occupancy is a distraction here, and the fix list is about bytes, not warps.

DISCUSSION · 0

No comments yet — be the first to share your approach.