Napkin Math, Cost & Capacity: the practice test
Memory footprints, 6ND, arithmetic intensity and the ridge point, bandwidth-bound decode, communication volume, GPU counts and time to train, cost per million tokens, TCO and buy versus rent. The estimation round almost every AI infra loop includes, with every assumption stated. This test drills exactly that: 12 easy, 11 medium and 7 hard questions, every one explained, every explanation linking into the worked material.
Sample questions, answered
Weights = parameters × bytes per parameter = 70 × 10⁹ × 2 B = 140 GB. One H100 has 80 GB, so the weights alone need two cards, and the KV cache and activations need space on top. The 70 GB answer uses one byte per parameter, which is fp8, not bf16; the 280 GB answer uses fp32. Bytes per parameter is the first number in every memory chain, and 2 for bf16 is the one to know cold.
KV per token = 2 (keys and values) × layers × KV heads × head dim × bytes = 2 × 80 × 8 × 128 × 2 B = 327,680 B, about 320 KB. Forgetting the factor of 2 for K and V gives 160 KB; using the 64 query heads instead of the 8 KV heads ignores grouped-query attention and gives 2.6 MB, which is what an MHA model would need. Every layer keeps its own K and V for every past token, so the per-layer answer is wrong too.
Go deeper than the quiz
A practice test measures recall. The material it draws from teaches the reasoning: