Core

Numerics: FP32, BF16, FP8 and FP4

Every number format is a trade between range (exponent bits), precision (mantissa bits) and throughput (fewer bits, more values per cycle through the tensor cores). bf16 won training because it keeps fp32's range; fp8 splits into E4M3 for precision and E5M2 for range and needs scaling factors; fp4 needs block scaling and careful outlier handling. Knowing which format goes where, and why accumulation stays fp32, is what the numerics question is really asking.

a free account unlocks the core curriculum tier · no card

RELATED CONCEPTS

Tensor Cores and Matrix Units Quantization for Inference Memory-Bound vs Compute-Bound Kernels Model Memory Footprint Tiled Matrix Multiplication GPU Generations: A100 to Blackwell

LESSONS THAT TEACH THIS

Foundations of AI InfrastructureWeights are the floor, not the budget13 min→Training at ScaleMixed precision is a numerics contract, not a speed setting14 min→Inference EngineeringWeights, activations and the cache are three separate precision decisions15 min→

PRACTICE THIS IN REAL QUESTIONS

GPU & Accelerator Architecturebf16, fp16, fp8: what is the difference at the bit level, and where does each one belong in training and serving?→Distributed Training & ParallelismExplain mixed-precision training. Why does fp16 need loss scaling and bf16 not, why keep fp32 master weights, and what changes with fp8?→CUDA, Triton & Kernel EngineeringWhy do tensor cores accumulate in fp32 when the inputs are bf16, and why do optimizers keep fp32 master weights?→CUDA, Triton & Kernel EngineeringIn an int8 or fp8 GEMM, what exactly gets scaled, where does the dequantization happen, and what breaks if you get it wrong?→GPU & Accelerator ArchitectureYou are moving a model to fp4 inference on Blackwell. What breaks first, and how would you measure whether the result is acceptable?→GPU & Accelerator ArchitectureCompare A100, H100 and B200 for a 70B serving fleet. Which gives the most tokens per dollar, and where do fp8 and fp4 change the ranking?→