Roofline calculator

Peak compute divided by memory bandwidth, in FLOPs per byte. On an H100 SXM that is 989 TFLOPS over 3.35 TB/s, about 295. A kernel with lower arithmetic intensity is memory-bound and cannot reach the peak no matter how well it is written; one above it is compute-bound.

Roughly the batch size, for bf16 weights: each weight is read once (2 bytes) and used for two FLOPs per sequence in the batch. At batch one that is about 1 FLOP per byte, hundreds of times below the H100 ridge, which is why single-stream decode reaches under 1% of peak.

It raises intensity by removing trips to HBM: the intermediate tensor stays in registers or shared memory instead of being written and read back. A fused softmax does the same FLOPs as the three separate kernels it replaces with a third of the memory traffic, so its point moves right along the roofline.