Storage is a throughput problem until the files get small

Checkpoints need sustained bandwidth and datasets of millions of tiny samples need something else entirely: a request rate. A storage system sized for the first will fail at the second, and the symptom is idle accelerators with a storage system reporting plenty of spare bandwidth.

14 MIN · PREMIUM