Mixed precision is a numerics contract, not a speed setting

Low precision is used for the multiplications and wider precision is kept for the sums and the master weights, because each format fails in a specific place. Knowing which operations must stay wide is what separates a run that converges from one that quietly does not.

14 MIN · PLUS