A synchronous training job across two clouds runs at the speed of the link between them, a thousand times slower than the fabric inside either. The arithmetic that kills the idea for one job, the two cases where it is tolerable, and what a platform does instead: partition the work, not the job, and move data ahead.
Updated Sep 2026 · Grounded in real AI infrastructure interview loops and written to a senior-engineer editorial bar, with every number worked and every diagram hand-built.
A synchronous training job across two clouds runs at the speed of the link between them, a thousand times slower than the fabric inside either. The arithmetic that kills the idea for one job, the two cases where it is tolerable, and what a platform does instead: partition the work, not the job, and move data ahead.
Ranked by how closely each one overlaps this question's topic, so the first card is the thing to read if the answer above moved too fast.
Scored on the bandwidth arithmetic (inter-cloud link versus fabric, with the all-reduce time), on the two exceptions (very large models with rare sync, and asynchronous workloads), and on the alternative design of partitioning jobs and pre-staging data.
No comments yet — be the first to share your approach.
