Google DeepMind Introduces Decoupled DiLoCo: An Asynchronous Training Architecture Achieving 88% Goodput Under High Hardware Failure Rates

Google DeepMind introduced Decoupled DiLoCo, a distributed training architecture that splits work into asynchronous, fault-isolated compute islands.

Instead of blocking synchronization, it uses compressed cross-cluster updates to keep training moving across distant data centers.

In simulations with high failure rates, it reached 88% goodput and cut inter-datacenter bandwidth needs from 198 Gbps to 0.84 Gbps across eight sites.

The approach could make frontier AI training more scalable and resilient by reducing stalls from chip failures and WAN bottlenecks.