Decoupled Diloco

“Together as Decoupled DiLoCo, it can tackle the key challenge of training at scale.”