Kimad: Adaptive Gradient Compression with Bandwidth Awareness

In distributed training, communication often emerges as a bottleneck. In response, we introduce Kimad, a solution that offers adaptive gradient compression. By consistently monitoring bandwidth, Kimad refines compression ratios to match specific neural network layer requirements. Our exhaustive test...

Descripción completa

Detalles Bibliográficos

Autores principales:	Xin, Jihao, Ilin, Ivan, Zhang, Shunkang, Canini, Marco, Richtárik, Peter
Formato:	Artículo
Publicado:	2023
Materias:	Machine Learning Distributed, Parallel, and Cluster Computing Information Theory
Acceso en línea:	https://arxiv.org/abs/2312.08053 http://arxiv.org/abs/2312.08053
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!