SignSGD-FV: Communication-Efficient Distributed Learning Through Heterogeneous Edges

This paper presents signSGD with federated voting (signSGD-FV), a communication-efficient distributed learning algorithm with heterogeneous edge workers. The FV aggregation leverages the log-likelihood ratio (LLR) weight assigned to each worker, and performs weighted majority voting aggregation by interpreting the conventional signSGD with majority voting (signSGD-MV) algorithm in a coding-theoretical approach. The LLR weights are estimated based on the aggregation results determined by the sign votes of workers, which shows the essence of federated voting. Our theoretical analyses and the experimental results on real-world datasets demonstrate the superiority of signSGD-FV for both communication efficiency and learning performance when the workers employ different sizes of mini-batches.