Random Orthogonalization for Private Wireless Federated Learning

We consider the problem of private wireless feder-ated learning through a massive MIMO multiple-access channel (MAC). In this problem, a parameter server (PS) having <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$M$</tex> antennas needs to train a global machine learning model with the aid of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$K$</tex> single-antenna users. Each user trains a local model to update the PS's global model without leaking information about the user's local model. By harnessing the additive nature of the MAC, the PS aggregates the local updates and updates the global model. We show that by adopting the random orthogonalization technique and careful noise injection by the users, maintaining the privacy of local models is possible under local differential privacy metrics without sacrificing the accuracy/convergence rate of the global machine learning model. We derive the exact achievable privacy level. Our results show that the privacy level is a function of the channel gains. We substantiate our findings by carrying out a standard classification task, which achieves an accuracy of 89% in less than 15 communication rounds while maintaining an acceptable privacy level of the users' local models. Moreover, numerical results show that the privacy leakage is decreasing in the number of users K, while it is increasing in the number of antennas at the PS M.