Constrained Optimization With Low-Bit Gradients: A Dynamical Systems Perspective

In constrained signal processing, which encompasses areas such as compressed sensing, noisy signal recovery, and matrix completion, the communication overhead of gradients, both inter- and intra-process, often emerges as a substantial bottleneck. Gradient compression significantly alleviates this problem by sending low-bit gradients while ensuring gradient descent convergence. Although such low-bit technique is effective in application, the theoretical basis still remains largely unexplored. In this work, we establish a unified framework for the convergence analysis of projected optimization methods with low-bit gradients, especially from the perspective of continuous-time nonsmooth dynamical systems. Moreover, we propose a provably convergent distributed gradient compression scheme for constrained low-bit signal processing applications. Numerical experiments are conducted to confirm the validility of the theoretical analysis, showing that our distributed algorithm effectively transmits low-bit gradients with negligible effect on the convergence rate for constrained nonconvex optimization.