3. Proximal Algorithms for Distributed Coupled Optimization

In this chapter, we consider a multi-node sharing problem, where each node possesses a local smooth function that is further considered as the average of several constituent functions, and the network aims to minimize a finite sum of all local functions plus a coupling function (possibly non-smooth). Due to its benefits in scalability, robustness, and flexibility, distributed optimization has been a significant focus in engineering research to tackle this problem. To accomplish this, an equivalent saddle-point problem of this problem that is amenable to distributed solutions is first formulated. Then, a novel distributed stochastic algorithm called VR-DPPD is proposed, which combines the variance-reduction technique of SAGA with the distributed proximal primal–dual method. We present a convergence analysis and demonstrate that if smooth local functions are strongly convex, VR-DPPD converges linearly to the exact optimal solution in expectation. With a novel linear convergent algorithm that achieves low computation costs, our work advances efforts to solve a general composite optimization problem with a convex (possibly non-smooth) coupling function. The viability and performance of VR-DPPD are demonstrated numerically.