Kun Yuan, Bicheng Ying, Jiageng Liu, and Ali H. Sayed
IEEE Transactions on Signal Processing 67 (2), 351-366. doi: 10.1109/TSP.2018.2872003.
In decentralized algorithms, computing nodes only communicate with their neighbors (no central servers involved), which eliminates server failure problems, relieves communication bottlenecks, and protects data privacy. Applications: supercomputers with millions of cores, networked self-driving cars.
However, decentralized algorithms often (1) fail to converge, (2) require more computation time, and/or (3) cost excessive communication compared to single-machine algorithms.
Our algorithm, Diffusion AVRG, solves (1) and outperforms state-of-the-art algorithms on both (2) and (3).
Fig. 1 Our algorithm, under the "best" setting shown in the figure, reduces the time cost of a standard machine learning task from 21.3 to 4.4 units of time.