Anzeige
Erschienen in:
2021 | OriginalPaper | Buchkapitel
Data Aggregation Aware Routing for Distributed Training
verfasst von : Zhaohong Chen, Xin Long, Yalan Wu, Long Chen, Jigang Wu, Shuangyin Liu
Erschienen in: Parallel and Distributed Computing, Applications and Technologies
Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.
Wählen Sie Textabschnitte aus um mit Künstlicher Intelligenz passenden Patente zu finden. powered by
Markieren Sie Textabschnitte, um KI-gestützt weitere passende Inhalte zu finden. powered by
Abstract
For distributed training, the communication overhead for parameter synchronization is heavy in the network. Data aggregation can efficiently alleviate network overheads. However, existing works on data aggregation are based on the streaming message data, which can not well adapt to the discrete communication for parameter synchronization. This paper formulates a data aggregation aware routing problem, with the objective of minimizing training finishing time for global model under the constraint of cache capacity. The problem is formulated as a mixed-integer non-linear programming problem, and it is proved to be NP-Hard. Then we propose a data aggregation aware routing algorithm to solve the formulated problem, by transmitting the data to the closest aggregation node in greedy to reduce the network overhead. Simulation results show that, the proposed algorithm can reduce average training finishing time by \(74\%\), and it can reduce the network overhead by \(33\%\) on average, compared with the shortest path algorithm.