Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
Blood Leukocyte Object Detection According to Model Parameter-Transfer and Deformable Convolution Read chapter Read first chapter

2021 | OriginalPaper | Chapter

Data Aggregation Aware Routing for Distributed Training

Authors : Zhaohong Chen, Xin Long, Yalan Wu, Long Chen, Jigang Wu, Shuangyin Liu

Published in: Parallel and Distributed Computing, Applications and Technologies

Publisher: Springer International Publishing

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

For distributed training, the communication overhead for parameter synchronization is heavy in the network. Data aggregation can efficiently alleviate network overheads. However, existing works on data aggregation are based on the streaming message data, which can not well adapt to the discrete communication for parameter synchronization. This paper formulates a data aggregation aware routing problem, with the objective of minimizing training finishing time for global model under the constraint of cache capacity. The problem is formulated as a mixed-integer non-linear programming problem, and it is proved to be NP-Hard. Then we propose a data aggregation aware routing algorithm to solve the formulated problem, by transmitting the data to the closest aggregation node in greedy to reduce the network overhead. Simulation results show that, the proposed algorithm can reduce average training finishing time by \(74\%\), and it can reduce the network overhead by \(33\%\) on average, compared with the shortest path algorithm.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
previous chapter See Fine Color from the Rough Black-and-White
next chapter A New Integer Programming Model for the File Transfer Scheduling Problem
Literature
1.
Rahman, H., Ahmed, N., Hussain, M.I.: A qos-aware hybrid data aggregation scheme for internet of things. Ann. Telecommun. 73(7), 475–486 (2018)CrossRef
2.
Redondi, A.E., Cesana, M., Fratta, L., Capone, A., Borgonovo, F.: A prediction-based approach for features aggregation in visual sensor networks. Ad Hoc Netw. 83(1), 55–67 (2019)CrossRef
3.
Cui, J., Boussetta, K., Valois, F.: Classification of data aggregation functions in wireless sensor networks. Comput. Netw. 178(1), 1–46 (2020)
4.
Chen, C.C.Y., Das, S.K.: Breadth-first traversal of trees and integer sorting in parallel. Inf. Process. Lett. 41(1), 39–49 (1992)MathSciNetCrossRef
5.
6.
7.
Yang, S., Li, F., Trajanovski, S., Chen, X., Wang, Y., Fu, X.: Delay-aware virtual network function placement and routing in edge clouds. IEEE Trans. Mob. Comput. 1–14 (2019)
8.
Li, C., Tang, J., Tang, H., Luo, Y.: Collaborative cache allocation and task scheduling for data-intensive applications in edge computing environment. Future Gener. Comput. Syst. 95, 249–264 (2019)CrossRef
9.
Szegedy, C., et al.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9. IEEE (2015)
10.
Qassim, H., Verma, A., Feinzimer, D.: Compressed residual-VGG16 CNN model for big data places image recognition. In: IEEE Annual Computing and Communication Workshop and Conference, pp. 169–175. IEEE (2018)
11.
Howard, A.G., et al.: Mobilenets: efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:​1704.​04861, 1–9 (2017)
12.
Zhang, X., Zhou, X., Lin, M., Sun, J.: Shufflenet: an extremely efficient convolutional neural network for mobile devices. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6848–6856. IEEE (2018)
13.
Garcia-Luna-Aceves, J.J.: A distributed, loop-free, shortest-path routing algorithm. In: Proceedings of the IEEE Conference on Computer Communications, pp. 1125–1137. IEEE (1988)
Metadata
Title
Data Aggregation Aware Routing for Distributed Training
Authors
Zhaohong Chen
Xin Long
Yalan Wu
Long Chen
Jigang Wu
Shuangyin Liu
Copyright Year
2021
Book
Parallel and Distributed Computing, Applications and Technologies

Print ISBN: 978-3-030-69243-8

Electronic ISBN: 978-3-030-69244-5

Copyright Year: 2021

DOI
https://doi.org/10.1007/978-3-030-69244-5_21

Premium Partner

    Image Credits