Skip to main content
main-content

Tipp

Weitere Artikel dieser Ausgabe durch Wischen aufrufen

Erschienen in: Photonic Network Communications 2/2022

14.02.2022 | Original Paper

Coflow scheduling and placement for packet-switched optical datacenter networks

verfasst von: Lin Wang, Xinbo Wang, Massimo Tornatore, Kwangjoon Kim, Biswanath Mukherjee

Erschienen in: Photonic Network Communications | Ausgabe 2/2022

Einloggen, um Zugang zu erhalten
share
TEILEN

Abstract

Data-parallel computing applications (DPCAs) (e.g., MapReduce, web search, etc.) are driving the need of scalable, low-latency, high-speed, and energy-efficient datacenters, because a DPCA consists of a series of heavy-computation stages within a datacenter, and each stage contains multiple parallel flows that must be completed before next stage starts, referred to as “Coflow”. These parallel flows are grouped as a Coflow. Coflow is a networking abstraction to convey application-level communication requirements by exposing rich semantics of DPCAs to underlying networks, e.g., latency of data transmission between two computation stages, known as “Coflow Completion Time” (CCT). Packet-switched optical network (PSON) is a practical intra-datacenter interconnect solution for DPCAs, as it is designed as a low-complexity and scalable one-stage switching architecture, using advanced optical networking technologies, such as Arrayed Waveguide Grating Routers and wavelength-division multiplexing. In this work, we study how to minimize CCT in PSON-enabled datacenters by placing senders and receivers of Coflows to proper transceiver nodes and scheduling data transmission wisely, for which we propose a Coflow-aware placement and scheduling algorithm, consisting of Min-Priority placement algorithm and Priority-aware scheduling algorithm. They are designed to cooperate with each other to jointly minimize CCT. Numerical simulations demonstrate the benefits of joint design of Coflow placement and scheduling algorithm, compared to state-of-the-art scheduling and placement algorithms designed without correlation.
Literatur
1.
Zurück zum Zitat Index, C.G.C.: Forecast and Methodology, 2015–2020. Cisco Systems Inc, San Jose (2016) Index, C.G.C.: Forecast and Methodology, 2015–2020. Cisco Systems Inc, San Jose (2016)
2.
Zurück zum Zitat Ananthanarayanan, G., et al.: Coordinated memory caching for parallel jobs. NSDI 20 12  (2012) Ananthanarayanan, G., et al.: Coordinated memory caching for parallel jobs. NSDI 20 12  (2012)
3.
Zurück zum Zitat Zahariaetal, M., et al.: Resilient distributed datasets: a fault-tolerant P2 abstraction for in-memory cluster computing. NSDI 2 (2012) Zahariaetal, M., et al.: Resilient distributed datasets: a fault-tolerant P2 abstraction for in-memory cluster computing. NSDI 2 (2012)
4.
Zurück zum Zitat Chowdhury, M., et al.: Coflow: a networking abstraction for cluster applications. In: 11th ACM Workshop on Hot Topics in Networks (2012) Chowdhury, M., et al.: Coflow: a networking abstraction for cluster applications. In: 11th ACM Workshop on Hot Topics in Networks (2012)
5.
Zurück zum Zitat Dogar, F., et al.: Decentralized task-aware scheduling for datacenter networks. ACM SIGCOMM Comput. Commun. Rev. 44(4), 431–442 (2014) CrossRef Dogar, F., et al.: Decentralized task-aware scheduling for datacenter networks. ACM SIGCOMM Comput. Commun. Rev. 44(4), 431–442 (2014) CrossRef
6.
Zurück zum Zitat Chowdhury, M., et al.: Efficient coflow scheduling with varys. ACM SIGCOMM Comput. Commun. 44(4), 443–454 (2014) CrossRef Chowdhury, M., et al.: Efficient coflow scheduling with varys. ACM SIGCOMM Comput. Commun. 44(4), 443–454 (2014) CrossRef
7.
Zurück zum Zitat Chowdhury, M., et al.: Efficient coflow scheduling without prior knowledge. ACM SIGCOMM Comput. Commun. Rev. 45(4), 393–406 (2015) CrossRef Chowdhury, M., et al.: Efficient coflow scheduling without prior knowledge. ACM SIGCOMM Comput. Commun. Rev. 45(4), 393–406 (2015) CrossRef
8.
Zurück zum Zitat Huang, X., et al.: Exploiting inter-flow relationship for coflow placement in datacenters. In: Proceedings of the 1st Asia-Pacific Workshop on Networking, pp. 113–119, Hong Kong, China (2017) Huang, X., et al.: Exploiting inter-flow relationship for coflow placement in datacenters. In: Proceedings of the 1st Asia-Pacific Workshop on Networking, pp. 113–119, Hong Kong, China (2017)
9.
Zurück zum Zitat Munir, A., et al.: Network scheduling aware task placement in datacenters. In: Proceedings of the 12th International on Conference on Emerging Networking Experiments and Technologies, Irvine, pp. 221–235 (2016) Munir, A., et al.: Network scheduling aware task placement in datacenters. In: Proceedings of the 12th International on Conference on Emerging Networking Experiments and Technologies, Irvine, pp. 221–235 (2016)
11.
Zurück zum Zitat Kachris, C., et al.: A survey on optical interconnects for data centers. IEEE Commun. Surv. Tutor. 14(4), 1021–1036 (2012) CrossRef Kachris, C., et al.: A survey on optical interconnects for data centers. IEEE Commun. Surv. Tutor. 14(4), 1021–1036 (2012) CrossRef
12.
Zurück zum Zitat R. Proietti, et al. “40 Gb∕s 8 × 8 low-latency optical switch for data centers,” Optical Fiber Communication Conf., Los Angeles, CA, 2011. R. Proietti, et al. “40 Gb∕s 8 × 8 low-latency optical switch for data centers,” Optical Fiber Communication Conf., Los Angeles, CA, 2011.
13.
Zurück zum Zitat Xi, K., et al.: Petabit optical switch for datacenter networks. Tech. Rep., Polytechnic Institute of New York University, New York (2010) Xi, K., et al.: Petabit optical switch for datacenter networks. Tech. Rep., Polytechnic Institute of New York University, New York (2010)
14.
Zurück zum Zitat Liboiron-Ladouceur, O., et al.: Energy-efficient design of a scalable optical multiplane interconnection architecture. IEEE J. Sel. Top. Quantum Electron. 17(2), 377–383 (2010) CrossRef Liboiron-Ladouceur, O., et al.: Energy-efficient design of a scalable optical multiplane interconnection architecture. IEEE J. Sel. Top. Quantum Electron. 17(2), 377–383 (2010) CrossRef
15.
Zurück zum Zitat Wang, L., et al.: Scheduling with machine-learning-based flow detection for packet-switched optical data center networks. J. Opt. Commun. Netw. 10(4), 365–375 (2018) CrossRef Wang, L., et al.: Scheduling with machine-learning-based flow detection for packet-switched optical data center networks. J. Opt. Commun. Netw. 10(4), 365–375 (2018) CrossRef
16.
Zurück zum Zitat Chowdhury, M., et al.: Leveraging endpoint flexibility in data-intensive clusters. ACM SIGCOMM Comput. Commun. Rev. 43(4), 231–242 (2013) CrossRef Chowdhury, M., et al.: Leveraging endpoint flexibility in data-intensive clusters. ACM SIGCOMM Comput. Commun. Rev. 43(4), 231–242 (2013) CrossRef
Metadaten
Titel
Coflow scheduling and placement for packet-switched optical datacenter networks
verfasst von
Lin Wang
Xinbo Wang
Massimo Tornatore
Kwangjoon Kim
Biswanath Mukherjee
Publikationsdatum
14.02.2022
Verlag
Springer US
Erschienen in
Photonic Network Communications / Ausgabe 2/2022
Print ISSN: 1387-974X
Elektronische ISSN: 1572-8188
DOI
https://doi.org/10.1007/s11107-021-00958-4

Weitere Artikel der Ausgabe 2/2022

Photonic Network Communications 2/2022 Zur Ausgabe