ABSTRACT
Low cable delay becomes a critical concern in High Performance Computing systems and high-density Data Centers since the switch delay becomes very low, e.g., 40 ns per switch, as the technology advances. The cable delay almost corresponds to the cable length. It significantly requires a short total end-to-end cable length when deploying a logical topology into a physical cabinet layout in a server room. Recent works illustrate that the problem of mapping any topology onto the floor plan with an efficient cable length can be modeled and solved as an optimization problem. This approach based on the Manhattan cabling method, with which the cables are organized into several horizontal and vertical pathways and the length of each cable becomes Manhattan distance. To reduce the aggregate cable length, we aggressively take the diagonal cabling method by increasing the number of directions on a surface. The analysis results with different topologies show that our approach can reduce the total cable length by up to 11%, which leads to 7% reduction of network latency.
- Infiniband too quick for ethernet to kill it. http://www.nextplatform.com/2015/04/01/infiniband-too-quick-for-ethernet-to-kill-it/, 2015.Google Scholar
- Y. Ajima, S. Sumimoto, and T. Shimizu. Tofu: A 6D Mesh/Torus Interconnect for Exascale Computers. IEEE Computer, 42:36--40, 2009. Google ScholarDigital Library
- M. Besta and T. Hoefler. Slim Fly: A Cost Effective Low-Diameter Network Topology. In Proc. of the IEEE/ACM International Conference on High Performance Computing, Networking, Storage and Analysis (SC14), pages 348--359, Nov. 2014. Google ScholarDigital Library
- I. Fujiwara, M. Koibuchi, and H. Casanova. Cabinet Layout Optimization of Supercomputer Topologies for Shorter Cable Length. In Proc. of International Conference on Parallel and Distributed Computing, Applications and Technologies (PDCAT), pages 227--232, Dec. 2012. Google ScholarDigital Library
- I. Fujiwara, M. Koibuchi, H. Matsutani, and H. Casanova. Skywalk: A topology for HPC networks with low-delay switches. In 2014 IEEE International Parallel and Distributed Processing Symposium, Phoenix, AZ, USA, May 19--23, 2014, pages 263--272, 2014. Google ScholarDigital Library
- P. Hannaford. Ten Cooling Solutions to Support High-Density Server Deployment. http://www.apcmedia.com/salestools/TDOY-5U362W/TDOY-5U362W_R5_EN.pdf.Google Scholar
- HP. Optimizing facility operation in high density data center environments, technoloogy brief. http://h18004.www1.hp.com/products/servers/technology/whitepapers/datacenter.html, 2007.Google Scholar
- HP. An Algorithmic Approach to Datacenter Cabling. http://www.labs.hpe.com/techreports/2015/HPL-2015-8.pdf, 2015.Google Scholar
- K. Scott Hemmert et al. Report on Institute for Advanced Architectures and Algorithms, Interconnection Networks Workshop 2008. http://ft.ornl.gov/pubs-archive/iaa-ic-2008-workshop-report-final.pdf.Google Scholar
- J. Kim, W. J. Dally, and D. Abts. Flattened butterfly: a cost-efficient topology for high-radix networks. In Proc, of the International Symposium on Computer Architecture (ISCA), pages 126--137, 2007. Google ScholarDigital Library
- J. Kim, W. J. Dally, S. Scott, and D. Abts. Technology-Driven, Highly-Scalable Dragonfly Topology. In Proc. of the International Symposium on Computer Architecture (ISCA), pages 77--88, 2008. Google ScholarDigital Library
- W. T. Neil Rasmussen. Data Center Projects: Establishing a Floor Plan. http://www.apcmedia.com/salestools/VAVR-6KYMZ7/VAVR-6KYMZ7_R2_EN.pdf?sdirect=true.Google Scholar
- V. K. Nguyen, N. T. X. Le, I. Fujiwara, and M. Koibuchi. Distributed shortcut networks: Layout-aware low-degree topologies exploiting small-world effect. In ICPP, pages 572--581, 2013. Google ScholarDigital Library
- J.-Y. Shin, B. Wong, and E. G. Sirer. Small-world datacenters. In Proceedings of the 2nd ACM Symposium on Cloud Computing (SOCC11), pages 2:1--2:13, New York, NY, USA, 2011. ACM. Google ScholarDigital Library
- A. Singla, C.-Y. Hong, L. Popa, and P. B. Godfrey. Jellyfish: Networking Data Centers Randomly. In Proc. of USENIX Symposium on Network Design and Implementation (NSDI), pages 225--238, 2012. Google ScholarDigital Library
- B. Towles, J. P. Grossman, B. Greskamp, and D. E. Shaw. Unifying on-chip and inter-node switching within the Anton 2 network. In Proc. of the ACM/IEEE International Symposium on Computer Architecture, (ISCA), pages 1--12, June. 2014. Google ScholarDigital Library
- T.-N. Truong, K.-V. Nguyen, I. Fujiwara, and M. Koibuchi. Layout-conscious expandable topology for low-degree interconnection networks. IEICE TRANSACTIONS on Information and Systems, 99(5):1275--1284, 2016.Google ScholarCross Ref
Index Terms
- A diagonal cabling approach to data center and HPC systems
Recommendations
Cabinet Layout Optimization of Supercomputer Topologies for Shorter Cable Length
PDCAT '12: Proceedings of the 2012 13th International Conference on Parallel and Distributed Computing, Applications and TechnologiesAs the scales of supercomputers increase total cable length becomes enormous, e.g., up to thousands of kilometers. Recent high-radix switches with dozens of ports make switch layout and system packaging more complex. In this study, we study the ...
Layout-conscious random topologies for HPC off-chip interconnects
HPCA '13: Proceedings of the 2013 IEEE 19th International Symposium on High Performance Computer Architecture (HPCA)As the scales of parallel applications and platforms increase the negative impact of communication latencies on performance becomes large. Random network topologies can be used to achieve low hop counts between nodes and thus low latency. However, ...
Comments