skip to main content
10.1145/3011077.3011095acmotherconferencesArticle/Chapter ViewAbstractPublication PagessoictConference Proceedingsconference-collections
research-article

A diagonal cabling approach to data center and HPC systems

Authors Info & Claims
Published:08 December 2016Publication History

ABSTRACT

Low cable delay becomes a critical concern in High Performance Computing systems and high-density Data Centers since the switch delay becomes very low, e.g., 40 ns per switch, as the technology advances. The cable delay almost corresponds to the cable length. It significantly requires a short total end-to-end cable length when deploying a logical topology into a physical cabinet layout in a server room. Recent works illustrate that the problem of mapping any topology onto the floor plan with an efficient cable length can be modeled and solved as an optimization problem. This approach based on the Manhattan cabling method, with which the cables are organized into several horizontal and vertical pathways and the length of each cable becomes Manhattan distance. To reduce the aggregate cable length, we aggressively take the diagonal cabling method by increasing the number of directions on a surface. The analysis results with different topologies show that our approach can reduce the total cable length by up to 11%, which leads to 7% reduction of network latency.

References

  1. Infiniband too quick for ethernet to kill it. http://www.nextplatform.com/2015/04/01/infiniband-too-quick-for-ethernet-to-kill-it/, 2015.Google ScholarGoogle Scholar
  2. Y. Ajima, S. Sumimoto, and T. Shimizu. Tofu: A 6D Mesh/Torus Interconnect for Exascale Computers. IEEE Computer, 42:36--40, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. M. Besta and T. Hoefler. Slim Fly: A Cost Effective Low-Diameter Network Topology. In Proc. of the IEEE/ACM International Conference on High Performance Computing, Networking, Storage and Analysis (SC14), pages 348--359, Nov. 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. I. Fujiwara, M. Koibuchi, and H. Casanova. Cabinet Layout Optimization of Supercomputer Topologies for Shorter Cable Length. In Proc. of International Conference on Parallel and Distributed Computing, Applications and Technologies (PDCAT), pages 227--232, Dec. 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. I. Fujiwara, M. Koibuchi, H. Matsutani, and H. Casanova. Skywalk: A topology for HPC networks with low-delay switches. In 2014 IEEE International Parallel and Distributed Processing Symposium, Phoenix, AZ, USA, May 19--23, 2014, pages 263--272, 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. P. Hannaford. Ten Cooling Solutions to Support High-Density Server Deployment. http://www.apcmedia.com/salestools/TDOY-5U362W/TDOY-5U362W_R5_EN.pdf.Google ScholarGoogle Scholar
  7. HP. Optimizing facility operation in high density data center environments, technoloogy brief. http://h18004.www1.hp.com/products/servers/technology/whitepapers/datacenter.html, 2007.Google ScholarGoogle Scholar
  8. HP. An Algorithmic Approach to Datacenter Cabling. http://www.labs.hpe.com/techreports/2015/HPL-2015-8.pdf, 2015.Google ScholarGoogle Scholar
  9. K. Scott Hemmert et al. Report on Institute for Advanced Architectures and Algorithms, Interconnection Networks Workshop 2008. http://ft.ornl.gov/pubs-archive/iaa-ic-2008-workshop-report-final.pdf.Google ScholarGoogle Scholar
  10. J. Kim, W. J. Dally, and D. Abts. Flattened butterfly: a cost-efficient topology for high-radix networks. In Proc, of the International Symposium on Computer Architecture (ISCA), pages 126--137, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. J. Kim, W. J. Dally, S. Scott, and D. Abts. Technology-Driven, Highly-Scalable Dragonfly Topology. In Proc. of the International Symposium on Computer Architecture (ISCA), pages 77--88, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. W. T. Neil Rasmussen. Data Center Projects: Establishing a Floor Plan. http://www.apcmedia.com/salestools/VAVR-6KYMZ7/VAVR-6KYMZ7_R2_EN.pdf?sdirect=true.Google ScholarGoogle Scholar
  13. V. K. Nguyen, N. T. X. Le, I. Fujiwara, and M. Koibuchi. Distributed shortcut networks: Layout-aware low-degree topologies exploiting small-world effect. In ICPP, pages 572--581, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. J.-Y. Shin, B. Wong, and E. G. Sirer. Small-world datacenters. In Proceedings of the 2nd ACM Symposium on Cloud Computing (SOCC11), pages 2:1--2:13, New York, NY, USA, 2011. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. A. Singla, C.-Y. Hong, L. Popa, and P. B. Godfrey. Jellyfish: Networking Data Centers Randomly. In Proc. of USENIX Symposium on Network Design and Implementation (NSDI), pages 225--238, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. B. Towles, J. P. Grossman, B. Greskamp, and D. E. Shaw. Unifying on-chip and inter-node switching within the Anton 2 network. In Proc. of the ACM/IEEE International Symposium on Computer Architecture, (ISCA), pages 1--12, June. 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. T.-N. Truong, K.-V. Nguyen, I. Fujiwara, and M. Koibuchi. Layout-conscious expandable topology for low-degree interconnection networks. IEICE TRANSACTIONS on Information and Systems, 99(5):1275--1284, 2016.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. A diagonal cabling approach to data center and HPC systems

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in
      • Published in

        cover image ACM Other conferences
        SoICT '16: Proceedings of the 7th Symposium on Information and Communication Technology
        December 2016
        442 pages
        ISBN:9781450348157
        DOI:10.1145/3011077

        Copyright © 2016 ACM

        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 8 December 2016

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article

        Acceptance Rates

        SoICT '16 Paper Acceptance Rate58of132submissions,44%Overall Acceptance Rate147of318submissions,46%
      • Article Metrics

        • Downloads (Last 12 months)1
        • Downloads (Last 6 weeks)0

        Other Metrics

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader