Transport efficiency for data-intensive science: deployment experiences and bottleneck analysis

Transferring massive datasets in data-intensive science (DIS) systems often relies on physical WAN infrastructure for network connectivity. This infrastructure is typically provided by various National Research and Education Networks (NRENs), including ESnet, GÉANT, Internet2, and RNP. Studying these systems presents significant challenge due to their complexity, scale, and the numerous factors influencing data transport. Traditionally, network performance studies focus on a single bottleneck. In contrast, the Quantitative Theory of Bottlenecks Structures (QTBS) provides a mathematical framework that analyzes performance through the network’s entire bottleneck structure, offering valuable insights for optimizing and understanding overall network performance. This paper tackles such challenges by employing QTBS and by deploying and evaluating a virtual infrastructure for data transport within a national-scale WAN. Our approach focuses on three key aspects: (i) assessing flow completion times related to bandwidth allocation for interdependent transfers within a network slice, (ii) evaluating the performance of TCP congestion control algorithms (BBR versus Cubic) for data transport, and (iii) conducting QTBS analysis to compute flow allocation shares, ultimately aiming for an optimal design. Results show BBR outperforming Cubic in scenarios with high number of threads and data volume and the high influence of the number of threads.