Skip to main content
Top
Published in: The Journal of Supercomputing 9/2021

11-02-2021

High-performance simulations of turbulent boundary layer flow using Intel Xeon Phi many-core processors

Authors: Ji-Hoon Kang, Jinyul Hwang, Hyung Jin Sung, Hoon Ryu

Published in: The Journal of Supercomputing | Issue 9/2021

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Direct numerical simulations (DNS) of turbulent flows have increasing importance because they not only provide fundamental understanding of turbulent flows but also complement and extend experimental results. DNS of high Reynolds numbers, however, require huge computing cost so high-performance computing has been strongly pursued. In this study, we examine the feasibility of cost-efficient DNS on Intel Xeon Phi many-core processors that are currently adopted by 10% of the 100 largest supercomputers in the world as listed in the Top500 site. For this purpose, we port and optimize our in-house turbulent flow solver named as DNS-TBL (direct numerical simulation-turbulent boundary layer) on Xeon Phi Knights Landing (KNL) many-core processors and conduct benchmark tests on KNL and conventional multicore processors. The key architectural features of KNL processors and strategies to exploit them for performance enhancement are discussed. The optimized code is validated by conducting numerical simulations of zero-pressure gradient turbulent boundary layers at high Reynolds numbers and by comparing simulated turbulent statistics to those reported in previous studies. With the details of optimization strategies and validation processes, this work can serve as a practical guideline for acceleration of large-scale and precise DNS with many-core computing.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Literature
1.
2.
go back to reference Smits AJ, McKeon BJ, Marusic I (2011) High-reynolds number wall turbulence. Ann Rev Fluid Mech 43:353CrossRef Smits AJ, McKeon BJ, Marusic I (2011) High-reynolds number wall turbulence. Ann Rev Fluid Mech 43:353CrossRef
3.
go back to reference Hanjalić K, Launder B (1972) A reynolds stress model of turbulence and its application to thin shear flows. J Fluid Mech 52:609CrossRef Hanjalić K, Launder B (1972) A reynolds stress model of turbulence and its application to thin shear flows. J Fluid Mech 52:609CrossRef
4.
go back to reference Speziale CG, Sarkar S, Gatski TB (1991) Modelling the pressure-strain correlation of turbulence: an invariant dynamical systems approach. J Fluid Mech 227:245CrossRef Speziale CG, Sarkar S, Gatski TB (1991) Modelling the pressure-strain correlation of turbulence: an invariant dynamical systems approach. J Fluid Mech 227:245CrossRef
5.
go back to reference Deardorff JW (1970) A numerical study of three-dimensional turbulent channel flow at large Reynolds numbers. J Fluid Mech 41:453CrossRef Deardorff JW (1970) A numerical study of three-dimensional turbulent channel flow at large Reynolds numbers. J Fluid Mech 41:453CrossRef
6.
go back to reference Germano M, Piomelli U, Moin P, Cabot WH (1991) A dynamic subgrid-scale eddy viscosity model. Phys Fluids 3:1760CrossRef Germano M, Piomelli U, Moin P, Cabot WH (1991) A dynamic subgrid-scale eddy viscosity model. Phys Fluids 3:1760CrossRef
7.
go back to reference Kim J, Moin P, Moser RD (1987) Turbulence statistics in fully developed channel flow at low Reynolds number. J Fluid Mech 177:133CrossRef Kim J, Moin P, Moser RD (1987) Turbulence statistics in fully developed channel flow at low Reynolds number. J Fluid Mech 177:133CrossRef
8.
go back to reference Lee M, Moser RD (2015) Direct numerical simulation of turbulent channel flow up to \(Re_{\tau } \approx 5200\). J Fluid Mech 774:395CrossRef Lee M, Moser RD (2015) Direct numerical simulation of turbulent channel flow up to \(Re_{\tau } \approx 5200\). J Fluid Mech 774:395CrossRef
9.
go back to reference Ahn J, Lee JH, Lee J, Kang JH, Sung HJ (2015) Direct numerical simulation of a 30R long turbulent pipe flow at \(Re_{\tau } {\approx } 3008\). Phys Fluids 27:065110CrossRef Ahn J, Lee JH, Lee J, Kang JH, Sung HJ (2015) Direct numerical simulation of a 30R long turbulent pipe flow at \(Re_{\tau } {\approx } 3008\). Phys Fluids 27:065110CrossRef
10.
go back to reference Hamilton JM, Kim J, Waleffe F (1995) Regeneration mechanisms of near-wall turbulence structures. J Fluid Mech 287:317CrossRef Hamilton JM, Kim J, Waleffe F (1995) Regeneration mechanisms of near-wall turbulence structures. J Fluid Mech 287:317CrossRef
12.
go back to reference Lozano-Durán A, Jiménez J (2014) Effect of the computational domain on direct simulations of turbulent channels up to Re\(_{\tau }{\approx }\)4200. Phys Fluids 26:011702CrossRef Lozano-Durán A, Jiménez J (2014) Effect of the computational domain on direct simulations of turbulent channels up to Re\(_{\tau }{\approx }\)4200. Phys Fluids 26:011702CrossRef
13.
go back to reference del Álamo JC, Jiménez J (2009) Estimation of turbulent convection velocities and correlations to Taylor’s approximation. J Fluid Mech 640:5MathSciNetCrossRef del Álamo JC, Jiménez J (2009) Estimation of turbulent convection velocities and correlations to Taylor’s approximation. J Fluid Mech 640:5MathSciNetCrossRef
14.
go back to reference Monty JP, Stewart RCWJA, Chong MS (2007) Large-scale features in turbulent pipe and channel flows. J Fluid Mech 589:147CrossRef Monty JP, Stewart RCWJA, Chong MS (2007) Large-scale features in turbulent pipe and channel flows. J Fluid Mech 589:147CrossRef
15.
go back to reference Lee JH, Sung HJ (2013) Comparison of very-large-scale motions of turbulent pipe and boundary layer simulations. Phys Fluids 25:045103CrossRef Lee JH, Sung HJ (2013) Comparison of very-large-scale motions of turbulent pipe and boundary layer simulations. Phys Fluids 25:045103CrossRef
16.
go back to reference Zagarola MV, Smits AJ (1998) Mean-flow scaling of turbulent pipe flow. J Fluid Mech 373:33CrossRef Zagarola MV, Smits AJ (1998) Mean-flow scaling of turbulent pipe flow. J Fluid Mech 373:33CrossRef
17.
go back to reference Lee M, Malaya N, Moser RD (2013) Petascale direct numerical simulation of turbulent channel flow on up to 786K cores, In SC ’13 Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis . https://doi.org/10.1145/2503210.2503298 Lee M, Malaya N, Moser RD (2013) Petascale direct numerical simulation of turbulent channel flow on up to 786K cores, In SC ’13 Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis . https://​doi.​org/​10.​1145/​2503210.​2503298
18.
19.
go back to reference Lee M, Ulerich R, Malaya N, Moser RD (2014) Experiences from leadership computing in simulations of turbulent fluid flows. Comput Sci Eng 16:24CrossRef Lee M, Ulerich R, Malaya N, Moser RD (2014) Experiences from leadership computing in simulations of turbulent fluid flows. Comput Sci Eng 16:24CrossRef
20.
go back to reference Du P, Weber R, Luszczek P, Tomov S, Peterson G, Dongarra J (2012) From CUDA to OpenCL: towards a performance-portable solution for multi-platform GPU programming. Parallel Comput 38:391CrossRef Du P, Weber R, Luszczek P, Tomov S, Peterson G, Dongarra J (2012) From CUDA to OpenCL: towards a performance-portable solution for multi-platform GPU programming. Parallel Comput 38:391CrossRef
22.
go back to reference Thibault JC, Senocak I (2009) CUDA implementation of a Navier-Stokes solver on multi-GPU desktop platforms for incompressible flows, In 47th AIAA Aerospace Sciences Meeting including The New Horizons Forum and Aerospace Exposition. https://doi.org/10.2514/6.2009-758 Thibault JC, Senocak I (2009) CUDA implementation of a Navier-Stokes solver on multi-GPU desktop platforms for incompressible flows, In 47th AIAA Aerospace Sciences Meeting including The New Horizons Forum and Aerospace Exposition. https://​doi.​org/​10.​2514/​6.​2009-758
24.
go back to reference Jespersen DC (2010) Acceleration of a CFD code with a GPU. Sci Program 18:193 Jespersen DC (2010) Acceleration of a CFD code with a GPU. Sci Program 18:193
25.
go back to reference Mudigere D, Sridharan S, Deshpande A, Park J, Heinecke A, Smelyanskiy M, Kaul B, Dubey P, Kaushik D, Keyes D (2015) Exploring shared-memory optimizations for an unstructured mesh CFD application on modern parallel systems, In IEEE 29th Int. Parallel and Distributed Processing Symposium. https://doi.org/10.1109/IPDPS.2015.114 Mudigere D, Sridharan S, Deshpande A, Park J, Heinecke A, Smelyanskiy M, Kaul B, Dubey P, Kaushik D, Keyes D (2015) Exploring shared-memory optimizations for an unstructured mesh CFD application on modern parallel systems, In IEEE 29th Int. Parallel and Distributed Processing Symposium. https://​doi.​org/​10.​1109/​IPDPS.​2015.​114
26.
go back to reference Economon TD, Palacios F, Alonso JJ, Bansal G, Mudigere D, Deshpande A, Heinecke A, Smelyanskiy M (2015) Towards high-performance optimizations of the unstructured open-source SU2 suite, In AIAA Infotech @ Aerospace (2015). https://doi.org/10.2514/6.2015-1949 Economon TD, Palacios F, Alonso JJ, Bansal G, Mudigere D, Deshpande A, Heinecke A, Smelyanskiy M (2015) Towards high-performance optimizations of the unstructured open-source SU2 suite, In AIAA Infotech @ Aerospace (2015). https://​doi.​org/​10.​2514/​6.​2015-1949
27.
go back to reference Zhu X, Phillips E, Spandan V, Donners J, Ruetsch G, Romero J, Ostilla-Mónico R, Yang Y, Lohse D, Verzicco R, Fatica M, Stevens RJ (2018) AFiD-GPU: a versatile Navier-Stokes solver for wall-bounded turbulent flows on GPU clusters. Comput Phys Commun 229:199CrossRef Zhu X, Phillips E, Spandan V, Donners J, Ruetsch G, Romero J, Ostilla-Mónico R, Yang Y, Lohse D, Verzicco R, Fatica M, Stevens RJ (2018) AFiD-GPU: a versatile Navier-Stokes solver for wall-bounded turbulent flows on GPU clusters. Comput Phys Commun 229:199CrossRef
28.
go back to reference Bernardini M, Modesti D, Salvadore F, Pirozzoli S (2020) STREAmS: a high-fidelity accelerated solver for direct numerical simulation of compressible turbulent flow, STREAmS: a high-fidelity accelerated solver for direct numerical simulation of compressible turbulent flow Bernardini M, Modesti D, Salvadore F, Pirozzoli S (2020) STREAmS: a high-fidelity accelerated solver for direct numerical simulation of compressible turbulent flow, STREAmS: a high-fidelity accelerated solver for direct numerical simulation of compressible turbulent flow
29.
go back to reference Costa P, Phillips E, Brandt L, Fatica M (2021) GPU acceleration of CaNS for massively-parallel direct numerical simulations of canonical fluid flows. Comput Math Appl 81:502MathSciNetCrossRef Costa P, Phillips E, Brandt L, Fatica M (2021) GPU acceleration of CaNS for massively-parallel direct numerical simulations of canonical fluid flows. Comput Math Appl 81:502MathSciNetCrossRef
30.
go back to reference Meng Q, Humphrey A, Schmidt J, Berzins M (2013) Preliminary experiences with the uintah framework on Intel Xeon Phi and stampede, Proceedings of the Conference on Extreme Science and Engineering Discovery Environment: Gateway to. Discovery. https://doi.org/10.1145/2484762.2484779 Meng Q, Humphrey A, Schmidt J, Berzins M (2013) Preliminary experiences with the uintah framework on Intel Xeon Phi and stampede, Proceedings of the Conference on Extreme Science and Engineering Discovery Environment: Gateway to. Discovery. https://​doi.​org/​10.​1145/​2484762.​2484779
31.
go back to reference Yoon M, Ahn J, Hwang J, Sung HJ (2016) Contribution of velocity-vorticity correlations to the frictional drag in wall-bounded turbulent flows. Phys Fluids 28:081702CrossRef Yoon M, Ahn J, Hwang J, Sung HJ (2016) Contribution of velocity-vorticity correlations to the frictional drag in wall-bounded turbulent flows. Phys Fluids 28:081702CrossRef
32.
go back to reference Hwang J, Sung HJ (2017) Influence of large-scale motions on the frictional drag in a turbulent boundary layer. J Fluids Mech 829:751MathSciNetCrossRef Hwang J, Sung HJ (2017) Influence of large-scale motions on the frictional drag in a turbulent boundary layer. J Fluids Mech 829:751MathSciNetCrossRef
33.
go back to reference Yoon M, Hwang J, Sung HJ (2018) Contribution of large-scale motions to the skin friction in a moderate adverse pressure gradient turbulent boundary layer. J Fluids Mech 848:288MathSciNetCrossRef Yoon M, Hwang J, Sung HJ (2018) Contribution of large-scale motions to the skin friction in a moderate adverse pressure gradient turbulent boundary layer. J Fluids Mech 848:288MathSciNetCrossRef
35.
go back to reference Kim K, Baek SJ, Sung HJ (2002) An implicit velocity decoupling procedure for the incompressible Navier-Stokes equations. Int J Numer Meth Fluids 38:125CrossRef Kim K, Baek SJ, Sung HJ (2002) An implicit velocity decoupling procedure for the incompressible Navier-Stokes equations. Int J Numer Meth Fluids 38:125CrossRef
36.
go back to reference Lam MD, Rothberg EE, Wolf ME (1991) The cache performance and optimizations of blocked algorithms. ACM SIGPLAN Not 26:63CrossRef Lam MD, Rothberg EE, Wolf ME (1991) The cache performance and optimizations of blocked algorithms. ACM SIGPLAN Not 26:63CrossRef
37.
go back to reference Schlatter P, Örlü R (2010) Assessment of direct numerical simulation data of turbulent boundary layers. J Fluid Mech 659:116CrossRef Schlatter P, Örlü R (2010) Assessment of direct numerical simulation data of turbulent boundary layers. J Fluid Mech 659:116CrossRef
38.
go back to reference Jacobs RG, Durbin PA (2001) Simulations of bypass transition. J Fluid Mech 428:185CrossRef Jacobs RG, Durbin PA (2001) Simulations of bypass transition. J Fluid Mech 428:185CrossRef
39.
go back to reference Wu X, Moin P, Wallace JM, Skarda J, Lozano-Durán A, Hickey JP (2017) Transitional-turbulent spots and turbulent-turbulent spots in boundary layers. Proc Natl Acad Sci 114:5292CrossRef Wu X, Moin P, Wallace JM, Skarda J, Lozano-Durán A, Hickey JP (2017) Transitional-turbulent spots and turbulent-turbulent spots in boundary layers. Proc Natl Acad Sci 114:5292CrossRef
Metadata
Title
High-performance simulations of turbulent boundary layer flow using Intel Xeon Phi many-core processors
Authors
Ji-Hoon Kang
Jinyul Hwang
Hyung Jin Sung
Hoon Ryu
Publication date
11-02-2021
Publisher
Springer US
Published in
The Journal of Supercomputing / Issue 9/2021
Print ISSN: 0920-8542
Electronic ISSN: 1573-0484
DOI
https://doi.org/10.1007/s11227-021-03642-6

Other articles of this Issue 9/2021

The Journal of Supercomputing 9/2021 Go to the issue

Premium Partner