Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
Hardware-Based Efficiency Advances in the EXA-DUNE Project Read chapter Read first chapter

2016 | OriginalPaper | Chapter

Hybrid Parallel Multigrid Methods for Geodynamical Simulations

Authors : Simon Bauer, Hans-Peter Bunge, Daniel Drzisga, Björn Gmeiner, Markus Huber, Lorenz John, Marcus Mohr, Ulrich Rüde, Holger Stengel, Christian Waluga, Jens Weismüller, Gerhard Wellein, Markus Wittmann, Barbara Wohlmuth

Published in: Software for Exascale Computing - SPPEXA 2013-2015

Publisher: Springer International Publishing

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Even on modern supercomputer architectures, Earth mantle simulations are so compute intensive that they are considered grand challenge applications. The dominating roadblocks in this branch of Geophysics are model complexity and uncertainty in parameters and data, e.g., rheology and seismically imaged mantle heterogeneity, as well as the enormous space and time scales that must be resolved in the computational models. This article reports on a massively parallel all-at-once multigrid solver for the Stokes system as it arises in mantle convection models. The solver employs the hierarchical hybrid grids framework and demonstrates that a system with coupled velocity components and with more than a trillion (1. 7 ⋅ 1012) degrees of freedom can be solved in about 1,000 s using 40,960 compute cores of JUQUEEN. The simulation framework is used to investigate the influence of asthenosphere thickness and viscosity on upper mantle velocities in a static scenario. Additionally, results for a time-dependent simulation with a time-variable temperature-dependent viscosity model are presented.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
previous chapter Handling Silent Data Corruption with the Sparse Grid Combination Technique
next chapter Partitioned Fluid–Structure–Acoustics Interaction on Distributed Data: Coupling via preCICE
Footnotes
1
listed top 9 of the TOP500 list, Nov. 2015.
 
Literature
1.
Bangerth, W., Burstedde, C., Heister, T., Kronbichler, M.: Algorithms and data structures for massively parallel generic adaptive finite element codes. ACM Trans. Math. Soft. 38 (2), 14:1–14:28 (2011)
2.
Bank, R.E., Welfert, B.D., Yserentant, H.: A class of iterative methods for solving saddle point problems. Numer. Math. 56 (7), 645–666 (1990)MathSciNetCrossRefMATH
3.
Baumgardner, J.R.: Three-dimensional treatment of convective flow in the Earth’s mantle. J. Stat. Phys. 39 (5/6), 501–511 (1985)CrossRef
4.
Becker, T.W., Boschi, L.: A comparison of tomographic and geodynamic mantle models. Geochem. Geophy. Geosy. 3, 1525–2027 (2002)CrossRef
5.
Bergen, B., Gradl, T., Rüde, U., Hülsemann, F.: A massively parallel multigrid method for finite elements. Comput. Sci. Eng. 8 (6), 56–62 (2006)CrossRef
6.
Bergen, B., Wellein, G., Hülsemann, F., Rüde, U.: Hierarchical hybrid grids: achieving TERAFLOP performance on large scale finite element simulations. Int. J. Parallel Emergent Distrib. Syst. 22 (4), 311–329 (2007)MathSciNetCrossRef
7.
Brandt, A.: Guide to multigrid development. In: Multigrid methods, pp. 220–312. Springer, Berlin/Heidelberg (1982). Republished as: Multigrid Techniques: 1984 guide with applications to fluid dynamics, revised edition, SIAM, 2011
8.
Brandt, A., Dinar, N.: Multigrid solutions to elliptic flow problems. In: Numerical methods for partial differential equations (Proc. Adv. Sem., Math. Res. Center, Univ. Wisconsin, Madison, Wis., 1978), Publ. Math. Res. Center Univ. Wisconsin, vol. 42, pp. 53–147. Academic Press, New York/London (1979)
9.
10.
11.
Bunge, H.P., Baumgardner, J.R.: Mantle convection modeling on parallel virtual machines. Comput. Phys. 9 (2), 207–215 (1995)CrossRef
12.
13.
Bunge, H.P., Richards, M.A., Baumgardner, J.R.: A sensitivity study of three-dimensional spherical mantle convection at 108 Rayleigh number: effects of depth-dependent viscosity, heating mode, and an endothermic phase change. J. Geophys. Res. 102, 11991–12007 (1997)CrossRef
14.
15.
Burstedde, C., Stadler, G., Alisic, L., Wilcox, L.C., Tan, E., Gurnis, M., Ghattas, O.: Large-scale adaptive mantle convection simulation. Geophys. J. Internat. 192 (3), 889–906 (2013)CrossRef
16.
Burstedde, C., Wilcox, L.C., Ghattas, O.: p4est: Scalable algorithms for parallel adaptive mesh refinement on forests of octrees. SIAM J. Sci. Comp. 33 (3), 1103–1133 (2011)
17.
CIG – Computational Infrastructure for Geodynamics: ASPECT: Advanced Solver for Problems in Earth’s ConvecTion, User Manual (2015), version 1.3
18.
19.
Davies, D.R., Davies, J.H., Bollada, P.C., Hassan, O., Morgan, K., Nithiarasu, P.: A hierarchical mesh refinement technique for global 3-D spherical mantle convection modelling. Geosci. Model Dev. 6 (4), 1095–1107 (2013)CrossRef
20.
Davies, D.R., Goes, S., Davies, J.H., Schuberth, B.S.A., Bunge, H.P., Ritsema, J.: Reconciling dynamic and seismic models of Earth’s lower mantle: the dominant role of thermal heterogeneity. Earth Planet. Sci. Lett. 353–354 (1), 253–269 (2012)CrossRef
21.
Dziewonski, A.M., Anderson, D.L.: Preliminary reference Earth model. Phys. Earth Plan. Int. 25, 297–356 (1981)CrossRef
22.
Elman, H.C., Silvester, D.J., Wathen, A.J.: Finite Elements and Fast Iterative Solvers: With Applications in Incompressible Fluid Dynamics. Oxford University Press, New York (2005)MATH
23.
Engelman, M.S., Sani, R.L., Gresho, P.M.: The implementation of normal and/or tangential boundary conditions in finite element codes for incompressible fluid flow. Int. J. Numer. Methods Fluids 2 (3), 225–238 (1982)MathSciNetCrossRefMATH
24.
Fichtner, A., Kennett, B.L.N., Igel, H., Bunge, H.P.: Full seismic waveform tomography for upper-mantle structure in the Australasian region using adjoint methods. Geophys. J. Int. 179 (3), 1703–1725 (2009)CrossRef
25.
Gaspar, F.J., Notay, Y., Oosterlee, C.W., Rodrigo, C.: A simple and efficient segregated smoother for the discrete Stokes equations. SIAM J. Sci. Comput. 36 (3), A1187–A1206 (2014)MathSciNetCrossRefMATH
26.
Girault, V., Raviart, P.A.: Finite Element Methods for Navier-Stokes Equations. Springer, New York (1986)CrossRefMATH
27.
Gmeiner, B., Huber, M., John, L., Rüde, U., Waluga, C., Wohlmuth, B.: Massively parallel large scale stokes flow simulation. In: Binder, K., Müller, M., Kremer, M., Schnurpfeil, A. (eds.) NIC Symposium 2016. Schriften des Forschungszentrums Jülich, NIC Series, vol. 48, pp. 333–341. ISBN:978-3-95806-109-5
28.
Gmeiner, B., Huber, M., John, L., Rüde, U., Wohlmuth, B.: A quantitative performance analysis for Stokes solvers at the extreme scale (submitted, arXiv:1511.02134)
29.
Gmeiner, B., Rüde, U., Stengel, H., Waluga, C., Wohlmuth, B.: Performance and scalability of hierarchical hybrid multigrid solvers for stokes systems. SIAM J. Sci. Comput. 37 (2), C143–C168 (2015)MathSciNetCrossRefMATH
30.
Gmeiner, B., Rüde, U., Stengel, H., Waluga, C., Wohlmuth, B.: Towards textbook efficiency for parallel multigrid. Numer. Math. Theory Methods Appl. 8, 22–46 (2015)MathSciNetCrossRefMATH
31.
Gmeiner, B., Waluga, C., Wohlmuth, B.: Local mass-corrections for continuous pressure approximations of incompressible flow. SIAM J. Numer. Anal. 52 (6), 2931–2956 (2014)MathSciNetCrossRefMATH
32.
Grand, S.P., van der Hilst, R.D., Widiyantoro, S.: Global seismic tomography: a snapshot of convection in the earth. GSA Today 7, 1–7 (1997)
33.
Hager, G., Treibig, J., Habich, J., Wellein, G.: Exploring performance and power properties of modern multi-core chips via simple machine models. Concurr. Comput. 28, 1–2 (2014)
34.
Hartley, R.A., Roberts, G.G., White, N., Richardson, C.: Transient convective uplift of an ancient buried landscape. Nat. Geosci. 4, 562–565 (2011)CrossRef
35.
36.
Höink, T., Lenardic, A.: Three-dimensional mantle convection simulations with a low-viscosity asthenosphere and the relationship between heat flow and the horizontal length scale of convection. Geophys. Res. Lett. 35, L10304 (2008)CrossRef
37.
Huber, M., Gmeiner, B., Rüde, U., Wohlmuth, B.: Resilience for multigrid software at the extreme scale (preprint, arXiv:1506.06185)
38.
Huber, M., John, L., Pustejovska, P., Rüde, U., Waluga, C., Wohlmuth, B.: Solution Techniques for the Stokes System: a priori and a posteriori modifications, resilient algorithms. In Proceedings of the ICIAM, Beijing (2015). arXiv:151105759
39.
Mitrovica, J.X.: Haskell [1935] revisited. J. Geophys. Res. 101, 555–569 (1996)CrossRef
40.
Müller, R.D., Sdrolias, M., Gaina, C., Roest, W.R.: Age, spreading rates, and spreading asymmetry of the world’s ocean crust. Geochem. Geophy. Geosy. 9, 1525–2027 (2008)CrossRef
41.
Oeser, J., Bunge, H.P., Mohr, M.: Cluster Design in the Earth Sciences: TETHYS. In: Gerndt, M., Kranzlmüller, D. (eds.) High Performance Computing and Communications – Second International Conference, HPCC 2006, Munich. Lecture Notes in Computer Science, vol. 4208, pp. 31–40. Springer (2006). http://​www.​springerlink.​com/​content/​l18628n708k11127​
42.
Parnell-Turner, R., White, N., Henstock, T., Murton, B., Maclennan, J., Jones, S.M.: A continuous 55 million year record of transient mantle plume activity beneath Iceland. Nat. Geosci. 7, 914–919 (2014)CrossRef
43.
Resovsky, J., Trampert, J.: Using probabilistic seismic tomography to test mantle velocity–density relationships. Earth Planet. Sci. Lett. 215 (1), 121–134 (2003)CrossRef
44.
Ricard, Y.: Physics of mantle convection. In: Schubert, G. (ed.) Treatise on Geophysics, vol. 7. Elsevier, Amsterdam (2007)
45.
Ritsema, J., von Heijst, H.J., Woodhouse, J.H.: Global transition zone tomography. J. Geophys. Res. 109, B02302 (2004)CrossRef
46.
Rudi, J., Malossi, A.C.I., Isaac, T., Stadler, G., Gurnis, M., Staar, P.W.J., Ineichen, Y., Bekas, C., Curioni, A., Ghattas, O.: An Extreme-scale Implicit Solver for Complex PDEs: Highly Heterogeneous Flow in Earth’s Mantle. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC ’15), pp. 5:1–5:12. ACM, New York (2015). http://​doi.​acm.​org/​10.​1145/​2807591.​2807675
47.
48.
Seton, M., Müller, R.D., Zahirovic, S., Gaina, C., Torsvik, T.H., Shephard, G., Talsma, A., Gurnis, M., Turner, M., Maus, S., Chandler, M.: Global continental and ocean basin reconstructions since 200 ma. Earth-Sci. Rev. 113, 212–270 (2012)CrossRef
49.
Stixrude, L., Lithgow-Bertelloni, C.: Thermodynamics of mantle minerals – I. Physical properties. Geophys. J. Int. 162, 610–632 (2005)CrossRef
50.
Sundar, H., Stadler, G., Biros, G.: Comparison of multigrid algorithms for high-order continuous finite element discretizations. Numer. Linear Algebra Appl. 22 (4), 664–680 (2015)MathSciNetCrossRefMATH
51.
Tackley, P.J.: Effects of strongly variable viscosity on three-dimensional compressible convection in planetary mantles. J. Geophys. Res. 101, 3311–3332 (1996)CrossRef
52.
Tackley, P.J.: Mantle convection and plate tectonics: toward an integrated physical and chemical theory. Science 16, 2002–2007 (2000)CrossRef
53.
Tackley, P.J., Stevenson, D.J., Glatzmaier, G.A., Schubert, G.: Effects of multiple phase transitions in a three-dimensional spherical model of convection in earth’s mantle. J. Geophys. Res. 99 (B8), 15877–15901 (1994)CrossRef
54.
Tan, E., Choi, E., Thoutireddy, P., Gurnis, M., Aivazis, M.: GeoFramework: coupling multiple models of mantle convection within a computational framework. Geochem. Geophy. Geosy. 7 (6), Q06001 (2006)CrossRef
55.
Urquiza, J.M., Garon, A., Farinas, M.I.: Weak imposition of the slip boundary condition on curved boundaries for Stokes flow. J. Comput. Phys. 256, 748–767 (2014)MathSciNetCrossRef
56.
Verfürth, R.: Finite element approximation of incompressible Navier-Stokes equations with slip boundary condition. Numer. Math. 50 (6), 697–721 (1987)MathSciNetCrossRefMATH
57.
58.
Waluga, C., Wohlmuth, B., Rüde, U.: Mass-corrections for the conservative coupling of flow and transport on collocated meshes. J. Comp. Phys. 305, 319–332 (2016)MathSciNetCrossRef
59.
Weismüller, J., Gmeiner, B., Ghelichkhan, S., Huber, M., John, L., Wohlmuth, B., Rüde, U., Bunge, H.P.: Fast asthenosphere motion in high-resolution global mantle flow models. Geophys. Res. Lett. 42 (18), 7429–7435 (2015)CrossRef
60.
Williams, S.W., Waterman, A., Patterson, D.A.: Roofline: an insightful visual performance model for floating-point programs and multicore architectures. Tech. Rep. UCB/EECS-2008-134, EECS Department, University of California, Berkeley (Oct 2008)
61.
Zhong, S., McNamara, A., Tan, E., Moresi, L., Gurnis, M.: A benchmark study on mantle convection in a 3-D spherical shell using CitcomS. Geochem. Geophy. Geosy. 9, Q10017 (2008)CrossRef
62.
Zhong, S., Zuber, M.T., Moresi, L., Gurnis, M.: The role of temperature-dependent viscosity and surface plates in spherical shell models of mantle convection. J. Geophys. Res. 105 (B5), 11063–11082 (2000)CrossRef
63.
Zulehner, W.: Analysis of iterative methods for saddle point problems: a unified approach. Math. Comput. 71 (238), 479–505 (2002)MathSciNetCrossRefMATH
Metadata
Title
Hybrid Parallel Multigrid Methods for Geodynamical Simulations
Authors
Simon Bauer
Hans-Peter Bunge
Daniel Drzisga
Björn Gmeiner
Markus Huber
Lorenz John
Marcus Mohr
Ulrich Rüde
Holger Stengel
Christian Waluga
Jens Weismüller
Gerhard Wellein
Markus Wittmann
Barbara Wohlmuth
Copyright Year
2016
Book
Software for Exascale Computing - SPPEXA 2013-2015

Print ISBN: 978-3-319-40526-1

Electronic ISBN: 978-3-319-40528-5

Copyright Year: 2016

DOI
https://doi.org/10.1007/978-3-319-40528-5_10

Premium Partner

    Image Credits