Skip to main content
Top
Published in: Journal of Scientific Computing 1/2017

26-07-2016

Large-Scale Optimization-Based Non-negative Computational Framework for Diffusion Equations: Parallel Implementation and Performance Studies

Published in: Journal of Scientific Computing | Issue 1/2017

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

It is well-known that the standard Galerkin formulation, which is often the formulation of choice under the finite element method for solving self-adjoint diffusion equations, does not meet maximum principles and the non-negative constraint for anisotropic diffusion equations. Recently, optimization-based methodologies that satisfy maximum principles and the non-negative constraint for steady-state and transient diffusion-type equations have been proposed. To date, these methodologies have been tested only on small-scale academic problems. The purpose of this paper is to systematically study the performance of the non-negative methodology in the context of high performance computing (HPC). PETSc and TAO libraries are, respectively, used for the parallel environment and optimization solvers. For large-scale problems, it is important for computational scientists to understand the computational performance of current algorithms available in these scientific libraries. The numerical experiments are conducted on the state-of-the-art HPC systems, and a single-core performance model is used to better characterize the efficiency of the solvers. Our studies indicate that the proposed non-negative computational framework for diffusion-type equations exhibits excellent strong scaling for real-world large-scale problems.

Graphical Abstract

This figure shows the fate of chromium after 180 days using the single-field Galerkin formulation. The white regions indicate the violation of the non-negative constraint.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Adams, R.J., Fournier, J.J.F.: Sobolev Spaces, 2nd edn. Academic press, Amsterdam (2003)MATH Adams, R.J., Fournier, J.J.F.: Sobolev Spaces, 2nd edn. Academic press, Amsterdam (2003)MATH
2.
go back to reference Ayachit, U.: The ParaView Guide: A Parallel Visualization Application. Kitware, Clifton Park (2015) Ayachit, U.: The ParaView Guide: A Parallel Visualization Application. Kitware, Clifton Park (2015)
3.
go back to reference Balay, S., Abhyankar, S., Adams, M.F., Brown, J., Brune, P., Buschelman, K., Eijkhout, V., Gropp, W.D., Kaushik, D., Knepley, M.G., McInnes, L.C., Rupp, K., Smith, B.F., Zampini, S., Zhang, H.: PETSc users manual. Technical Report ANL-95/11 - Revision 3.6, Argonne National Laboratory, 2015 Balay, S., Abhyankar, S., Adams, M.F., Brown, J., Brune, P., Buschelman, K., Eijkhout, V., Gropp, W.D., Kaushik, D., Knepley, M.G., McInnes, L.C., Rupp, K., Smith, B.F., Zampini, S., Zhang, H.: PETSc users manual. Technical Report ANL-95/11 - Revision 3.6, Argonne National Laboratory, 2015
4.
go back to reference Benson, S.J., Munson, T.S.: Flexible complementary solvers for large-scale applications. Optim. Methods Softw. 21, 155–168 (2006)MathSciNetCrossRefMATH Benson, S.J., Munson, T.S.: Flexible complementary solvers for large-scale applications. Optim. Methods Softw. 21, 155–168 (2006)MathSciNetCrossRefMATH
5.
go back to reference Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press, Cambridge, UK (2004)CrossRefMATH Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press, Cambridge, UK (2004)CrossRefMATH
6.
go back to reference Childs, H., Brugger, E., Whitlock, B., Meredith, J., Ahern, S., Pugmire, D., Biagas, K., Miller, M., Harrison, C., Weber, G.H., Krishnan, H., Fogal, T., Sanderson, A., Garth, C., Bethel, E.W., Camp, D., Rübel, O., Durant, M., Favre, J.M., Navrátil, P.: VisIt: An end-user tool for visualizing and analyzing very large data. In: High Performance Visualization-Enabling Extreme-Scale Scientific Insight, (pp. 357–372) (2012). https://wci.llnl.gov/simulation/computer-codes/visit Childs, H., Brugger, E., Whitlock, B., Meredith, J., Ahern, S., Pugmire, D., Biagas, K., Miller, M., Harrison, C., Weber, G.H., Krishnan, H., Fogal, T., Sanderson, A., Garth, C., Bethel, E.W., Camp, D., Rübel, O., Durant, M., Favre, J.M., Navrátil, P.: VisIt: An end-user tool for visualizing and analyzing very large data. In: High Performance Visualization-Enabling Extreme-Scale Scientific Insight, (pp. 357–372) (2012). https://​wci.​llnl.​gov/​simulation/​computer-codes/​visit
7.
go back to reference Ciarlet, P.G., Raviart, P.-A.: Maximum principle and uniform convergence for the finite element method. Comput. Methods Appl. Methods Eng. 2, 17–31 (1973)MathSciNetCrossRefMATH Ciarlet, P.G., Raviart, P.-A.: Maximum principle and uniform convergence for the finite element method. Comput. Methods Appl. Methods Eng. 2, 17–31 (1973)MathSciNetCrossRefMATH
8.
go back to reference Evans, L.C.: Partial Differential Equations. American Mathematical Society, Providence (1998)MATH Evans, L.C.: Partial Differential Equations. American Mathematical Society, Providence (1998)MATH
9.
go back to reference Facchinei, F., Pang, J.-S.: Finite-Dimensional Variational Inequalities and Complementarity Problems. Springer, New York (2003)MATH Facchinei, F., Pang, J.-S.: Finite-Dimensional Variational Inequalities and Complementarity Problems. Springer, New York (2003)MATH
10.
go back to reference Graser, C., Kornhuber, R.: Multigrid methods for obstacle problems. J. Comput. Math. 27, 1–44 (2009)MathSciNetMATH Graser, C., Kornhuber, R.: Multigrid methods for obstacle problems. J. Comput. Math. 27, 1–44 (2009)MathSciNetMATH
11.
go back to reference Gropp, W.D., Kaushik, D.K., Keyes, D.E., Smith, B.F.: Toward realistic performance bounds for implicit CFD codes. In Proceedings of Parallel CFD ‘99, pp. 233–240. Elsevier, (1999) Gropp, W.D., Kaushik, D.K., Keyes, D.E., Smith, B.F.: Toward realistic performance bounds for implicit CFD codes. In Proceedings of Parallel CFD ‘99, pp. 233–240. Elsevier, (1999)
12.
go back to reference Hammond, G.E., Lichtner, P.C.: Field-scale model for the natural attenuation of uranium at the Hanford 300 Area using high-performance computing. Water Resour. Res. 46, W09602 (2010)CrossRef Hammond, G.E., Lichtner, P.C.: Field-scale model for the natural attenuation of uranium at the Hanford 300 Area using high-performance computing. Water Resour. Res. 46, W09602 (2010)CrossRef
13.
go back to reference Harp, D.R., Vesselinov, V.V.: Contaminant remediation decision analysis using information gap theory. Stoch. Environ. Res. Risk Assess. 27(1), 159–168 (2013)CrossRef Harp, D.R., Vesselinov, V.V.: Contaminant remediation decision analysis using information gap theory. Stoch. Environ. Res. Risk Assess. 27(1), 159–168 (2013)CrossRef
14.
go back to reference Heikoop, J.M., Johnson, T.M., Birdsell, K.H., Longmire, P., Hickmott, D.D., Jacobs, E.P., Broxton, D.E., Katzman, D., Vesselinov, V.V., Ding, M., Vanimana, D.T., Reneaua, S.L., Goering, T.J., Glessnerb, J., Basu, A.: Isotopic evidence for reduction of anthropogenic hexavalent chromium in Los Alamos National Laboratory groundwater. Chem. Geol. 373, 1–9 (2014)CrossRef Heikoop, J.M., Johnson, T.M., Birdsell, K.H., Longmire, P., Hickmott, D.D., Jacobs, E.P., Broxton, D.E., Katzman, D., Vesselinov, V.V., Ding, M., Vanimana, D.T., Reneaua, S.L., Goering, T.J., Glessnerb, J., Basu, A.: Isotopic evidence for reduction of anthropogenic hexavalent chromium in Los Alamos National Laboratory groundwater. Chem. Geol. 373, 1–9 (2014)CrossRef
15.
go back to reference Hjelmstad, K.D.: Fundamentals of Structural Mechanics, 2nd edn. Springer, New York (2005) Hjelmstad, K.D.: Fundamentals of Structural Mechanics, 2nd edn. Springer, New York (2005)
16.
go back to reference Karra, S., Painter, S.L., Lichtner, P.C.: Three-phase numerical model for subsurface hydrology in permafrost-affected regions (PFLOTRAN-ICE v1.0). Cryosphere 8(5), 1935–1950 (2014)CrossRef Karra, S., Painter, S.L., Lichtner, P.C.: Three-phase numerical model for subsurface hydrology in permafrost-affected regions (PFLOTRAN-ICE v1.0). Cryosphere 8(5), 1935–1950 (2014)CrossRef
17.
go back to reference Karypis, G., Kumar, V.: A fast and highly quality multilevel scheme for partitioning irregular graphs. SIAM J. Sci. Comput. 20, 359–392 (1999)MathSciNetCrossRefMATH Karypis, G., Kumar, V.: A fast and highly quality multilevel scheme for partitioning irregular graphs. SIAM J. Sci. Comput. 20, 359–392 (1999)MathSciNetCrossRefMATH
18.
go back to reference Kelkar, S., Lewis, K., Karra, S., Zyvoloski, G., Rapaka, S., Viswanathan, H., Mishra, P.K., Chu, S., Coblentz, D., Pawar, R.: A simulator for modeling coupled thermo-hydro-mechanical processes in subsurface geological media. Int. J. Rock Mech. Min. Sci. 70, 569–580 (2014) Kelkar, S., Lewis, K., Karra, S., Zyvoloski, G., Rapaka, S., Viswanathan, H., Mishra, P.K., Chu, S., Coblentz, D., Pawar, R.: A simulator for modeling coupled thermo-hydro-mechanical processes in subsurface geological media. Int. J. Rock Mech. Min. Sci. 70, 569–580 (2014)
19.
go back to reference Kinderlehrer, D., Stampacchia, G.: An Introduction to Variational Inequalities and Their Applications. SIAM Classics in Applied Mathematics, New York (2000)CrossRefMATH Kinderlehrer, D., Stampacchia, G.: An Introduction to Variational Inequalities and Their Applications. SIAM Classics in Applied Mathematics, New York (2000)CrossRefMATH
20.
go back to reference Kirby, R.C.: FIAT: Numerical Construction of Finite Element Basis Functions, Chapter 13. Springer, New York (2012) Kirby, R.C.: FIAT: Numerical Construction of Finite Element Basis Functions, Chapter 13. Springer, New York (2012)
21.
go back to reference Knepley, M.G., Brown, J., Rupp, K., Smith, B.F.: Achieving High Performance with Unified Residual Evaluation. arXiv:1309.1204, September (2013) Knepley, M.G., Brown, J., Rupp, K., Smith, B.F.: Achieving High Performance with Unified Residual Evaluation. arXiv:​1309.​1204, September (2013)
22.
go back to reference Knepley, M.G., Karpeev, D.A.: Mesh algorithms for PDE with vieve I: Mesh distribution. Sci. Program. 17, 215–230 (2009) Knepley, M.G., Karpeev, D.A.: Mesh algorithms for PDE with vieve I: Mesh distribution. Sci. Program. 17, 215–230 (2009)
23.
go back to reference Lange, M., Knepley, M.G., Gorman., G.J.: Flexible, scalable mesh and data management using PETSc DMPlex. In Proceedings of the 3rd International Conference on Exascale Applications and Software, EASC ‘15, pp. 71–76. University of Edinburgh, (2015) Lange, M., Knepley, M.G., Gorman., G.J.: Flexible, scalable mesh and data management using PETSc DMPlex. In Proceedings of the 3rd International Conference on Exascale Applications and Software, EASC ‘15, pp. 71–76. University of Edinburgh, (2015)
24.
go back to reference Lichtner, P.C., Hammond, G.E., Lu, C., Karra, S., Bisht, G., Andre, B., Mills, R.T., Kumar, J.: PFLOTRAN user manual: A massively parallel reactive flow and transport model for describing surface and subsurface processes. Technical Report Report No.: LA-UR-15-20403, Los Alamos National Laboratory, (2015) Lichtner, P.C., Hammond, G.E., Lu, C., Karra, S., Bisht, G., Andre, B., Mills, R.T., Kumar, J.: PFLOTRAN user manual: A massively parallel reactive flow and transport model for describing surface and subsurface processes. Technical Report Report No.: LA-UR-15-20403, Los Alamos National Laboratory, (2015)
25.
go back to reference Lichtner, P.C., Karra, S.: Modeling multiscale-multiphase-multicomponent reactive flows in porous media: Application to CO\(_2\) sequestration and enhanced geothermal energy using pflotran. In R. Al-Khoury and J. Bundschuh, editors, Computational Models for CO\(_2\) Geo-sequestration & Compressed Air Energy Storage, pp. 81–136. CRC Press, http://www.crcnetbase.com/doi/pdfplus/10.1201/b16790-6, (2014) Lichtner, P.C., Karra, S.: Modeling multiscale-multiphase-multicomponent reactive flows in porous media: Application to CO\(_2\) sequestration and enhanced geothermal energy using pflotran. In R. Al-Khoury and J. Bundschuh, editors, Computational Models for CO\(_2\) Geo-sequestration & Compressed Air Energy Storage, pp. 81–136. CRC Press, http://​www.​crcnetbase.​com/​doi/​pdfplus/​10.​1201/​b16790-6, (2014)
26.
go back to reference Liska, R., Shashkov, M.: Enforcing the discrete maximum principle for linear finite element solutions of second-order elliptic problems. Commun. Comput. Phys. 3(4), 852–877 (2008)MathSciNetMATH Liska, R., Shashkov, M.: Enforcing the discrete maximum principle for linear finite element solutions of second-order elliptic problems. Commun. Comput. Phys. 3(4), 852–877 (2008)MathSciNetMATH
27.
go back to reference Lo, Y.J., Williams, S., Straalen, B.V., Ligocki, T.J., Cordery, M.J., Wright, N.J., Hall, M.W., Oliker, L.: Roofline: an insightful visual performance model for multicore architectures. High Perform. Comput. Syst. Perfom. Model. Benchmarking Simul. 8966, 129–148 (2015) Lo, Y.J., Williams, S., Straalen, B.V., Ligocki, T.J., Cordery, M.J., Wright, N.J., Hall, M.W., Oliker, L.: Roofline: an insightful visual performance model for multicore architectures. High Perform. Comput. Syst. Perfom. Model. Benchmarking Simul. 8966, 129–148 (2015)
28.
go back to reference Logg, A.: Efficient representation of computational Meshes. Int. J. Comput. Sci. Eng. 4, 283–295 (2009)CrossRef Logg, A.: Efficient representation of computational Meshes. Int. J. Comput. Sci. Eng. 4, 283–295 (2009)CrossRef
29.
go back to reference Logg, A., Mardal, K.A., Wells, G.N.: Automated Solution of Differential Equations by the Finite Element Method. Springer, New York (2012)CrossRefMATH Logg, A., Mardal, K.A., Wells, G.N.: Automated Solution of Differential Equations by the Finite Element Method. Springer, New York (2012)CrossRefMATH
30.
go back to reference May, D.A., Brown, J., Laetitia, L.L.: pTatin3D: High-performance methods for long-term lithospheric dynamics. In:Proceedings of the International Conference for High Performance Computing, Network, Storage and Analysis, SC ‘14, pp. 274–284. IEEE Press, (2014) May, D.A., Brown, J., Laetitia, L.L.: pTatin3D: High-performance methods for long-term lithospheric dynamics. In:Proceedings of the International Conference for High Performance Computing, Network, Storage and Analysis, SC ‘14, pp. 274–284. IEEE Press, (2014)
32.
go back to reference Mudunuru, M.K., Nakshatrala, K.B.: On enforcing maximum principles and achieving element-wise species balance for advection-diffusion-reaction equations under the finite element method. J. Comput. Phys. 305, 448–493 (2016)MathSciNetCrossRefMATH Mudunuru, M.K., Nakshatrala, K.B.: On enforcing maximum principles and achieving element-wise species balance for advection-diffusion-reaction equations under the finite element method. J. Comput. Phys. 305, 448–493 (2016)MathSciNetCrossRefMATH
33.
go back to reference Mudunuru, M.K., Nakshatrala, K.B.: On mesh restrictions to satisfy comparison principles, maximum principles, and the non-negative constraint: Recent developments and new results. Mechanics of Advanced Materials and Structures (2016). doi:10.1080/15502287.2016.1166160 MATH Mudunuru, M.K., Nakshatrala, K.B.: On mesh restrictions to satisfy comparison principles, maximum principles, and the non-negative constraint: Recent developments and new results. Mechanics of Advanced Materials and Structures (2016). doi:10.​1080/​15502287.​2016.​1166160 MATH
34.
go back to reference Munson, T., Sarich, J., Wild, S., Benson, S., McInnes, L.C.: TAO 2.0 users manual. Technical Report ANL/MCS-TM-322, Mathematics and Computer Science Division, Argonne National Laboratory. http://www.mcs.anl.gov/tao (2012) Munson, T., Sarich, J., Wild, S., Benson, S., McInnes, L.C.: TAO 2.0 users manual. Technical Report ANL/MCS-TM-322, Mathematics and Computer Science Division, Argonne National Laboratory. http://​www.​mcs.​anl.​gov/​tao (2012)
35.
go back to reference Nagarajan, H., Nakshatrala, K.B.: Enforcing the non-negativity constraint and maximum principles for diffusion with decay on general computational grids. Int. J. Numer. Methods Fluids 67, 820–847 (2011)MathSciNetCrossRefMATH Nagarajan, H., Nakshatrala, K.B.: Enforcing the non-negativity constraint and maximum principles for diffusion with decay on general computational grids. Int. J. Numer. Methods Fluids 67, 820–847 (2011)MathSciNetCrossRefMATH
36.
go back to reference Nakshatrala, K.B., Mudunuru, M.K., Valocchi, A.J.: A numerical framework for diffusion-controlled bimolecular-reactive systems to enforce maximum principles and the non-negative constraint. J. Comput. Phys. 253, 278–307 (2013)MathSciNetCrossRefMATH Nakshatrala, K.B., Mudunuru, M.K., Valocchi, A.J.: A numerical framework for diffusion-controlled bimolecular-reactive systems to enforce maximum principles and the non-negative constraint. J. Comput. Phys. 253, 278–307 (2013)MathSciNetCrossRefMATH
37.
go back to reference Nakshatrala, K.B., Nagarajan, H., Shabouei, M.: A numerical methodology for enforcing maximum principles and the non-negative constraint for transient diffusion equations. Commun. Comput. Phys. 19, 53–93 (2016)MathSciNetCrossRef Nakshatrala, K.B., Nagarajan, H., Shabouei, M.: A numerical methodology for enforcing maximum principles and the non-negative constraint for transient diffusion equations. Commun. Comput. Phys. 19, 53–93 (2016)MathSciNetCrossRef
38.
go back to reference Nakshatrala, K.B., Valocchi, A.J.: Non-negative mixed finite element formulations for a tensorial diffusion equation. J. Comput. Phys. 228, 6726–6752 (2009)MathSciNetCrossRefMATH Nakshatrala, K.B., Valocchi, A.J.: Non-negative mixed finite element formulations for a tensorial diffusion equation. J. Comput. Phys. 228, 6726–6752 (2009)MathSciNetCrossRefMATH
39.
go back to reference Payette, G.S., Nakshatrala, K.B., Reddy, J.N.: On the performance of high-order finite elements with respect to maximum principles and the nonnegative constraint for diffusion-type equations. Int. J. Numer. Methods Eng. 91, 742–771 (2012)MathSciNetCrossRefMATH Payette, G.S., Nakshatrala, K.B., Reddy, J.N.: On the performance of high-order finite elements with respect to maximum principles and the nonnegative constraint for diffusion-type equations. Int. J. Numer. Methods Eng. 91, 742–771 (2012)MathSciNetCrossRefMATH
40.
go back to reference Pruess, K.: The TOUGH codes–a family of simulation tools for multiphase flow and transport processes in permeable media. Vadose Zone J. 3(3), 738–746 (2004) Pruess, K.: The TOUGH codes–a family of simulation tools for multiphase flow and transport processes in permeable media. Vadose Zone J. 3(3), 738–746 (2004)
42.
go back to reference Ulbrich, M.: Semismooth Newton Methods for Variational Inequalities and Constrained Optimization Problems in Function Spaces. SIAM, Philadelphia (2011) Ulbrich, M.: Semismooth Newton Methods for Variational Inequalities and Constrained Optimization Problems in Function Spaces. SIAM, Philadelphia (2011)
43.
go back to reference US EPA. Cleaning up the nation’s waste sites: Markets and technology trends. Technical Report EPA 542-R-04-015, (2004) US EPA. Cleaning up the nation’s waste sites: Markets and technology trends. Technical Report EPA 542-R-04-015, (2004)
44.
go back to reference Williams, S., Waterman, A., Patterson, D.: Roofline: an insightful visual performance model for multicore architectures. Commun. ACM 52, 65–76 (2009)CrossRef Williams, S., Waterman, A., Patterson, D.: Roofline: an insightful visual performance model for multicore architectures. Commun. ACM 52, 65–76 (2009)CrossRef
45.
go back to reference Zyvoloski, G.: FEHM: A control volume finite element code for simulating subsurface multi-phase multi-fluid heat and mass transfer. Los Alamos Unclassified Report LA-UR-07-3359, (2007) Zyvoloski, G.: FEHM: A control volume finite element code for simulating subsurface multi-phase multi-fluid heat and mass transfer. Los Alamos Unclassified Report LA-UR-07-3359, (2007)
Metadata
Title
Large-Scale Optimization-Based Non-negative Computational Framework for Diffusion Equations: Parallel Implementation and Performance Studies
Publication date
26-07-2016
Published in
Journal of Scientific Computing / Issue 1/2017
Print ISSN: 0885-7474
Electronic ISSN: 1573-7691
DOI
https://doi.org/10.1007/s10915-016-0250-5

Other articles of this Issue 1/2017

Journal of Scientific Computing 1/2017 Go to the issue

Premium Partner