skip to main content
research-article

Algorithm 1016: PyMGRIT: A Python Package for the Parallel-in-time Method MGRIT

Published: 20 April 2021 Publication History

Abstract

In this article, we introduce the Python framework PyMGRIT, which implements the multigrid-reduction-in-time (MGRIT) algorithm for solving (non-)linear systems arising from the discretization of time-dependent problems. The MGRIT algorithm is a reduction-based iterative method that allows parallel-in-time simulations, i.e., calculating multiple time steps simultaneously in a simulation, using a time-grid hierarchy. The PyMGRIT framework includes many different variants of the MGRIT algorithm, ranging from different multigrid cycle types and relaxation schemes, various coarsening strategies, including time-only and space-time coarsening, and the ability to utilize different time integrators on different levels in the multigrid hierachy. The comprehensive documentation with tutorials and many examples and the fully documented code allow an easy start into the work with the package. The functionality of the code is ensured by automated serial and parallel tests using continuous integration. PyMGRIT supports serial runs suitable for prototyping and testing of new approaches, as well as parallel runs using the Message Passing Interface (MPI). In this manuscript, we describe the implementation of the MGRIT algorithm in PyMGRIT and present the usage from both a user and a developer point of view. Three examples illustrate different aspects of the package itself, especially running tests with pure time parallelism, as well as space-time parallelism through the coupling of PyMGRIT with PETSc or Firedrake.

Supplementary Material

ZIP File (1016.zip)
Software for PyMGRIT: A Python Package for the Parallel-in-time Method MGRIT

References

[1]
Satish Balay, Shrirang Abhyankar, Mark F. Adams, Jed Brown, Peter Brune, Kris Buschelman, Lisandro Dalcin, Alp Dener, Victor Eijkhout, William D. Gropp, Dmitry Karpeyev, Dinesh Kaushik, Matthew G. Knepley, Dave A. May, Lois Curfman McInnes, Richard Tran Mills, Todd Munson, Karl Rupp, Patrick Sanan, Barry F. Smith, Stefano Zampini, Hong Zhang, and Hong Zhang. 2020. PETSc Users Manual. Technical Report ANL-95/11 - Revision 3.13. Argonne National Laboratory. Retrieved from https://www.mcs.anl.gov/petsc.
[2]
Andreas Bartel, Sascha Baumanns, and Sebastian Schöps. 2011. Structural analysis of electrical circuits including magnetoquasistatic devices. Appl. Numer. Math. 61 (Sept. 2011), 1257--1270.
[3]
Matthias Bolten, Stephanie Friedhoff, Jens Hahne, and Sebastian Schöps. 2019. Parallel-in-Time Simulation of an Electrical Machine using MGRIT. arxiv:math.NA/1912.03106, 2019.
[4]
Achi Brandt. 1977. Multi-level adaptive solutions to boundary-value problems. Math. Comput. 31, 138 (1977), 333--390.
[5]
Andrew J. Christlieb, Colin B. Macdonald, and Benjamin W. Ong. 2010. Parallel high-order integrators. SIAM J. Sci. Comput. 32, 2 (2010), 818--835.
[6]
Idoia Cortes Garcia, Herbert De Gersem, and Sebastian Schöps. 2019. A structural analysis of field/circuit coupled problems based on a generalised circuit element. Numer. Algor. (Mar. 2019), 1--22.
[7]
Lisandro D. Dalcin, Rodrigo R. Paz, Pablo A. Kler, and Alejandro Cosimo. 2011. Parallel distributed computing using python. Adv. Water Resour. 34, 9 (2011), 1124--1139. New Computational Methods and Software Tools.
[8]
Hans De Sterck, Robert D. Falgout, Stephanie Friedhoff, Oliver A. Krzysik, and Scott P. MacLachlan. 2019. Optimizing MGRIT and Parareal coarse-grid operators for linear advection. arxiv:math.NA/1910.03726, 2019.
[9]
Patrick Dular and Christophe Geuzaine. 2020. GetDP: A General Environment for the Treatment of Discrete Problems. Retrieved from http://www.getdp.info.
[10]
Patrick Dular, Christophe Geuzaine, François Henrotte, and Willy Legros. 1998. A general environment for the treatment of discrete problems and its application to the finite element method. IEEE Trans. Magnet. 34, 5 (Sep. 1998), 3395--3398.
[11]
Alok Dutt, Leslie Greengard, and Vladimir Rokhlin. 2000. Spectral deferred correction methods for ordinary differential equations. BIT Numer. Math. 40, 2 (June 2000), 241--266.
[12]
Matthew Emmett, Torbjörn Klatt, Robert Speck, Daniel Ruprecht, and Selman Terzi. 2020. Github repository for PFASST. Retrieved from https://github.com/Parallel-in-Time/PFASST.
[13]
Matthew Emmett and Michael Minion. 2012. Toward an efficient parallel in time method for partial differential equations. Commun. Appl. Math. Computat. Sci. 7, 1 (2012), 105--132.
[14]
Robert D. Falgout, Stephanie Friedhoff, Tzanio Kolev, Scott MacLachlan, and Jacob B. Schroder. 2014. Parallel time integration with multigrid. SIAM J. Sci. Comput. 36, 6 (2014), C635--C661.
[15]
Robert D. Falgout, Aaron Katz, Tzanio Kolev, Jacob B. Schroder, Andrew Wissink, and Ulrike M. Yang. 2015. Parallel Time Integration with Multigrid Reduction for a Compressible Fluid Dynamics Application. Technical Report, LLNL-JRNL-663416. Lawrence Livermore National Laboratory.
[16]
Robert D. Falgout, Matthieu Lecouvez, and Carol S. Woodward. 2019. A parallel-in-time algorithm for variable step multistep methods. J. Computat. Sci. 37 (2019), 101029, 12.
[17]
Robert D. Falgout, Thomas Manteuffel, Ben O’Neill, and Jacob B. Schroder. 2017. Multigrid reduction in time for nonlinear parabolic problems: A case study. SIAM J. Sci. Comput. 39, 5 (2017), S298--S322.
[18]
Mauricio V. Ferreira da Luz, Patrick Dular, Nelson Sadowski, Christophe Geuzaine, and João P. A. Bastos. 2002. Analysis of a permanent magnet generator with dual formulations using periodicity conditions and moving band. IEEE Trans. Magnet. 38, 2 (Mar. 2002), 961--964.
[19]
Stephanie Friedhoff, Jens Hahne, Iryna Kulchytska-Ruchka, and Sebastian Schöps. 2020. Exploring parallel-in-time approaches for eddy current problems. In Progress in Industrial Mathematics at ECMI 2018 (The European Consortium for Mathematics in Industry), István Faragó, Ferenc Izsák, and Péter L. Simon (Eds.), Vol. 30. Springer, Berlin.
[20]
Stephanie Friedhoff, Jens Hahne, and Sebastian Schöps. 2019. Multigrid-reduction-in-time for eddy current problems. PAMM 19, 1 (2019), e201900262.
[21]
Martin J. Gander. 2015. 50 years of time parallel time integration. In Multiple Shooting and Time Domain Decomposition. Springer, 69--113.
[22]
Martin J. Gander and Stefan Güttel. 2013. PARAEXP: A parallel integrator for linear initial-value problems. SIAM J. Sci. Comput. 35, 2 (2013), C123--C142.
[23]
Martin J. Gander, Iryna Kulchytska-Ruchka, Innocent Niyonzima, and Sebastian Schöps. 2019. A new Parareal algorithm for problems with discontinuous sources. SIAM J. Sci. Comput. 41, 2 (2019), B375--B395.
[24]
Christophe Geuzaine. 2007. GetDP: A general finite-element solver for the de rham complex. PAMM 7, 1 (2007), 1010603--1010604.
[25]
Christophe Geuzaine and Jean-François Remacle. 2009. Gmsh: A 3-D finite element mesh generator with built-in pre- and post-processing facilities. Int. J. Numer. Methods Eng. 79, 11 (2009), 1309--1331.
[26]
Christophe Geuzaine and Jean-François Remacle. 2020. Gmsh: A three-dimensional finite element mesh generator with built-in pre- and post-processing facilities. Retrieved from http://www.gmsh.info.
[27]
Johan Gyselinck, Lieven Vandevelde, and Jan Melkebeek. 2001. Multi-slice FE modeling of electrical machines with skewed slots-the skew discretization error. IEEE Trans. Magnet. 37 (10 2001), 3233--3237.
[28]
Stefanie Günther, Lars Ruthotto, Jacob B. Schroder, Eric C. Cyr, and Nicolas R. Gauger. 2020. Layer-parallel training of deep residual neural networks. SIAM J. Math. Data Sci. 2, 1 (2020), 1--23.
[29]
Jens Hahne and Stephanie Friedhoff. 2020a. Code coverage for PyMGRIT. Retrieved from https://codecov.io/gh/pymgrit/pymgrit.
[30]
Jens Hahne and Stephanie Friedhoff. 2020b. Documentation for PyMGRIT. Retrieved from https://pymgrit.github.io/pymgrit/.
[31]
Jens Hahne and Stephanie Friedhoff. 2020c. Github repository for PyMGRIT. Retrieved from https://github.com/pymgrit/pymgrit.
[32]
Jens Hahne and Stephanie Friedhoff. 2020d. PyPI site for PyMGRIT. Retrieved from https://pypi.org/project/pymgrit/.
[33]
A. Howse, H. De Sterck, Robert D. Falgout, Scott MacLachlan, and Jacob B. Schroder. 2019. Parallel-in-time multigrid with adaptive spatial coarsening for the linear advection and inviscid burgers equations. SIAM J. Sci. Comput. 41, 1 (2019), A538--A565.
[34]
John D. Hunter. 2007. Matplotlib: A 2D graphics environment. Comput. Sci. Eng. 9, 3 (2007), 90--95.
[35]
Jülich Supercomputing Centre. 2019. JUWELS: Modular tier-0/1 supercomputer at the Jülich supercomputing centre. J. Large-scale Res. Facil. 5, A135 (2019).
[36]
David I. Ketcheson, Kyle T. Mandli, Aron J. Ahmadia, Amal Alghamdi, Manuel Quezada de Luna, Matteo Parsani, Matthew G. Knepley, and Matthew Emmett. 2012. PyClaw: Accessible, extensible, scalable tools for wave propagation problems. SIAM J. Sci. Comput. 34, 4 (Nov. 2012), C210--C231.
[37]
Thomas Kluyver, Benjamin Ragan-Kelley, Fernando Perez, Brian Granger, Matthias Bussonnier, Jonathan Frederic, Kyle Kelley, Jessica Hamrick, Jason Grout, Sylvain Corlay, Paul Ivanov, Damián Avila, Safia Abdalla, Carol Willing, et al. 2016. Jupyter notebooks—A publishing format for reproducible computational workflows. In Positioning and Power in Academic Publishing: Players, Agents and Agendas. IOS Press, 87--90.
[38]
Holger Krekel. 2020. Welcome to the tox automation project. Retrieved from https://tox.readthedocs.io/en/latest/.
[39]
Holger Krekel, Bruno Oliveira, Ronny Pfannschmidt, Floris Bruynooghe, Brianna Laugher, and Florian Bruhin. 2004. pytest. Retrieved from https://github.com/pytest-dev/pytest.
[40]
Lydia Kronsjö. 1975. A note on the “nested iterations” methods. Nordisk Tidskr. Informationsbehandling (BIT) 15, 1 (1975), 107--110.
[41]
Lydia Kronsjö and Germund Dahlquist. 1972. On the design of nested iterations for elliptic difference equations. Nordisk Tidskr. Informationsbehandling (BIT) 12 (1972), 63--71.
[42]
LBNL. 2020. Github repository for LibPFASST. Retrieved from https://github.com/libpfasst/LibPFASST.
[43]
Matthieu Lecouvez, Robert D. Falgout, Carol S. Woodward, and Philip Top. 2016. A parallel multigrid reduction in time method for power systems. In Proceedings of the IEEE Power and Energy Society General Meeting (PESGM’16). 1--5.
[44]
Jacques-Louis Lions, Yvon Maday, and Gabriel Turinici. 2001. Résolution d’EDP par un schéma en temps “pararéel.” Comptes Rendus de l’Académie des Sciences. Série I. Mathématique 332, 7 (2001), 661--668.
[45]
LLBL. 2020. Website for PFASST codes. Retrieved from https://pfasst.lbl.gov/codes.
[46]
LLNL. 2020. Website for XBraid. Retrieved from https://www.llnl.gov/casc/xbraid.
[47]
Anders Logg, Kent-Andre Mardal, Garth N. Wells, et al. 2012. Automated Solution of Differential Equations by the Finite Element Method. Springer.
[48]
Allan S. Nielsen, Gilles Brunner, and Jan S. Hesthaven. 2018. Communication-aware adaptive parareal with application to a nonlinear hyperbolic system of partial differential equations. J. Comput. Phys. 371 (2018), 483--505.
[49]
Benjamin W. Ong, Ronald D. Haynes, and Kyle Ladd. 2016. Algorithm 965: RIDC methods: A family of parallel time integrators. ACM Trans. Math. Softw. 43, 1 (Aug. 2016).
[50]
Florian Rathgeber, David A. Ham, Lawrence Mitchell, Michael Lange, Fabio Luporini, Andrew T. T. Mcrae, Gheorghe-Teodor Bercea, Graham R. Markall, and Paul H. J. Kelly. 2016. Firedrake: Automating the finite element method by composing abstractions. ACM Trans. Math. Softw. 43, 3 (Dec. 2016).
[51]
Sebastian Schöps, Innocent Niyonzima, and Markus Clemens. 2018. Parallel-in-time simulation of eddy current problems using Parareal. IEEE Trans. Magnet. 54, 3 (Mar 2018).
[52]
Martin Schreiber, Pedro da Silva Peixoto, Andreas Schmitt, and François P. Hamon. 2020. Website for SWEET. Retrieved from https://schreiberx.github.io/sweetsite/.
[53]
Jacob B. Schroder, Robert D. Falgout, Carol S. Woodward, Philip Top, and Matthieu Lecouvez. 2018. Parallel-in-time solution of power systems with scheduled events. In Proceedings of the IEEE Power Energy Society General Meeting (PESGM’18). 1--5.
[54]
Robert Speck. 2019. Algorithm 997: PySDC—prototyping spectral deferred corrections. ACM Trans. Math. Softw. 45, 3 (Aug. 2019).
[55]
Robert Speck. 2020. Github repository for pySDC. Retrieved from https://github.com/Parallel-in-Time/pySDC.
[56]
Stefan van der Walt, S. Chris Colbert, and Gael Varoquaux. 2011. The NumPY array: A structure for efficient numerical computation. Comput. Sci. Eng. 13, 2 (2011), 22--30.
[57]
Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, Stéfan J. van der Walt, Matthew Brett, Joshua Wilson, K. Jarrod Millman, Nikolay Mayorov, Andrew R. J. Nelson, Eric Jones, Robert Kern, Eric Larson, C. J. Carey, İlhan Polat, Yu Feng, Eric W. Moore, Jake VanderPlas, Denis Laxalde, Josef Perktold, and Robert Cimrman. 2020. SciPy 1.0: Fundamental algorithms for scientific computing in Python. Nat. Meth. 17 (2020), 261--272.

Cited By

View all
  • (2024)Task graph-based performance analysis of parallel-in-time methodsParallel Computing10.1016/j.parco.2023.103050118:COnline publication date: 1-Feb-2024
  • (2024)Coarse-grid operator optimization in multigrid reduction in time for time-dependent Stokes and Oseen problemsJapan Journal of Industrial and Applied Mathematics10.1007/s13160-024-00652-841:3(1315-1339)Online publication date: 26-Apr-2024
  • (2023)A parallel-in-time collocation method using diagonalization: theory and implementation for linear problemsCommunications in Applied Mathematics and Computational Science10.2140/camcos.2023.18.5518:1(55-85)Online publication date: 21-Dec-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Mathematical Software
ACM Transactions on Mathematical Software  Volume 47, Issue 2
June 2021
243 pages
ISSN:0098-3500
EISSN:1557-7295
DOI:10.1145/3459727
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 April 2021
Accepted: 01 January 2021
Revised: 01 December 2020
Received: 01 July 2020
Published in TOMS Volume 47, Issue 2

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Multigrid-reduction-in-time (MGRIT)
  2. parallel-in-time integration

Qualifiers

  • Research-article
  • Research
  • Refereed

Funding Sources

  • Bundesministerium für Bildung und Forschung

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)33
  • Downloads (Last 6 weeks)1
Reflects downloads up to 21 Oct 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Task graph-based performance analysis of parallel-in-time methodsParallel Computing10.1016/j.parco.2023.103050118:COnline publication date: 1-Feb-2024
  • (2024)Coarse-grid operator optimization in multigrid reduction in time for time-dependent Stokes and Oseen problemsJapan Journal of Industrial and Applied Mathematics10.1007/s13160-024-00652-841:3(1315-1339)Online publication date: 26-Apr-2024
  • (2023)A parallel-in-time collocation method using diagonalization: theory and implementation for linear problemsCommunications in Applied Mathematics and Computational Science10.2140/camcos.2023.18.5518:1(55-85)Online publication date: 21-Dec-2023
  • (2022)Asynchronous Truncated Multigrid-Reduction-in-TimeSIAM Journal on Scientific Computing10.1137/21M143314945:3(S281-S306)Online publication date: 22-Nov-2022
  • (undefined)Task Graph-Based Performance Analysis of Parallel-in-Time MethodsSSRN Electronic Journal10.2139/ssrn.4201056

View Options

Get Access

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media