A space-time parallel algorithm with adaptive mesh refinement for computational fluid dynamics. (English) Zbl 07704917
Summary: This paper describes a space-time parallel algorithm with space-time adaptive mesh refinement (AMR). AMR with subcycling is added to multigrid reduction-in-time (MGRIT) in order to provide solution efficient adaptive grids with a reduction in work performed on coarser grids. This algorithm is achieved by integrating two software libraries: XBraid (Parallel time integration with multigrid. https://computation.llnl.gov/projects/parallel-timeintegration-multigrid) and Chombo (Chombo software package for AMR applications – design document, 2014). The former is a parallel time integration library using multigrid and the latter is a massively parallel structured AMR library. Employing this adaptive space-time parallel algorithm is Chord [S. M. Guzik et al., Comput. Fluids 123, 202–217 (2015; Zbl 1390.65091)], a computational fluid dynamics (CFD) application code for solving compressible fluid dynamics problems. For the same solution accuracy, speedups are demonstrated from the use of space-time parallelization over the time-sequential integration on Couette flow and Stokes’ second problem. On a transient Couette flow case, at least a \(1.5\times\) speedup is achieved, and with a time periodic problem, a speedup of up to \(13.7\times\) over the time-sequential case is obtained. In both cases, the speedup is achieved by adding processors and exploring additional parallelization in time. The numerical experiments show the algorithm is promising for CFD applications that can take advantage of the time parallelism. Future work will focus on improving the parallel performance and providing more tests with complex fluid dynamics to demonstrate the full potential of the algorithm.
MSC:
| 65Nxx | Numerical methods for partial differential equations, boundary value problems |
Keywords:
time-parallel; mesh parallel-in-time; adaptivity; multigrid; MGRIT; high-order CFD; finite-volumeCitations:
Zbl 1390.65091References:
| [1] | AMReX: Software framework for block structured AMR. https://amrex-codes.github.io/amrex/. Accessed 3 Oct 2018 |
| [2] | CASTRO: An adaptive mesh, astrophysical radiation hydrodynamics simulation code. https://amrex-astro.github.io/Castro/. Accessed 16 May 2019 |
| [3] | Overture: An object-oriented toolkit for solving partial differential equations in complex geometry. https://www.overtureframework.org/. Accessed 16 May 2019 |
| [4] | p4est: Parallel adaptive mesh refinement on forests of octrees. http://www.p4est.org/. Accessed 16 May 2019 |
| [5] | SAMRAI: Structured adaptive mesh refinement application infrastructure. https://computation.llnl.gov/projects/samrai. Accessed 3 Oct 2018 |
| [6] | XBraid: Parallel time integration with multigrid. https://computation.llnl.gov/projects/parallel-time-integration-multigrid. Accessed 24 Sep 2018 |
| [7] | Adams, M., Colella, P., Graves, D.T., Johnson, J.N., Johansen, H.S., Keen, N.D., Ligocki, T.J., Martin, D.F., McCorquodale, P.W., Modiano, D., Schwartz, P.O., Sternberg, T.D., Van Straalen, B.: Chombo software package for AMR applications - design document. Technical Report LBNL-6616E, Lawrence Berkeley National Laboratory (2014) |
| [8] | Anderson, J., Modern Compressible Flow: With Historical Perspective (2002), Boston: McGraw-Hill Education, Boston |
| [9] | Batchelor, GK, An Introduction to Fluid Dynamics (2000), Cambridge: Cambridge University Press, Cambridge · Zbl 0958.76001 · doi:10.1017/CBO9780511800955 |
| [10] | Brandt, A.: Multi-level adaptive computations in fluid dynamics. In: Technical Report AIAA-79-1455. AIAA, Williamsburg, VA (1979) |
| [11] | Briggs, WL; Henson, VE; McCormick, SF, A Multigrid Tutorial (2000), Philadelphia: Society for Industrial and Applied Mathematics, Philadelphia · Zbl 0958.65128 · doi:10.1137/1.9780898719505 |
| [12] | Christopher, J., Gao, X., Guzik, S.M., Falgout, R., Schroder, J.: Parallel in time for a fully space-time adaptive mesh refinement algorithm. In: AIAA Scitech 2020 Forum. American Institute of Aeronautics and Astronautics, Orlando, FL (2020). doi:10.2514/6.2020-0340 |
| [13] | Christopher, J., Gao, X., Guzik, S.M., Falgout, R.D., Schroder, J.B.: Space-time adaptivity with multigrid reduction in time for the compressible navier-stokes equations. In: 19th Copper Mountain Conference On Multigrid Methods, pp. 10 (2019) |
| [14] | Colella, P.; Dorr, MR; Hittinger, JAF; Martin, DF, High-order finite-volume methods in mapped coordinates, JCP, 230, 2952-2976 (2011) · Zbl 1218.65119 |
| [15] | Colella, P.; Sekora, MD, A limiter for PPM that preserves accuracy at smooth extrema, J. Comput. Phys., 227, 15, 7069-7076 (2008) · Zbl 1152.65090 · doi:10.1016/j.jcp.2008.03.034 |
| [16] | Day, MS; Bell, JB, Numerical simulation of laminar reacting flows with complex chemistry, Combust. Theory Model., 4, 535-556 (2000) · Zbl 0970.76065 · doi:10.1088/1364-7830/4/4/309 |
| [17] | De Sterck, H.; Falgout, RD; Howse, AJM; MacLachlan, SP; Schroder, JB, Parallel-in-time multigrid with adaptive spatial coarsening for the linear advection and inviscid burgers equations, SIAM J. Sci. Comput., 41, A538 (2017) · Zbl 1407.65040 |
| [18] | Dubey, A.; Almgren, A.; Bell, J.; Berzins, M.; Brandt, S.; Bryan, G.; Colella, P.; Graves, D.; Lijewski, M.; Löffler, F.; O’Shea, B.; Schnetter, E.; Van Straalen, B.; Weide, K., A survey of high level frameworks in block-structured adaptive mesh refinement packages, J. Parallel Distrib. Comput., 74, 12, 3217-3227 (2014) · doi:10.1016/j.jpdc.2014.07.001 |
| [19] | Falgout, RD; Friedhoff, S.; Kolev, TV; MacLachlan, SP; Schroder, JB, Parallel time integration with multigrid, SIAM J. Sci. Comput, 36, C635-C661 (2014) · Zbl 1310.65115 · doi:10.1137/130944230 |
| [20] | Falgout, RD; Friedhoff, S.; Kolev, TV; MacLachlan, SP; Schroder, JB, Parallel time integration with multigrid, SIAM J. Sci. Comput., 36, 6, C635-C661 (2014) · Zbl 1310.65115 · doi:10.1137/130944230 |
| [21] | Falgout, RD; Friedhoff, S.; Vandewalle, S.; Kolev, TV; MacLachlan, SP; Schroder, JB, Multigrid methods with space-time concurrency, Comput. Vis. Sci., 18, 4-5, 123-143 (2017) · Zbl 1448.65146 · doi:10.1007/s00791-017-0283-9 |
| [22] | Falgout, R.D., Katz, A., Kolev, T.V., Schroder, J., Wissink, A.M., Yang, U.: Parallel time integration with multigrid reduction for a compressible fluid dynamics application. Technical report Lawrence Livermore National Lab (2014) |
| [23] | Falgout, R.D., Katz, A., Kolev, T.V., Schroder, J.B., Wissink, A., Yang, U.M.: Parallel time integration with multigrid reduction for a compressible fluid dynamics application. Technical Report LLNL-JRNL-663416, Lawrence Livermore National Laboratory (2015) |
| [24] | Falgout, R.D., Manteuffel, T.A., Schroder, J.B., Southworth, B.: Parallel-in-time for moving meshes. Technical Report LLNL-TR-681918, Lawrence Livermore National Laboratory (2016) |
| [25] | Friedhoff, S., Hahne, J., Kulchytska-Ruchka, I., Schöps, S.: Exploring parallel-in-time approaches for eddy current problems. arXiv preprint arXiv:1810.13263 (2018) · Zbl 1469.78053 |
| [26] | Fryxell, B.; Olson, K.; Ricker, P.; Timmes, FX; Zingale, M.; Lamb, DQ; MacNeice, P.; Rosner, R.; Truran, JW; Tufo, H., FLASH: an adaptive mesh hydrodynamics code for modeling astrophysical thermonuclear flashes, ApJS, 131, 1, 273 (2000) · doi:10.1086/317361 |
| [27] | Gander, MJ; Carraro, T.; Geiger, M.; Körkel, S.; Rannacher, R., 50 years of time parallel time integration, Multiple Shooting and Time Domain Decomposition, 69-114 (2015), Berlin: Springer, Berlin · Zbl 1337.65127 · doi:10.1007/978-3-319-23321-5_3 |
| [28] | Gander, MJ; Neumüller, M., Analysis of a new space-time parallel multigrid algorithm for parabolic problems, SIAM J. Sci. Comput., 38, 4, A2173-A2208 (2016) · Zbl 1342.65225 · doi:10.1137/15M1046605 |
| [29] | Gao, X.: A parallel solution-adaptive method for turbulent non-premixed combusting flows. Ph.D. thesis, University of Toronto (2008) |
| [30] | Gao, X., Guzik, S.M., Olschanowsky, C.: A high-performance fourth-order finite-volume algorithm for compressible viscous flows on structured grids. Submitted to AIAA J · Zbl 1368.76040 |
| [31] | Gao, X., Guzik, S.M.J.: A fourth-order scheme for the compressible navier-stokes equations. In: AIAA 2015-0298, 53rd AIAA Aerospace Sciences Meeting (2015) |
| [32] | Gao, X., Guzik, S.M.J., Colella, P.: Fourth order boundary treatment for viscous fluxes on cartesian grid finite-volume methods. In: AIAA 2014-1277, 52nd AIAA Aerospace Sciences Meeting (2014) |
| [33] | Gao, X.; Northrup, S.; Groth, C., Parallel solution-adaptive method for two-dimensional non-premixed combusting flows, Progress Comput. Fluid. Dyn. Int. J., 11, 2, 76-95 (2011) · Zbl 1278.76067 · doi:10.1504/PCFD.2011.038834 |
| [34] | Gao, X., Owen, L.D., Guzik, S.M.: A parallel adaptive numerical method with generalized curvilinear coordinate transformation for compressible navier-stokes equations. Int. J. Numer. Meth. Fluids, Accepted |
| [35] | Gao, X., Owen, L.D., Guzik, S.M.: A high-order finite-volume method for combustion. In: AIAA 2016-1808, 54th AIAA Aerospace Sciences Meeting (2016) |
| [36] | Garnier, E., Adams, N., Sagaut, P.: Large Eddy Simulation for Compressible Flows. Scientific Computation. Springer Netherlands (2009). doi:10.1007/978-90-481-2819-8 · Zbl 1179.76005 |
| [37] | Georgiadis, N.J., Rizzetta, D.P., Fureby, C.: Large-eddy simulation: Current capabilities, recommended practices, and future research pp. 27 (2009) |
| [38] | Groth, C., De Zeeuw, D., Powell, K., Gombosi, T., Stout, Q.: A parallel solution-adaptive scheme for ideal magnetohydrodynamics. In: 14th Computational Fluid Dynamics Conference. American Institute of Aeronautics and Astronautics, Norfolk, VA, U.S.A. (1999). doi:10.2514/6.1999-3273 |
| [39] | Günther, S.; Gauger, NR; Schroder, JB, A non-intrusive parallel-in-time adjoint solver with the XBraid library, Comput. Vis. Sci., 19, 3-4, 85-95 (2018) · Zbl 07704539 · doi:10.1007/s00791-018-0300-7 |
| [40] | Günther, S.; Gauger, NR; Schroder, JB, A non-intrusive parallel-in-time approach for simultaneous optimization with unsteady PDEs, Optim. Methods Softw., 34, 1306 (2018) · Zbl 1428.35641 · doi:10.1080/10556788.2018.1504050 |
| [41] | Günther, S., Ruthotto, L., Schroder, J., Cyr, E., Gauger, N.: Layer-parallel training of deep residual neural networks. SIAM Journal on Data Science (2019 (submitted)). ArXiv preprint arXiv:1812.04352 · Zbl 1508.68306 |
| [42] | Guzik, SM; Gao, X.; Olschanowsky, C., A high-performance finite-volume algorithm for solving partial differential equations governing compressible viscous flows on structured grids, Comput. Math Appl., 72, 2098-2118 (2016) · Zbl 1368.76040 · doi:10.1016/j.camwa.2016.08.004 |
| [43] | Guzik, SM; Gao, X.; Owen, LD; McCorquodale, P.; Colella, P., A freestream-preserving fourth-order finite-volume method in mapped coordinates with adaptive-mesh refinement, Comput. Fluids, 123, 202-217 (2015) · Zbl 1390.65091 · doi:10.1016/j.compfluid.2015.10.001 |
| [44] | Guzik, S.M.J., McCorquodale, P., Colella, P.: A freestream-preserving high-order finite-volume method for mapped grids with adaptive-mesh refinement. In: AIAA 2012-0574, 50th AIAA Aerospace Sciences Meeting (2012) · Zbl 1390.65091 |
| [45] | Hessenthaler, A.; Nordsletten, D.; Röhrle, O.; Schroder, J.; Falgout, R., Convergence of the multigrid-reduction-in-time algorithm for the linear elasticity equations, Numer. Linear Algebra Appl., 25, 3, e2155 (2018) · Zbl 1513.65345 · doi:10.1002/nla.2155 |
| [46] | Horton, G.; Vandewalle, S., A space-time multigrid method for parabolic partial differential equations, SIAM J. Sci. Comput., 16, 4, 848-864 (1995) · Zbl 0828.65105 · doi:10.1137/0916050 |
| [47] | Howse, AJ; Sterck, HD; Falgout, RD; MacLachlan, S.; Schroder, J., Parallel-in-time multigrid with adaptive spatial coarsening for the linear advection and inviscid burgers equations, SIAM J. Sci. Comput., 41, 1, A538-A565 (2019) · Zbl 1407.65040 · doi:10.1137/17M1144982 |
| [48] | Langer, U.; Matculevich, S.; Repin, S., Guaranteed error bounds and local indicators for adaptive solvers using stabilised space-time IgA approximations to parabolic problems, Comput. Math. Appl., 78, 8, 2641-2671 (2019) · Zbl 1443.65185 · doi:10.1016/j.camwa.2019.04.009 |
| [49] | Lecouvez, M., Falgout, R.D., Woodward, C.S., Top, P.: A parallel multigrid reduction in time method for power systems. In: Power and Energy Society General Meeting (PESGM) pp. 1-5 (2016) |
| [50] | Lohner, R., Othmer, C., Mrosek, M., Figueroa, A., Degro, A.: Overnight industrial LES for external aerodynamics. In: AIAA Scitech 2020 Forum. American Institute of Aeronautics and Astronautics. doi:10.2514/6.2020-2031 · Zbl 1521.76231 |
| [51] | Margolin, LG; Rider, WJ; Grinstein, FF, Modeling turbulent flow with implicit LES, J. Turbul., 7, 1-27 (2006) · Zbl 1273.76213 · doi:10.1080/14685240500331595 |
| [52] | Martin, D.; Colella, P.; Anghel, M.; Alexander, F., Adaptive mesh refinement for multiscale nonequilibrium physics, Comput. Sci. Eng., 7, 3, 24-31 (2005) · doi:10.1109/MCSE.2005.45 |
| [53] | Martin, DF; Colella, P.; Graves, D., A cell-centered adaptive projection method for the incompressible Navier-Stokes equations in three dimensions, J. Comput. Phys., 227, 3, 1863-1886 (2008) · Zbl 1137.76040 · doi:10.1016/j.jcp.2007.09.032 |
| [54] | McCorquodale, P.; Colella, P., A high-order finite-volume method for conservation laws on locally refined grids, Commun. App. Math. Comput. Sci., 6, 1, 1-25 (2011) · Zbl 1252.65163 · doi:10.2140/camcos.2011.6.1 |
| [55] | Miniati, F.; Colella, P., Block structured adaptive mesh and time refinement for hybrid, hyperbolic + N-body systems, J. Comput. Phys., 227, 1, 400-430 (2007) · Zbl 1128.85007 · doi:10.1016/j.jcp.2007.07.035 |
| [56] | Owen, L.; Guzik, S.; Gao, X., A high-order adaptive algorithm for multispecies gaseous flows on mapped domains, Comput. Fluids, 170, 249-260 (2018) · Zbl 1410.76256 · doi:10.1016/j.compfluid.2018.05.010 |
| [57] | Owen, LD; Gao, X.; Guzik, SM, Techniques for improving monotonicity in a fourth-order finite-volume algorithm solving shocks and detonations, J. Comput. Phys. (2020) · Zbl 1440.76099 · doi:10.1016/j.jcp.2020.109515 |
| [58] | Owen, L.D., Guzik, S.M., , Gao, X.: High-order CFD modeling of multispecies flows. In: Fall Meeting of the Western States Section of the Combustion Institute (2015). WSSCI 2015-134IE-0030 |
| [59] | Owen, L.D., Guzik, S.M., Gao, X.: A fourth-order finite-volume algorithm for compressible flow with chemical reactions on mapped grids (AIAA 2017-4498) (2017) |
| [60] | Sagaut, P.: Large Eddy Simulation for Incompressible Flows: An Introduction, 3 edn. Scientific Computation. Springer, Berlin (2006). doi:10.1007/b137536 · Zbl 1091.76001 |
| [61] | Schroder, J.B., Lecouvez, M., Falgout, R.D., Woodward, C.S., Top, P.: Parallel-in-time solution of power systems with scheduled events. Power and Energy Society General Meeting (PESGM) pp. 1-5 (2018) |
| [62] | Simon, HD, Parallel Computational Fluid Dynamics: Implementations and Results (1992), Cambridge: MIT Press, Cambridge · Zbl 0805.65150 |
| [63] | Steinbach, O., Space-time finite element methods for parabolic problems, Comput. Methods Appl. Math. (2015) · Zbl 1329.65229 · doi:10.1515/cmam-2015-0026 |
| [64] | Steinbach, O.; Yang, H., Comparison of algebraic multigrid methods for an adaptive space-time finite-element discretization of the heat equation in 3D and 4D: Comparison of AMG for adaptive space-time FEM in 3D and 4D, Numer. Linear Algebra Appl., 25, 3, e2143 (2018) · Zbl 1513.65084 · doi:10.1002/nla.2143 |
| [65] | Sterck, H.D., Falgout, R.D., Friedhoff, S., Krzysik, O.A., MacLachlan, S.P.: Optimizing mgrit and parareal coarse-grid operators for linear advection. arXiv:1910.03726. (2019) · Zbl 07396250 |
| [66] | Van Straalen, B., Colella, P., Graves, D.T., Keen, N.: Petascale block-structured AMR applications without distributed meta-data. In: Proceedings, Part II. Lecture Notes in Computer Science. Euro-Par 2011 Parallel Processing—17th International Conference, Springer (2011). ISBN 978-3-642-23396-8 |
| [67] | Yang, Z., Large-eddy simulation: past, present and the future, Chin. J. Aeronaut., 28, 1, 11-24 (2015) · doi:10.1016/j.cja.2014.12.007 |
| [68] | Yue, X., Shu, S., Xu, X., Bu, W., Pan, K.: Parallel-in-time with fully finite element multigrid for 2-d space-fractional diffusion equations. arXiv preprint arXiv:1805.06688 (2018) |
| [69] | Zhang, Q.; Johansen, H.; Colella, P., A fourth-order accurate finite-volume method with structured adaptive mesh refinement for solving the advection-diffusion equation, SIAM J. Sci. Comput., 34, 2, B179-B201 (2012) · Zbl 1302.76139 · doi:10.1137/110820105 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
