×

Recursive algorithms for distributed forests of octrees. (English) Zbl 1323.65105

Summary: The forest-of-octrees approach to parallel adaptive mesh refinement and coarsening has recently been demonstrated in the context of a number of large-scale partial differential equation-based applications. Efficient reference software has been made freely available to the public both in the form of the standalone p4est library and more indirectly by the general-purpose finite element library deal.II, which has been equipped with a p4est backend. Although linear octrees, which store only leaf octants, have an underlying tree structure by definition, it is not fully exploited in previously published mesh-related algorithms. This is because tree branches are not explicitly stored and because the topological relationships in meshes, such as the adjacency between cells, introduce dependencies that do not respect the octree hierarchy. In this work, we combine hierarchical and topological relationships between octants to design efficient recursive algorithms that operate on distributed forests of octrees. We present three important algorithms with recursive implementations. The first is a parallel search for leaves matching any of a set of multiple search criteria, such as leaves that contain points or intersect polytopes. The second is a ghost layer construction algorithm that handles arbitrarily refined octrees that are not covered by previous algorithms, which require a 2:1 condition between neighboring leaves. The third is a universal mesh topology iterator. This iterator visits every cell in a partition, as well as every interface (face, edge, and corner) between these cells. The iterator calculates the local topological information for every interface that it visits, taking into account the nonconforming interfaces that increase the complexity of describing the local topology. To demonstrate the utility of the topology iterator, we use it to compute the numbering and encoding of higher-order \(C^0\) nodal basis functions used for finite elements. We analyze the complexity of the new recursive algorithms theoretically and assess their performance, both in terms of single-processor efficiency and in terms of parallel scalability, demonstrating good weak and strong scaling up to 458,000 cores of the JUQUEEN supercomputer.

MSC:

65M50 Mesh generation, refinement, and adaptive methods for the numerical solution of initial value and initial-boundary value problems involving PDEs
65Y05 Parallel numerical computation
65M60 Finite element, Rayleigh-Ritz and Galerkin methods for initial value and initial-boundary value problems involving PDEs
65Y20 Complexity and performance of numerical algorithms
65Y15 Packaged methods for numerical algorithms
65N50 Mesh generation, refinement, and adaptive methods for boundary value problems involving PDEs
65N30 Finite element, Rayleigh-Ritz and Galerkin methods for boundary value problems involving PDEs

References:

[1] V. Akçelik, J. Bielak, G. Biros, I. Epanomeritakis, A. Fernandez, O. Ghattas, E. J. Kim, J. Lopez, D. R. O’Hallaron, T. Tu, and J. Urbanic, {\it High resolution forward and inverse earthquake modeling on terascale computers}, in Proceedings of the International Conference for High Performance Computing, Networking, Storage, and Analysis, ACM/IEEE, 2003.
[2] W. Bangerth, C. Burstedde, T. Heister, and M. Kronbichler, {\it Algorithms and data structures for massively parallel generic adaptive finite element codes}, ACM Trans. Math. Software, 38 (2011), pp. 14:1-14:28. · Zbl 1365.65247
[3] W. Bangerth, R. Hartmann, and G. Kanschat, {\it deal.II–A general-purpose object-oriented finite element library}, ACM Trans. Math. Software, 33 (2007), p. 24. · Zbl 1365.65248
[4] C. Burstedde, {\it p4est: Parallel AMR on forests of octrees}, http://www.p4est.org/http://www.p4est.org/. · Zbl 1230.65106
[5] C. Burstedde, G. Stadler, L. Alisic, L. C. Wilcox, E. Tan, M. Gurnis, and O. Ghattas, {\it Large-scale adaptive mantle convection simulation}, Geophys. J. Int., 192 (2013), pp. 889-906.
[6] C. Burstedde, L. C. Wilcox, and O. Ghattas, {\it p4est: Scalable algorithms for parallel adaptive mesh refinement on forests of octrees}, SIAM J. Sci. Comput., 33 (2011), pp. 1103-1133. · Zbl 1230.65106
[7] P. Colella, J. Bell, N. Keen, T. J. Ligocki, M. Lijewski, and B. V. Straalen, {\it Performance and scaling of locally-structured grid methods for partial differential equations}, J. Phys. Conf. Ser., 78 (2007), pp. 1-13.
[8] P. Colella, D. T. Graves, N. Keen, T. J. Ligocki, D. F. Martin, P. W. McCorquodale, D. Modiano, P. O. Schwartz, T. D. Sternberg, and B. Van Straalen, {\it Chombo Software Package for AMR Applications. Design Document.}, Applied Numerical Algorithms Group, NERSC Division, Lawrence Berkeley National Laboratory, Berkeley, CA, 2007.
[9] K. Devine, E. Boman, R. Heaphy, B. Hendrickson, and C. Vaughan, {\it Zoltan data management services for parallel dynamic applications}, Comput. Sci. Eng., 4 (2002), pp. 90-97.
[10] J. Dreher and R. Grauer, {\it Racoon: A parallel mesh-adaptive framework for hyperbolic conservation laws}, Parallel Comput., 31 (2005), pp. 913-932.
[11] P. F. Fischer, G. W. Kruse, and F. Loth, {\it Spectral element methods for transitional flows in complex geometries}, J. Sci. Comput., 17 (2002), pp. 81-98. · Zbl 1001.76075
[12] T. Goodale, G. Allen, G. Lanfermann, J. Masso, T. Radke, E. Seidel, and J. Shalf, {\it The Cactus framework and toolkit: Design and applications}, in Proceedings of the 5th International Conference Vector and Parallel Processing, Springer, New York, 2003. · Zbl 1027.65524
[13] R. A. Haring, M. Ohmacht, T. W. Fox, M. K. Gschwind, D. L. Satterfield, K. Sugavanam, P. W. Coteus, P. Heidelberger, M. A. Blumrich, R. W. Wisniewski, et al., {\it The IBM Blue Gene/Q compute chip}, Micro, IEEE, 32 (2012), pp. 48-60.
[14] T. Isaac, C. Burstedde, and O. Ghattas, {\it Low-cost parallel algorithms for 2:1 octree balance}, in Proceedings of the 26th IEEE International Parallel & Distributed Processing Symposium, IEEE, 2012; also available online from http://dx.doi.org/10.1109/IPDPS.2012.47.
[16] O. S. Lawlor, S. Chakravorty, T. L. Wilmarth, N. Choudhury, I. Dooley, G. Zheng, and L. V. Kalé, {\it ParFUM: A parallel framework for unstructured meshes for scalable dynamic physics applications}, Engineering with Computers, 22 (2006), pp. 215-235.
[17] J. Luitjens, M. Berzins, and T. C. Henderson, {\it Scalable parallel AMR for the Uintah multiphysics code}, in Petascale Computing Algorithms and Applications, D. A. Bader, ed., Chapman & Hall, London, 2008, pp. 67-81.
[18] P. MacNeice, K. M. Olson, C. Mobarry, R. de Fainchtein, and C. Packer, {\it Paramesh: A parallel adaptive mesh refinement community toolkit}, Comput. Phys. Comm., 126 (2000), pp. 330-354. · Zbl 0953.65088
[19] J. P. May, {\it A Concise Course in Algebraic Topology}, University of Chicago Press, Chicago, 1999. · Zbl 0923.55001
[20] G. M. Morton, {\it A computer oriented geodetic data base; and a new technique in file sequencing}, Technical report, IBM, 1966.
[21] C. D. Norton, J. Z. Lou, and T. A. Cwik, {\it Status and directions for the PYRAMID parallel unstructured AMR library}, in Proceedings of the 15th IEEE International Parallel and Distributed Processing Symposium (IPDPS), 2001, p. 120.
[22] S. Popinet, {\it Gerris: A tree-based adaptive solver for the incompressible Euler equations in complex geometries}, J. Comput. Phys., 190 (2003), pp. 572-600. · Zbl 1076.76002
[23] W. C. Rheinboldt and C. K. Mesztenyi, {\it On a data structure for adaptive finite element mesh refinements}, ACM Trans. Math. Software, 6 (1980), pp. 166-187. · Zbl 0437.65081
[24] R. S. Sampath, S. S. Adavani, H. Sundar, I. Lashuk, and G. Biros, {\it Dendro: Parallel algorithms for multigrid and AMR methods on 2:1 balanced octrees}, in Proceedings of the International Conference for High Performance Computing, Networking, Storage, and Analysis, ACM/IEEE, 2008.
[25] C. Sert and A. Beskok, {\it Spectral element formulations on non-conforming grids: A comparative study of pointwise matching and integral projection methods}, J. Comput. Phys., 211 (2006), pp. 300-325. · Zbl 1120.65338
[26] J. R. Stewart and H. C. Edwards, {\it A framework approach for developing parallel adaptive multiphysics applications}, Finite Elem. Anal. Des., 40 (2004), pp. 1599-1617.
[27] H. Sundar, R. Sampath, and G. Biros, {\it Bottom-up construction and 2:1 balance refinement of linear octrees in parallel}, SIAM J. Sci. Comput., 30 (2008), pp. 2675-2708. · Zbl 1186.68554
[28] T. Tu, D. R. O’Hallaron, and O. Ghattas, {\it Scalable parallel octree meshing for terascale applications}, in Proceedings of the International Conference for High Performance Computing, Networking, Storage, and Analysis, ACM/IEEE, 2005.
[29] L. C. Wilcox, G. Stadler, C. Burstedde, and O. Ghattas, {\it A high-order discontinuous Galerkin method for wave propagation through coupled elastic-acoustic media}, J. Comput. Phys., 229 (2010), pp. 9373-9396. · Zbl 1427.74071
[30] M. Zhou, O. Sahni, K. D. Devine, M. S. Shephard, and K. E. Jansen, {\it Controlling unstructured mesh partitions for massively parallel simulations}, SIAM J. Sci. Comput., 32 (2010), pp. 3201-3227. · Zbl 1221.65358
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.