Performance evaluation of a parallel sparse lattice Boltzmann solver. (English) Zbl 1137.76045
Summary: We develop a performance prediction model for a parallelized sparse lattice Boltzmann solver and present performance results for simulations of flow in a variety of complex geometries. A special focus is on partitioning and memory/load balancing strategy for geometries with a high solid fraction and/or complex topology such as porous media, fissured rocks and geometries from medical applications. The topology of lattice nodes representing the fluid fraction of the computational domain is mapped on a graph. Graph decomposition is performed with both multilevel recursive-bisection and multilevel \(k\)-way schemes based on modified Kernighan-Lin and Fiduccia-Mattheyses partitioning algorithms. Performance results and optimization strategies are presented for a variety of platforms, showing a parallel efficiency of almost 80% for the largest problem size. A good agreement between the performance model and experimental results is demonstrated.
Software:
LUDWIGReferences:
| [1] | Succi, Sauro, The Lattice Boltzmann Equation for Fluid Dynamics and Beyond (2001), Oxford: Oxford New York · Zbl 0990.76001 |
| [2] | Chen, Shiyi; Doolen, Garry D., Lattice Boltzmann method for fluid flows, Annual Review Fluid Mechanics, 30, 329-364 (1998) · Zbl 1398.76180 |
| [3] | Qian, Y. H.; d’Humieres, D.; Lallemand, P., Lattice BGK models for Navier-Stokes equation, Europhysics Letters, 17, 479-484 (1992) · Zbl 1116.76419 |
| [4] | Koponen, A.; Kataja, M.; Timonen, J.; Kandhai, Drona, Simulation of single-fluid flow in porous media, International Journal of Modern Physics C, 9, 1505-1522 (1998) |
| [5] | Kandhai, Drona; Hlushkou, D.; Hoekstra, Alfons G.; Sloot, Peter M. A.; van As, H.; Tallarek, U., Influence of stagnant zones on transient and asymptotic dispersion in macroscopically homogeneous porous media, Physical Review Letters, 88, 23, 234501-1-234501-4 (2002) |
| [6] | Clague, D. S.; Kandhai, Drona; Zhang, R.; Sloot, Peter M. A., On the hydraulic permeability of (un)bounded fibrous media using the lattice-Boltzmann method, Physical Review E, 61, 1, 616-625 (2000) |
| [7] | Chen, Shiyi; Wang, Zheng; Shan, Xiaowen; Doolen, Gary D., Lattice Boltzmann computational fluid dynamics in three dimensions, Journal of Statistical Physics, 68, 3/4, 379-400 (1992) · Zbl 0925.76516 |
| [8] | Artoli, Abdel M.; Hoekstra, Alfons G.; Sloot, Peter M. A., Mesoscophic simulations of systolic flow in the human abdominal aorta, Journal of Biomechanics, 39, 5, 873-884 (2006) |
| [9] | Artoli, Abdel M. M.; Hoekstra, Alfons G.; Sloot, Peter M. A., Simulation of a systolic cycle in a realistic artery with the lattice Boltzmann BGK method, International Journal of Modern Physics B, 17, 1-2, 95-98 (2003) |
| [10] | Cosgrove, J. A.; Buick, J. M.; Tonge, S. J.; Munro, C. G.; Greated, C. A.; Campbell, D. M., Application of the lattice Boltzmann method to transition in oscillatory channel flow, Journal of Physics A: Mathematical and General, 36, 2609-2620 (2003) · Zbl 1064.76086 |
| [11] | Fang, Haiping; Wamg, Zuowei; Lin, Zhifang; Liu, Muren, Lattice Boltzmann method for simulating the viscous flow in large destensible blood vessels, Physical Review E, 65, 051925-052011 (2002) |
| [12] | Artoli, Abdel M.; Hoekstra, Alfons G.; Sloot, Peter M. A., Optimizing lattice Boltzmann simulations for unsteady flows, Computers & Fluids, 35, 2, 227-240 (2006) · Zbl 1099.76051 |
| [13] | Kandhai, Drona; Koponen, A.; Hoekstra, Alfons G.; Kataja, M.; Timonen, J.; Sloot, Peter M. A., Lattice Boltzmann hydrodynamics on parallel systems, Computer Physics Communications, 111, 14-26 (1998) · Zbl 0942.76062 |
| [14] | Li, Wei; Wei, Xiaoming; Kaufman, Arie E., Implementing lattice Boltzmann computation on graphics hardware, The Visual Computer, 19, 7-8, 444-456 (2003) |
| [15] | Desplat, Jean-Christophe; Pagonabarraga, Ignacio; Ludwig, Peter Bladon., A parallel lattice-Boltzmann code for complex fluids, Computer Physics Communications, 134, 3, 273 (2002) · Zbl 1032.76055 |
| [16] | Pohl, Thomas; Deserno, Frank; Thurey, Nils; Rude, Ulrich; Lammers, Peter; Wellein, Gerhard; Zeiser, Thomas, Performance evaluation of parallel large-scale lattice Boltzmann applications on three supercomputing architectures, (SC’04: Proceedings of the 2004 ACM/IEEE Conference on Supercomputing (2004), IEEE Computer Society: IEEE Computer Society Washington, DC, USA), 21 |
| [17] | M. Schulz, M. Krafczyk, J. Toelke, E. Rank, Parallelization strategies and efficiency of CFD computations in complex geometries using lattice-Blotzmann methods on high-performance computers, in: 3rd International FORTWIHR Conference on HPSEC, Erlangen, March 12-14, 2001, 2002, p. 115.; M. Schulz, M. Krafczyk, J. Toelke, E. Rank, Parallelization strategies and efficiency of CFD computations in complex geometries using lattice-Blotzmann methods on high-performance computers, in: 3rd International FORTWIHR Conference on HPSEC, Erlangen, March 12-14, 2001, 2002, p. 115. |
| [18] | Dupuis, Alexandre; Chopard, Bastien, An object oriented approach to lattice gas modeling, Future Generation Computer Systems, 16, 5, 523-532 (2000) |
| [19] | Tlke, J.; Freudiger, S.; Krafczyk, M., An adaptive scheme for LBE multiphase flow simulations on hierarchical grids, Computers and Fluids, 35, 8-9, 820 (2006) · Zbl 1177.76332 |
| [20] | Lammers, P.; Wellein, G.; Zeiser, T.; Breuer, M., Have the vectors the continuing ability to parry the attack of the killer micros?, (Resch, M.; Bönisch, T.; Benkert, K.; Furui, T.; Seo, Y.; Bez, W., High Performance Computing on Vector Systems. Proceedings of the High Performance Computing Center Stuttgart, March 2005 (2006), Springer: Springer Berlin), 25-39 · Zbl 1303.76094 |
| [21] | Wellein, G.; Zeiser, T.; Lammers, P.; Küster, U., Towards optimal performance for lattice Boltzmann applications on terascale computers, (Deane, A.; Brenner, G.; Ecer, A.; etal., Parallel Computational Fluid Dynamics: Theory and Applications, Proceedings of the 2005 International Conference on Parallel Computational Fluid Dynamics, May 24-27, College Park, MD, USA (2006), Elsevier: Elsevier Amsterdam), 31-40 |
| [22] | Pan, C.; Prins, J.; Miller, C. T., A high-performance lattice Boltzmann implementation to model flow in porous media, Computer Physics Communication, 158, 2, 89 (2004) · Zbl 1196.76069 |
| [23] | Karypis, George; Kumar, Vipin, Multilevel \(k\)-way partitioning scheme for irregular graphs, Journal of Parallel and Distributed Computing, 481, 1, 96-129 (1998) · Zbl 0918.68073 |
| [24] | Amine Abou-Rjeili, George Karypis, Multilevel algorithms for partitioning power-law graphs, in: IEEE International Parallel & Distributed Processing Symposium (IPDPS), vol. 10, 2006.; Amine Abou-Rjeili, George Karypis, Multilevel algorithms for partitioning power-law graphs, in: IEEE International Parallel & Distributed Processing Symposium (IPDPS), vol. 10, 2006. |
| [25] | George Karypis, Vipin Kumar. Multilevel graph partitioning schemes, in: International Conference on Parallel Processing, vol. 3, 1995, p. 113-122.; George Karypis, Vipin Kumar. Multilevel graph partitioning schemes, in: International Conference on Parallel Processing, vol. 3, 1995, p. 113-122. · Zbl 0918.68073 |
| [26] | Fox, G.; Johnson, M.; Lyzenga, G.; Otto, S.; Solman, J.; Walker, D., Solving Problems on Concurent Processors (1988), Prentice-Hall Int., Inc. |
| [27] | J. Bernsdorf, S.E. Harrison, S.M. Smith, P.V. Lawford, D.R. Hose, High performance computing on vector systems 2006, in: Proceedings of the High Performance Computing Center Stuttgart, March, 2006.; J. Bernsdorf, S.E. Harrison, S.M. Smith, P.V. Lawford, D.R. Hose, High performance computing on vector systems 2006, in: Proceedings of the High Performance Computing Center Stuttgart, March, 2006. · Zbl 1102.76079 |
| [28] | Wellein, G.; Hager, G.; Zeiser, T.; Donath, S., On the single processor performance of simple lattice Boltzmann kernels, Computers and Fluids, 35, 8-9, 910-919 (2006) · Zbl 1177.76335 |
| [29] | J.B. White III, S.W. Bova, Where’s the overlap? An analysis of popular MPI implementations, in: MPIDC’99: Proceedings of the Message Passing Interface Developer’s and User’s Conference 1999, March 10-12, Atlanta, 1999.; J.B. White III, S.W. Bova, Where’s the overlap? An analysis of popular MPI implementations, in: MPIDC’99: Proceedings of the Message Passing Interface Developer’s and User’s Conference 1999, March 10-12, Atlanta, 1999. |
| [30] | http://www.hpce.nec.com/; http://www.hpce.nec.com/ |
| [31] | http://www.sara.nl/userinfo/lisa/description/index.html/; http://www.sara.nl/userinfo/lisa/description/index.html/ |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
