Adaptive kinetic-fluid solvers for heterogeneous computing architectures. (English) Zbl 1349.76754
Summary: We show feasibility and benefits of porting an adaptive multi-scale kinetic-fluid code to CPU-GPU systems. Challenges are due to the irregular data access for adaptive Cartesian mesh, vast difference of computational cost between kinetic and fluid cells, and desire to evenly load all CPUs and GPUs during grid adaptation and algorithm refinement. Our Unified Flow Solver (UFS) combines Adaptive Mesh Refinement (AMR) with automatic cell-by-cell selection of kinetic or fluid solvers based on continuum breakdown criteria. Using GPUs enables hybrid simulations of mixed rarefied-continuum flows with a million of Boltzmann cells each having a \(24 \times 24 \times 24\) velocity mesh. We describe the implementation of CUDA kernels for three modules in UFS: the direct Boltzmann solver using the discrete velocity method (DVM), the Direct Simulation Monte Carlo (DSMC) solver, and a mesoscopic solver based on the Lattice Boltzmann Method (LBM), all using adaptive Cartesian mesh. Double digit speedups on single GPU and good scaling for multi-GPUs have been demonstrated.
MSC:
| 76M28 | Particle methods and lattice-gas methods |
| 65M75 | Probabilistic methods, particle methods, etc. for initial value and initial-boundary value problems involving PDEs |
| 65Y05 | Parallel numerical computation |
| 65Y10 | Numerical algorithms for specific classes of architectures |
Keywords:
unified flow solver; adaptive mesh refinement; discrete velocity method; Boltzmann kinetic equation; direct simulation Monte Carlo; lattice Boltzmann method; graphics processing units; CUDA; MPIReferences:
| [1] | Birdsall, C. K.; Langdon, A. B., Plasma Physics via Computer Simulations (2004), Taylor and Francis |
| [2] | Bird, G. A., Molecular Gas Dynamics and the Direct Simulation of Gas Flows (1994), Oxford Science Publications |
| [3] | Aristov, V. V., Direct Methods for Solving the Boltzmann Equation and Study of Non-Equilibrium Flows (2001), Kluwer Academic Publishers: Kluwer Academic Publishers Dordrecht · Zbl 0983.82011 |
| [4] | (Shoucri, M., Eulerian Codes for the Numerical Solution of the Kinetic Equations of Plasmas (2010), Nova Publishers) |
| [5] | Kolobov, V. I.; Arslanbekov, R. R.; Aristov, V. V.; Frolova, A. A.; Zabelok, S. A., Unified solver for rarefied and continuum flows with adaptive mesh and algorithm refinement, J. Comput. Phys., 223, 589 (2007) · Zbl 1183.76851 |
| [6] | Miller, R. E.; Tadmor, E. B., A unified framework and performance benchmark of fourteen multiscale atomistic/continuum coupling methods, Model. Simul. Mater. Sci. Eng., 17, Article 053001 pp. (2009) |
| [7] | Tóth, G., Adaptive numerical algorithms in space weather modeling, J. Comput. Phys., 231, 870-903 (2012) |
| [8] | Kolobov, V. I.; Arslanbekov, R. R., Towards adaptive kinetic-fluid simulations of weakly ionized plasmas, J. Comput. Phys., 231, 839 (2012) · Zbl 1417.76030 |
| [9] | Radtke, G. A.; Péraud, J.-P. M.; Hadjiconstantinou, N. G., On efficient simulations of multiscale kinetic transport, Philos. Trans. R. Soc. A, 371, Article 20120182 pp. (2013) · Zbl 1353.76030 |
| [10] | Rieke, M.; Trost, T.; Grauer, R., Coupled Vlasov and two-fluid codes on GPUs, J. Comput. Phys., 283, 436 (2015) · Zbl 1351.76316 |
| [11] | Garsia, A. L.; Bell, J. B.; Crutchfield, W. Y.; Alder, B. J., Adaptive mesh and algorithm refinement using direct simulation Monte Carlo, J. Comp. Physiol., 54, 134 (1999) · Zbl 0954.76075 |
| [12] | Arslanbekov, R. R.; Kolobov, V. I.; Frolova, A. A., Analysis of compressible viscous flow solvers with adaptive Cartesian mesh, (20th AIAA Computational Fluid Dynamics Conference (2011)), AIAA 2011-3381 |
| [14] | Gerris Flow Solver |
| [15] | Agbaglah, G., Parallel simulation of multiphase flows using octree adaptivity and the volume-of-fluid method, C. R., Méc., 339, 194 (2011) · Zbl 1217.76044 |
| [16] | Arslanbekov, R. R.; Kolobov, V. I.; Frolova, A. A., Kinetic solvers with adaptive mesh in phase space, Phys. Rev. E, 88, Article 063301 pp. (2013) |
| [18] | Zabelok, S.; Arslanbekov, R.; Kolobov, V., Multi-GPU kinetic solvers using MPI and CUDA, (Fan, Jing, Rarefied Gas Dynamics, Proc. 29th Int. Symp.. Rarefied Gas Dynamics, Proc. 29th Int. Symp., AIP Conference Proc., vol. 1628 (2014), Melville: Melville New York), 539-546 |
| [20] | Navarro, C. A.; Hitschfeld-Kahler, N.; Mateu, L., A survey on parallel computing and its applications in data-parallel problems using GPU architectures, Commun. Comput. Phys., 15, 285 (2014) · Zbl 1388.65212 |
| [22] | Vanka, S. P., Computational fluid dynamics on graphics processing units: 2012 Freeman scholar lecture, J. Fluids Eng., 135, Article 061401 pp. (2013) |
| [24] | Goldsworthy, M. J., A GPU-CUDA based direct simulation Monte Carlo algorithm for real gas flows, Comput. Fluids, 94, 58-68 (2014) · Zbl 1391.76221 |
| [25] | Kashkovsky, A., 3D DSMC computations on a heterogeneous CPU-GPU cluster with a large number of GPUs, AIP Conf. Proc., 1628, 192 (2014) |
| [26] | Kloss, Y.; Shuvalov, P.; Tcheremissine, F., Solving Boltzmann equation on GPU, Proc. Comput. Sci., 1, 1083-1091 (2010) |
| [27] | Frezzotti, A.; Ghiroldi, G.; Gibelli, L., Solving the Boltzmann equation on GPUs, Comput. Fluids, 50, 136-146 (2011) · Zbl 1271.76301 |
| [29] | Zabelok, S. A.; Aristov, V. V.; Frolova, A. A.; Kolobov, V. I.; Arslanbekov, R. R., Parallel implementation of the unified flow solver, (Rarefied Gas Dynamics: 25th International Symposium. Rarefied Gas Dynamics: 25th International Symposium, St. Petersburg, Russia (2006)) · Zbl 1183.76851 |
| [30] | Degond, P.; Dimarco, G.; Mieussens, L., A multiscale kinetic-fluid solver with dynamic localization of kinetic effects, J. Comput. Phys., 229, 4907-4933 (2010) · Zbl 1346.82035 |
| [31] | Dimarco, G.; Pareschi, L., Numerical methods for kinetic equations, Acta Numer., 23, 369-520 (2014) · Zbl 1398.65260 |
| [32] | Malkov, E. A.; Ivanov, M. S.; Poleshkin, S. O., Discrete velocity scheme for solving the Boltzmann equation accelerated with the GP GPU, (Mareshal, M.; Santos, A., 28th International Symposium on Rarefied Gas Dynamics. 28th International Symposium on Rarefied Gas Dynamics, AIP Conf. Proc., vol. 1501 (2012), American Institute of Physics: American Institute of Physics Melville, NY), 318-325 |
| [33] | Zabelok, S. A.; Stroganov, A. V.; Aristov, V. V., Deterministic GPU Boltzmann solver, (Fan, Jing, Rarefied Gas Dynamics, Proc. 29th Int. Symp.. Rarefied Gas Dynamics, Proc. 29th Int. Symp., AIP Conf. Proc., vol. 1628 (2014), Melville: Melville New York), 1009-1015 |
| [34] | Shan, X.; Yuan, X.-F.; Chen, H., Kinetic theory representation of hydrodynamics: a way beyond the Navier-Stokes equation, J. Fluid Mech., 550, 413 (2006) · Zbl 1097.76061 |
| [35] | Ambrus, V. E.; Sofonea, V., High-order thermal lattice Boltzmann models derived by means of Gauss quadrature in the spherical coordinate system, Phys. Rev. E, 86, Article 016708 pp. (2012) |
| [36] | Eitel-Amor, G.; Meinke, M.; Schroder, W., A lattice-Boltzmann method with hierarchically refined meshes, Comput. Fluids, 75, 127 (2013) · Zbl 1277.76082 |
| [37] | Fakhari, A.; Lee, T., Finite-difference lattice Boltzmann method with a block-structured adaptive-mesh-refinement technique, Phys. Rev. E, 89, Article 033310 pp. (2014) |
| [40] | Januszewski, M.; Kostur, M., Sailfish: a flexible multi-GPU implementation of the lattice Boltzmann method, Comput. Phys. Commun., 185, 2350-2368 (2014) · Zbl 1344.76004 |
| [41] | Mazzeo, M. D.; Coveney, P. V., HemeLB: a high performance parallel lattice-Boltzmann code for large scale fluid flow in complex geometries, Comput. Phys. Commun., 178, 894-914 (2008) · Zbl 1196.76008 |
| [42] | Desplat, J.-C.; Pagonabarraga, I.; Bladon, P., LUDWIG: a parallel lattice-Boltzmann code for complex fluids, Comput. Phys. Commun., 134, 273-290 (2001) · Zbl 1032.76055 |
| [43] | Hasert, M.; Masilamani, K.; Zimny, S.; Klimach, H.; Qi, J.; Bernsdorf, J.; Roller, S., Complex fluid simulations with the parallel tree-based lattice Boltzmann solver Musubi, J. Comput. Sci., 5, 784-794 (2014) |
| [44] | Patil, D. V.; Lakshmisha, K. N., Finite volume TVD formulation of lattice Boltzmann simulation on unstructured mesh, J. Comput. Phys., 228, 5262-5279 (2009) · Zbl 1280.76054 |
| [45] | Wolfe, M., Understanding the CUDA data parallel threading model |
| [47] | Ivanov, M. S., Parallel object-oriented software system for DSMC, (27th Int. Symp. on Rarefied Gas Dynamics. 27th Int. Symp. on Rarefied Gas Dynamics, AIP Conf. Proc., vol. 1333 (2011)), 211 |
| [48] | Arslanbekov, R. R.; Kolobov, V. I.; Frolova, A. A., Immersed boundary method for Boltzmann and Navier-Stokes solvers with adaptive Cartesian mesh, (27 Symp. on Rarefied Gas Dynamics. 27 Symp. on Rarefied Gas Dynamics, AIP Conf. Proc., vol. 1333 (2011)), 873 |
| [49] | Dechriste, G.; Mieussens, L., A Cartesian cut cell method for rarefied flow simulations around moving obstacles · Zbl 1349.76783 |
| [50] | Shrestha, S.; Tiwari, S.; Klar, A.; Hardt, S., Numerical simulation of a moving rigid body in a rarefied gas, J. Comput. Phys., 292, 239-252 (2015) · Zbl 1349.76788 |
| [51] | GNU Triangulated Surface Library |
| [52] | Su, C.-C.; Smith, M. R.; Kuo, F.-A.; Wu, J.-S.; Hsieh, C.-W.; Tseng, K.-C., Large-scale simulations on multiple graphics processing units (GPUs) for the direct simulation Monte Carlo method, J. Comp. Physiol., 231, 7932 (2012) |
| [53] | Ziegler, U., Self-gravitational adaptive mesh magnetohydrodynamics with the NIRVANA code, Astron. Astrophys., 435, 385 (2005) |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
