×

A robust, discrete-gradient descent procedure for optimisation with time-dependent PDE and norm constraints. (English) Zbl 1536.65119

The paper addresses the challenge of efficiently solving norm-constrained, nonlinear optimal control problems in fluid dynamics. These problems are crucial in various applications, such as improving transport and mixing efficiencies in complex fluids, analyzing the stability of thermo-acoustic oscillations, modeling transitions between multiple basins of attraction in dynamical systems, and forecasting oceanographic systems. Due to the nonlinear nature of the governing partial differential equations (PDEs) and the high computational cost of direct numerical simulations, efficient optimization routines are essential for tackling these problems.
The authors propose a numerically consistent, discrete formulation of the direct-adjoint looping method, accompanied by gradient descent and line-search algorithms with global convergence guarantees. They derive the discrete adjoint equations for time-dependent PDE constraints using multistep schemes and compare the performance of the discrete and continuous adjoint approaches on several example problems. Furthermore, the authors utilize tools from the optimization on manifolds community to develop a consistent and efficient gradient update procedure that restricts the control variable to a spherical manifold, eliminating the need for equality constraints.
The main findings of the manuscript demonstrate the superiority of the discrete adjoint approach in providing accurate gradient estimates and the effectiveness of the proposed gradient descent procedure in achieving robust convergence to stationary points with small residual errors. The authors test their methodology on three fluid dynamics problems: a dynamo problem, the Swift-Hohenberg multi-stability problem, and an optimal mixing problem. In all cases, the discrete adjoint formulation outperforms the continuous adjoint approach, and the gradient descent procedure exhibits linear to super-linear convergence rates.
The significance of this research lies in its contribution to the efficient solution of norm-constrained optimization problems in fluid dynamics. The proposed discrete adjoint formulation and gradient descent procedure offer a robust and computationally efficient framework for tackling a wide range of applications in the field. Moreover, the authors provide an accompanying library, SphereManOpt, which implements their methodology and can be readily used by the scientific community. This work has the potential to advance the understanding and optimization of complex fluid systems, with implications for various domains, such as oceanography, mixing processes, and stability analysis.

MSC:

65M70 Spectral, collocation and related methods for initial value and initial-boundary value problems involving PDEs
65M06 Finite difference methods for initial value and initial-boundary value problems involving PDEs
65N35 Spectral, collocation and related methods for boundary value problems involving PDEs
65K10 Numerical optimization and variational techniques
65M12 Stability and convergence of numerical methods for initial value and initial-boundary value problems involving PDEs
65M15 Error bounds for initial value and initial-boundary value problems involving PDEs
76W05 Magnetohydrodynamics and electrohydrodynamics
86A30 Geodesy, mapping problems
49M41 PDE constrained optimization (numerical aspects)
41A60 Asymptotic approximations, asymptotic expansions (steepest descent, etc.)
41A50 Best approximation, Chebyshev systems
93C20 Control/observation systems governed by partial differential equations
15A15 Determinants, permanents, traces, other special matrix functions
35Q35 PDEs in connection with fluid mechanics
35Q86 PDEs in connection with geophysics

References:

[1] Absil, Pierre-Antoine; Mahony, Robert; Sepulchre, Rodolphe, Optimization algorithms on matrix manifolds, (2009), Princeton University Press
[2] Ascher, Uri M.; Ruuth, Steven J.; Spiteri, Raymond J., Implicit-explicit Runge-Kutta methods for time-dependent partial differential equations, Appl. Numer. Math., 25, 151-167, (1997) · Zbl 0896.65061 · doi:10.1016/S0168-9274(97)00056-1
[3] Ascher, Uri M.; Ruuth, Steven J.; Wetton, Brian T. R., Implicit-explicit methods for time-dependent partial differential equations, SIAM J. Numer. Anal., 32, 797-823, (1995) · Zbl 0841.65081 · doi:10.1137/0732037
[4] Auroux, Didier; Blum, Jacques, Multidisciplinary methods for analysis optimization and control of complex systems, 6, Data assimilation methods for an oceanographic problem, 180-194, (2005), Springer · Zbl 1125.86315 · doi:10.1007/3-540-27167-8_4
[5] Berg, Jens; Nordström, Jan, On the impact of boundary conditions on dual consistent finite difference discretizations, J. Comput. Phys., 236, 41-55, (2013) · Zbl 1286.65107 · doi:10.1016/j.jcp.2012.11.019
[6] Boumal, Nicolas, An introduction to optimization on smooth manifolds, (2022) · Zbl 1532.90001
[7] Boumal, Nicolas; Mishra, Bamdev; Absil, Pierre-Antoine; Sepulchre, Rodolphe, Manopt, a Matlab Toolbox for Optimization on Manifolds, J. Mach. Learn. Res., 15, 42, 1455-1459, (2014) · Zbl 1319.90003
[8] Burns, Keaton J.; Vasil, Geoffrey M.; Oishi, Jeffrey S.; Lecoanet, Daniel; Brown, Benjamin P., Dedalus: A flexible framework for numerical simulations with spectral methods, Phys. Rev. Res., 2, (2020) · doi:10.1103/physrevresearch.2.023068
[9] Dauxois, Thierry; Peacock, Thomas; Bauer, Peter; Caulfield, Colm P.; Cenedese, Claudia; Gorlé, Catherine; Haller, George; Ivey, Greg N.; Linden, Paul F.; Meiburg, Eckart, Confronting grand challenges in environmental fluid mechanics, Phys. Rev. Fluids, 6, 2, (2021)
[10] De Pando, Miguel Fosas; Sipp, Denis; Schmid, Peter J., Efficient evaluation of the direct and adjoint linearized dynamics from compressible flow solvers, J. Comput. Phys., 231, 23, 7739-7755, (2012) · Zbl 1322.76047 · doi:10.1016/j.jcp.2012.06.038
[11] Douglas, S. C.; Amari, S.; Kung, S. Y., Ninth IEEE Signal Processing Workshop on Statistical Signal and Array Processing (Cat. No.98TH8381), Gradient adaptation under unit-norm constraints, (1998), IEEE · doi:10.1109/ssap.1998.739355
[12] Eggl, Maximilian F.; Schmid, Peter J., A gradient-based framework for maximizing mixing in binary fluids, J. Comput. Phys., 368, 131-153, (2018) · Zbl 1392.76019 · doi:10.1016/j.jcp.2018.04.030
[13] Farrell, Patrick E.; Ham, David A.; Funke, Simon W.; Rognes, Marie E., Automated derivation of the adjoint of high-level transient finite element programs, SIAM J. Sci. Comput., 35, 4, C369-C393, (2013) · Zbl 1362.65103 · doi:10.1137/120873558
[14] Fikl, Alexandru; Le Chenadec, Vincent; Sayadi, Taraneh, Control and optimization of interfacial flows using adjoint-based techniques, Fluids, 5, 3, 156, (2020) · doi:10.3390/fluids5030156
[15] Fikl, Alexandru; Le Chenadec, Vincent; Sayadi, Taraneh; Schmid, Peter J., A comprehensive study of adjoint-based optimization of non-linear systems with application to Burgers’ equation, 46th AIAA Fluid Dynamics Conference, 3805, (2016)
[16] Foures, Dimitry P. G.; Caulfield, Colm P.; Schmid, Peter J., Localization of flow structures using-norm optimization, J. Fluid Mech., 729, 672-701, (2013) · Zbl 1291.76124 · doi:10.1017/jfm.2013.333
[17] Foures, Dimitry P. G.; Caulfield, Colm P.; Schmid, Peter J., Optimal mixing in two-dimensional plane Poiseuille flow at finite Péclet number, J. Fluid Mech., 748, 241-277, (2014) · Zbl 1416.76036 · doi:10.1017/jfm.2014.182
[18] Franco, Michael; Persson, Per-Olof; Pazner, Will; Zahr, Matthew J., AIAA Scitech 2019 Forum, An adjoint method using fully implicit runge-kutta schemes for optimization of flow problems, (2019) · doi:10.2514/6.2019-0351
[19] Griewank, Andreas; Walther, Andrea, Algorithm 799: revolve: an implementation of checkpointing for the reverse or adjoint mode of computational differentiation, ACM Trans. Math. Softw., 26, 1, 19-45, (2000) · Zbl 1137.65330 · doi:10.1145/347837.347846
[20] Hager, William W., Runge-Kutta methods in optimal control and the transformed adjoint system, Numer. Math., 87, 247-282, (2000) · Zbl 0991.49020 · doi:10.1007/s002110000178
[21] Hager, William W.; Zhang, Hongchao, A survey of nonlinear conjugate gradient methods, Pac. J. Optim., 2, 1, 35-58, (2006) · Zbl 1117.90048
[22] Hartmann, Ralf, Adjoint consistency analysis of discontinuous Galerkin discretizations, SIAM J. Numer. Anal., 45, 6, 2671-2696, (2007) · Zbl 1189.76341 · doi:10.1137/060665117
[23] Hascoët, Laurent; Pascual, Valérie, The Tapenade Automatic Differentiation tool: Principles, Model, and Specification, ACM Trans. Math. Softw., 39, 3, 43 p. pp., (2013) · Zbl 1295.65026 · doi:10.1145/2450153.2450158
[24] Heffernan, Conor; Caulfield, Colm P., Robust and efficient identification of optimal mixing perturbations using proxy multiscale measures, Philos. Trans. R. Soc. Lond., Ser. A, 380, 2225, 17 p. pp., (2022) · Zbl 1515.76139
[25] Hinze, Michael; Pinnau, René; Ulbrich, Michael; Ulbrich, Stefan, Optimization with PDE constraints, 23, (2008), Springer
[26] Johnson, Steven G., The NLopt nonlinear-optimization package, (2014)
[27] Juniper, Matthew P.; Sujith, R. I., Sensitivity and nonlinearity of thermoacoustic oscillations, Annu. Rev. Fluid Mech., 50, 661-689, (2018) · Zbl 1384.76050 · doi:10.1146/annurev-fluid-122316-045125
[28] Kerswell, Rich R., Nonlinear Nonmodal Stability Theory, Annu. Rev. Fluid Mech., 50, 319-345, (2018) · Zbl 1384.76022 · doi:10.1146/annurev-fluid-122316-045042
[29] Knobloch, E., Spatial Localization in Dissipative Systems, Ann. Rev. Cond. Matter Phys., 6, 1, 325-359, (2015) · doi:10.1146/annurev-conmatphys-031214-014514
[30] Kord, Ali; Capecelatro, Jesse, Optimal perturbations for controlling the growth of a Rayleigh-Taylor instability, J. Fluid Mech., 876, 150-185, (2019) · Zbl 1419.76224 · doi:10.1017/jfm.2019.532
[31] Kraft, Dieter, A software package for sequential quadratic programming, 28, 33 p. pp., (1988), DFVLR Obersfaffeuhofen · Zbl 0646.90065
[32] Larmor, Joseph, 17. How Could a Rotating Body such as the Sun Become a Magnet?, Reports of the British Association, 87, 159-160, (1919)
[33] Lecoanet, Daniel; Kerswell, Rich R., Connection between nonlinear energy optimization and instantons, Phys. Rev. E, 97, (2018) · doi:10.1103/PhysRevE.97.012212
[34] Leykekhman, Dmitriy, Investigation of commutative properties of discontinuous Galerkin methods in PDE constrained optimal control problems, J. Sci. Comput., 53, 3, 483-511, (2012) · Zbl 1262.65071 · doi:10.1007/s10915-012-9582-y
[35] Luenberger, David G., Introduction to linear and nonlinear programming, xii+356 p. pp., (1973), Addison-wesley Reading · Zbl 0297.90044
[36] Mannix, Paul M.; Skene, Calum S.; Auroux, Didier; Marcotte, Florence, mannixp/SphereManOpt: Initial Release, (2022) · doi:10.5281/zenodo.7261386
[37] Marcotte, Florence; Caulfield, Colm P., Optimal mixing in two-dimensional stratified plane Poiseuille flow at finite Péclet and Richardson numbers, J. Fluid Mech., 853, 359-385, (2018) · Zbl 1415.76215 · doi:10.1017/jfm.2018.565
[38] Mathew, George; Mezić, Igor; Petzold, Linda, A multiscale measure for mixing, Physica D, 211, 1-2, 23-46, (2005) · Zbl 1098.37067 · doi:10.1016/j.physd.2005.07.017
[39] Maugars, Bruno; Bourasseau, Sébastien; Content, Cédric; Michel, Bertrand; Berthoul, Bérenger; Ramirez, Jorge Nunez; Raud, Pascal; Hascoët, Laurent, Algorithmic Differentiation for an efficient CFD solver, ECCOMAS 2022-8th European Congress on Computational Methods in Applied Sciences and Engineering, (2022)
[40] Mitusch, Sebastian K.; Funke, Simon W.; Dokken, Jørgen S., dolfin-adjoint 2018.1: automated adjoints for FEniCS and Firedrake, J. Open Source Softw., 4, 38, 1292, (2019) · doi:10.21105/joss.01292
[41] Moré, Jorge J.; Thuente, David J., Line search algorithms with guaranteed sufficient decrease, ACM Trans. Math. Softw., 20, 3, 286-307, (1994) · Zbl 0888.65072 · doi:10.1145/192115.192132
[42] Mu, Mu; Duan, W.; Wang, Q.; Zhang, R., An extension of conditional nonlinear optimal perturbation approach and its applications, Nonlinear Process. Geophys., 17, 2, 211-220, (2010) · doi:10.5194/npg-17-211-2010
[43] Müller, Jens-Dominik; Cusdin, P., On the performance of discrete adjoint CFD codes using automatic differentiation, Int. J. Numer. Methods Fluids, 47, 8-9, 939-945, (2005) · Zbl 1134.76431 · doi:10.1002/fld.885
[44] Naumann, Uwe, The art of differentiating computer programs: an introduction to algorithmic differentiation, (2011), Society for Industrial and Applied Mathematics · Zbl 1275.65015 · doi:10.1137/1.9781611972078
[45] Nielsen, Eric J.; Diskin, Boris, Discrete adjoint-based design for unsteady turbulent flows on dynamic overset unstructured grids, AIAA J., 51, 6, 1355-1373, (2013) · doi:10.2514/1.J051859
[46] Pringle, Chris C. T.; Kerswell, Rich R., Using nonlinear transient growth to construct the minimal seed for shear flow turbulence, Phys. Rev. Lett., 105, (2010) · doi:10.1103/PhysRevLett.105.154502
[47] Sandu, Adrian, Computational Science - ICCS 2006, 3994, On the properties of Runge-Kutta discrete adjoints, 550-557, (2006) · Zbl 1157.65421 · doi:10.1007/11758549_76
[48] Sato, Hiroyuki, Riemannian Optimization and Its Applications, (2021), Springer · Zbl 1456.90001 · doi:10.1007/978-3-030-62391-3
[49] Sato, Hiroyuki, Riemannian conjugate gradient methods: General framework and specific algorithms with convergence analyses, SIAM J. Optim., 32, 4, 2690-2717, (2022) · Zbl 1506.65079 · doi:10.1137/21M1464178
[50] Sato, Hiroyuki, Riemannian optimization on unit sphere with \(p\), (2022) · Zbl 1531.90134
[51] Schmid, Peter J.; de Pando, Miguel Fosas; Peake, Nigel, Stability analysis for n-periodic arrays of fluid systems, Phys. Rev. Fluids, 2, 11, (2017) · Zbl 1383.76417 · doi:10.1103/PhysRevFluids.2.113902
[52] Skene, Calum S.; Yeh, Chi-An; Schmid, Peter J.; Taira, Kunihiko, Sparsifying the resolvent forcing mode via gradient-based optimisation, J. Fluid Mech., 944, 24 p. pp., (2022) · Zbl 1500.76021 · doi:10.1017/jfm.2022.519
[53] Terwisscha van Scheltinga, A. D.; Dijkstra, H. A., Conditional nonlinear optimal perturbations of the double-gyre ocean circulation, Nonlinear Process. Geophys., 15, 5, 727-734, (2008) · doi:10.5194/npg-15-727-2008
[54] Thiffeault, Jean-Luc, Using multiscale norms to quantify mixing and transport, Nonlinearity, 25, 2, R1-R44, (2012) · Zbl 1325.37007 · doi:10.1088/0951-7715/25/2/R1
[55] Townsend, James; Koep, Niklas; Weichwald, Sebastian, Pymanopt: A Python Toolbox for Optimization on Manifolds using Automatic Differentiation, J. Mach. Learn. Res., 17, 137, 1-5, (2016) · Zbl 1416.65580
[56] Utke, Jean; Hascoet, Laurent; Heimbach, Patrick; Hill, Chris; Hovland, Paul; Naumann, Uwe, Toward adjoinable MPI, 2009 IEEE International Symposium on Parallel & Distributed Processing, 1-8, (2009)
[57] Vishnampet, Ramanathan; Bodony, Daniel J.; Freund, Jonathan B., A practical discrete-adjoint method for high-fidelity compressible turbulence simulations, J. Comput. Phys., 285, 173-192, (2015) · Zbl 1351.76190 · doi:10.1016/j.jcp.2015.01.009
[58] Willis, Ashley P., Optimization of the magnetic dynamo, Phys. Rev. Lett., 109, (2012) · doi:10.1103/PhysRevLett.109.251101
[59] Wright, Stephen; Nocedal, Jorge, Numerical optimization, 35, (1999), Springer · Zbl 0930.65067
[60] Zahr, Matthew J.; Persson, Per-Olof, An adjoint method for a high-order discretization of deforming domain conservation laws for optimization of flow problems, J. Comput. Phys., 326, 516-543, (2016) · Zbl 1373.35329 · doi:10.1016/j.jcp.2016.09.012
[61] Zhang, Hong; Constantinescu, Emil M.; Smith, Barry F., PETSc TSAdjoint: A Discrete Adjoint ODE Solver for First-Order and Second-Order Sensitivity Analysis, SIAM J. Sci. Comput., 44, 1, C1-C24, (2022) · Zbl 1481.65009 · doi:10.1137/21M140078X
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.