Detecting stochastic governing laws with observation on stationary distributions. (English) Zbl 1509.60112
Summary: Mathematical models for complex systems are often accompanied with uncertainties. The goal of this paper is to extract a stochastic differential equation governing model with observation on stationary probability distributions. We develop a neural network method to learn the drift and diffusion terms of the stochastic differential equation. We introduce a new loss function containing the Hellinger distance between the observation data and the learned stationary probability density function. We discover that the learnt stochastic differential equation provides a fair approximation of the data-driven dynamical system after minimizing this loss function during the training method. The effectiveness of our method is demonstrated in numerical experiments.
MSC:
| 60H10 | Stochastic ordinary differential equations (aspects of stochastic analysis) |
| 37H10 | Generation, random and stochastic difference and differential equations |
| 68T05 | Learning and adaptive systems in artificial intelligence |
| 37H05 | General theory of random and stochastic dynamical systems |
References:
| [1] | Arnold, L., Random Dynamical Systems (2003), Springer: Springer New York, Corrected 2nd printing |
| [2] | Pasquero, C.; Tziperman, E., Statistical parameterization of heterogeneous oceanic convection, J. Phys. Oceanogr., 37, 214-229 (2007) |
| [3] | C. Penland, P. Sura, Sensitivity of an ocean model to“details” of stochastic forcing, in: Proc. ECMWF Workshop on Represenation of Subscale Processes using Stochastic-Dynamic Models, Reading, England, 6-8 June, 2005. |
| [4] | Gao, T.; Duan, J., Quantifying model uncertainty in dynamical systems driven by non-gaussian Lévy stable noise with observations on mean exit time or escape probability, Commun. Nonlinear Sci. Numer. Simul., 39, 1-6 (2016) · Zbl 1510.62126 |
| [5] | Wu, D.; Fu, M.; Duan, J., Discovering mean residence time and escape probability from data of stochastic dynamical systems, Chaos, 29, 9, Article 093122 pp. (2019) · Zbl 1423.37049 |
| [6] | Hung, C. L.; Zhang, X.; Gemelke, N.; Chin, C., Observation of scale invariance and universality in two-dimensional Bose gases, Nature, 470, 7333, 236-239 (2011) |
| [7] | Hairapetian, G.; Stenzel, R., Observation of a stationary, current-free double layer in a plasma, Phys. Rev. Lett., 65, 2, 175 (1990) |
| [8] | Yarmchuk, E. J.; Gordon, M. J.V.; Packard, R. E., Observation of stationary vortex arrays in rotating superfluid helium, Phys. Rev. Lett., 43, 3, 214-217 (1979) |
| [9] | Gefen, O.; Fridman, O.; Ronin, I.; Balaban, N. Q., Direct observation of single stationary-phase bacteria reveals a surprisingly long period of constant protein production activity, Proc. Natl. Acad. Sci., 111, 1, 556-561 (2014) |
| [10] | Arnold, L.; Wishtutz, V., Stationary solutions of linear systems with additive and multiplicative noise, Stochastics, 7, 1-2, 133-155 (1982) · Zbl 0479.60061 |
| [11] | Liberzon, D.; Brockett, R. W., Nonlinear feedback systems perturbed by noise: steady-state probability distributions and optimal control, IEEE Trans. Autom. Control, 45, 6, 1116-1130 (2000) · Zbl 0972.93076 |
| [12] | Gray, A. H., Uniqueness of steady-state solutions to the Fokker-Planck equation, J. Math. Phys., 6, 4, 644-647 (1965) |
| [13] | Khasminskii, R., Stochastic Stability of Differential Equations (2012), Springer · Zbl 1259.60058 |
| [14] | Schmalfuss, B., Lyapunov functions and non-trivial stationary solutions of stochastic differential equations, Dyn. Syst. Int. J., 16, 4, 303-317 (2001) · Zbl 1038.60050 |
| [15] | Gerber, F.; Nychka, D. W., Fast covariance parameter estimation of spatial Gaussian process models using neural networks, Statist. Probab., 10, 1 (2021) · Zbl 07851322 |
| [16] | Batz, P.; Ruttor, A.; Opper, M., Variational estimation of the drift for stochastic differential equations from the empirical density, J. Stat. Mech. Theory Exp., 8, Article 083404 pp. (2016) · Zbl 1456.62049 |
| [17] | Batz, P.; Ruttor, A.; Opper, M., Approximate Bayes learning of stochastic differential equations, Phys. Rev. E, 98, 2, Article 022109 pp. (2018) |
| [18] | Opper, M., An estimator for the relative entropy rate of path measures for stochastic differential equations, J. Comput. Phys., 330, 127-133 (2017) · Zbl 1378.60087 |
| [19] | Opper, M., Variational inference for stochastic differential equations, Ann. Physics, 531, 3, Article 1800233 pp. (2019) · Zbl 07759716 |
| [20] | T. Ryder, A. Golightly, A.S. McGough, D. Prangle, Black-box variational inference for stochastic differential equations, in: ICML, 2018, pp. 4423-4432. |
| [21] | Boninsegna, L.; Nüske, F.; Clementi, C., Sparse learning of stochastic dynamical equations, J. Chem. Phys., 148, 24, Article 241723 pp. (2018) |
| [22] | Tabar, R., Analysis and Data-Based Reconstruction of Complex Nonlinear Dynamical Systems (2019), Springer: Springer Berlin/Heidelberg, Germany · Zbl 1422.60005 |
| [23] | X. Li, T.K.L. Wong, R.T. Chen, D. Duvenaud, Scalable gradients for stochastic differential equations, in: AISTATS, 2020, pp. 3870-3882. |
| [24] | J. Jia, A.R. Benson, Neural jump stochastic differential equations, in: NIPS, 2019, p. 32. |
| [25] | Yang, L.; Daskalakis, C.; Karniadakis, G. E., Generative ensemble regression: Learning particle dynamics from observations of ensembles with physics-informed deep generative models, SIAM J. Sci. Comput., 44, 1, B80-B99 (2022) · Zbl 1491.60117 |
| [26] | Zhang, H.; Xu, Y.; Li, Y.; Kurths, J., Statistical solution to SDEs with \(\alpha \)-stable Lévy noise via deep neural network, Int. J. Dyn. Control, 8, 4, 1129-1140 (2020) |
| [27] | Xu, Y.; Zhang, H.; Li, Y.; Zhou, K.; Liu, Q.; Kurths, J., Solving Fokker-Planck equation using deep learning, Chaos, 30, 1, Article 013133 pp. (2020) · Zbl 1431.35210 |
| [28] | Zhang, H.; Xu, Y.; Liu, Q.; Wang, X.; Li, Y., Solving Fokker-Planck equations using deep kd-tree with a small amount of data, Nonlinear Dynam. (2022) |
| [29] | Li, Y.; Duan, J., A data-driven approach for discovering stochastic dynamical systems with non-Gaussian Lévy noise, Physica D, 417, Article 132830 pp. (2021) · Zbl 07493986 |
| [30] | Lu, Y.; Li, Y.; Duan, J., Extracting stochastic governing laws by non-local Kramers-Moyal formulaePhil, Trans. R. Soc. A., 380, Article 20210195 pp. (2022) |
| [31] | Chen, X.; Yang, L.; Duan, J.; Karniadakis, G. E., Solving inverse stochastic problems from discrete particle observations using the Fokker-Planck equation and physics-informed neural networks, SIAM J. Sci. Comput., 43, 3, B811-30 (2021) · Zbl 1480.35377 |
| [32] | Yang, Y.; Nurbekyan, L.; Negrini, E.; Martin, R.; Pasha, M., Optimal transport for parameter identification of chaotic dynamics via invariant measures (2021), arXiv preprint arXiv:2104.15138 |
| [33] | Duan, J., An Introduction to Stochastic Dynamics (2015), Cambridge University Press: Cambridge University Press New York · Zbl 1359.60003 |
| [34] | Dietrich, F.; Makeev, A.; Kevrekidis, G.; Evangelou, N.; Bertalan, T.; Reich, S.; Kevrekidis, I. (2021), arXiv preprint arXiv:2106.09004 |
| [35] | Chen, X.; Duan, J.; Hu, J.; Li, D., Data-driven method to learn the most probable transition pathway and stochastic differential equation, Physica D, 443, Article 133559 pp. (2023) · Zbl 1504.60107 |
| [36] | Cha, S., Comprehensive survey on distance/similarity measures between probability density functions, Int. J. Math. Models Methods Appl. Sci., 1, 4, 300-307 (2007) |
| [37] | Beran, R., Minimum hellinger distance estimates for parametric models, Ann. Statist., 5, 3, 445-463 (1977) · Zbl 0381.62028 |
| [38] | Raissi, M.; Perdikaris, P.; Karniadakis, G. E., Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, J. Comput. Phys., 378, 686-707 (2019) · Zbl 1415.68175 |
| [39] | Chen, X.; Duan, J.; Karniadakis, G. E., Learning and meta-learning of stochastic advection-diffusion-reaction systems from sparse measurements, European J. Appl. Math., 32, 3, 397-420 (2020) · Zbl 1538.68040 |
| [40] | Chen, X.; Wu, F.; Duan, J.; Kurths, J.; Li, X., Most probable dynamics of a genetic regulatory network under stable Lévy noise, Appl. Math. Comput., 348, 425-436 (2019) · Zbl 1428.92036 |
| [41] | Baydin, A. G.; Pearlmutter, B. A.; Radul, A. A.; Siskind, J. M., Automatic differentiation in machine learning: a survey, J. Mach. Learn. Res., 18, 1-43 (2018) · Zbl 06982909 |
| [42] | Klebaner, F. C., Introduction to Stochastic Calculus with Applications (2005), Imperial College Press: Imperial College Press London · Zbl 1077.60001 |
| [43] | Wang, H.; Cheng, X.; Duan, J.; Kurths, J.; Li, X., Likelihood for transcriptions in a genetic regulatory system under asymmetric stable Lévy noise, Chaos, 28, Article 013121 pp. (2018) · Zbl 1390.92053 |
| [44] | Cheng, X.; Wang, H.; Wang, X.; Duan, J.; Li, X., Most probable transition pathways and maximal likely trajectories in a genetic regulatory system, Physica A, 531, Article 121779 pp. (2019) · Zbl 07569470 |
| [45] | Shukla, K.; Ameya, D.; Karniadakis, G., Parallel physics-informed neural networks via domain decomposition, J. Comput. Phys., 447, Article 110683 pp. (2021) · Zbl 07516435 |
| [46] | Yang, X.; Liu, Y.; Mi, C.; Wang, X., Active learning kriging model combining with kernel-density-estimation-based importance sampling method for the estimation of low failure probability, J. Mech. Des., 140, Article 051402 pp. (2018) |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
