Predicting shallow water dynamics using echo-state networks with transfer learning. (English) Zbl 07638958
Summary: In this paper we demonstrate that reservoir computing can be used to learn the dynamics of the shallow-water equations. In particular, while most previous applications of reservoir computing have required training on a particular trajectory to further predict the evolution along that trajectory alone, we show the capability of reservoir computing to predict trajectories of the shallow-water equations with initial conditions not seen in the training process. However, in this setting, we find that the performance of the network deteriorates for initial conditions with ambient conditions (such as total water height and average velocity) that are different from those in the training dataset. To circumvent this deficiency, we introduce a transfer learning approach wherein a small additional training step with the relevant ambient conditions is used to improve the predictions.
MSC:
| 76-10 | Mathematical modeling or simulation for problems pertaining to fluid mechanics |
| 68T07 | Artificial neural networks and deep learning |
Software:
PDE-NetReferences:
| [1] | Arcomano, T.; Szunyogh, I.; Pathak, J.; Wikner, A.; Hunt, BR; Ott, E., A machine learning-based global atmospheric forecast model, Geophys. Res. Lett., 47, 9, 2020-087776 (2020) · doi:10.1029/2020GL087776 |
| [2] | Audusse, E.; Chalons, C.; Ung, P., A simple well-balanced and positive numerical scheme for the shallow-water system, Commun. Math. Sci., 13, 5, 1317-1332 (2015) · Zbl 1327.76026 · doi:10.4310/CMS.2015.v13.n5.a11 |
| [3] | Berg, J.; Nyström, K., Data-driven discovery of PDEs in complex datasets, J. Comput. Phys., 384, 239-252 (2019) · Zbl 1451.68239 · doi:10.1016/j.jcp.2019.01.036 |
| [4] | Bollt, E., On explaining the surprising success of reservoir computing forecaster of chaos? the universal machine learning dynamical system with contrast to VAR and DMD, Chaos Interdiscip. J. Nonlinear Sci., 31, 1, 013108 (2021) · Zbl 1512.68265 · doi:10.1063/5.0024890 |
| [5] | Buehner, M.; Young, P., A tighter bound for the echo state property, IEEE Trans. Neural Netw., 17, 3, 820-824 (2006) · doi:10.1109/TNN.2006.872357 |
| [6] | Caluwaerts, K., Wyffels, F., Dieleman, S., Schrauwen, B.: The spectral radius remains a valid indicator of the echo state property for large reservoirs. In: The 2013 International Joint Conference on Neural Networks (IJCNN), pp. 1-6 (2013). doi:10.1109/IJCNN.2013.6706899 |
| [7] | Champion, K.; Lusch, B.; Kutz, JN; Brunton, SL, Data-driven discovery of coordinates and governing equations, Proc. Natl. Acad. Sci., 116, 45, 22445-22451 (2019) · Zbl 1433.68396 · doi:10.1073/pnas.1906995116 |
| [8] | Chattopadhyay, A.; Hassanzadeh, P.; Subramanian, D., Data-driven predictions of a multiscale Lorenz 96 chaotic system using machine-learning methods: reservoir computing, artificial neural network, and long short-term memory network, Nonlinear Process. Geophys., 27, 3, 373-389 (2020) · doi:10.5194/npg-27-373-2020 |
| [9] | Chen, X.: Effective data-driven models for chaotic and turbulent dynamics. Ph.D. thesis, University of Houston (2021) |
| [10] | DeGennaro, AM; Urban, NM; Nadiga, BT; Haut, T., Model structural inference using local dynamic operators, Int. J. Uncertain. Quantif., 9, 1, 59-83 (2019) · Zbl 1498.62085 · doi:10.1615/Int.J.UncertaintyQuantification.2019025828 |
| [11] | Gandhi, M.; Jaeger, H., Echo state property linked to an input: exploring a fundamental characteristic of recurrent neural networks, Neural Comput., 25, 671-696 (2013) · Zbl 1269.92006 · doi:10.1162/NECO_a_00411 |
| [12] | Golub, GH; Hansen, PC; O’Leary, DP, Tikhonov regularization and total least squares, SIAM J. Matrix Anal. Appl., 21, 1, 185-194 (1999) · Zbl 0945.65042 · doi:10.1137/S0895479897326432 |
| [13] | Goutal, N.: Proceedings of the 2nd Workshop on Dam-break Wave Simulation. Department Laboratoire National d’Hydraulique, Groupe Hydraulique Fluviale (1997) |
| [14] | Inubushi, M., Goto, S.: Transferring reservoir computing: formulation and application to fluid physics. In: International Conference on Artificial Neural Networks, pp. 193-199. Springer (2019) |
| [15] | Inubushi, M.; Goto, S., Transfer learning for nonlinear dynamics and its application to fluid turbulence, Phys. Rev. E, 102, 4 (2020) · doi:10.1103/PhysRevE.102.043301 |
| [16] | Jaeger, H.: The “echo state” approach to analysing and training recurrent neural networks-with an erratum note. German National Research Center for Information Technology GMD Technical Report, Bonn, Germany vol. 148, no. 34, p. 13 (2001) |
| [17] | Jaeger, H., Echo state network, Scholarpedia, 2, 9, 2330 (2007) · doi:10.4249/scholarpedia.2330 |
| [18] | Jaeger, H.; Haas, H., Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication, Science, 304, 5667, 78-80 (2004) · doi:10.1126/science.1091277 |
| [19] | Jaeger, H.; Lukoševičius, M.; Popovici, D.; Siewert, U., Optimization and applications of echo state networks with leaky-integrator neurons, Neural Netw., 20, 3, 335-352 (2007) · Zbl 1132.68554 · doi:10.1016/j.neunet.2007.04.016 |
| [20] | Li, J., Sun, G., Zhao, G., Li-wei, H.L.: Robust low-rank discovery of data-driven partial differential equations. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 767-774 (2020) |
| [21] | Long, Z., Lu, Y., Ma, X., Dong, B.: PDE-Net: Learning PDEs from data. In: International Conference on Machine Learning, pp. 3208-3216. PMLR (2018) |
| [22] | Maass, W.; Natschläger, T.; Markram, H., Real-time computing without stable states: A new framework for neural computation based on perturbations, Neural Comput., 14, 11, 2531-2560 (2002) · Zbl 1057.68618 · doi:10.1162/089976602760407955 |
| [23] | Manjunath, G.; Jaeger, H., Echo state property linked to an input: exploring a fundamental characteristic of recurrent neural networks, Neural Comput., 25, 3, 671-696 (2013) · Zbl 1269.92006 · doi:10.1162/NECO_a_00411 |
| [24] | Maslyaev, M., Hvatov, A., Kalyuzhnaya, A.: Data-driven partial differential equations discovery approach for the noised multi-dimensional data. In: International Conference on Computational Science, pp. 86-100. Springer (2020) |
| [25] | Nadiga, B., Reservoir computing as a tool for climate predictability studies, J. Adv. Model. Earth Syst., 13, 2020-002290 (2021) · doi:10.1029/2020MS002290 |
| [26] | Pandey, S.; Schumacher, J., Reservoir computing model of two-dimensional turbulent convection, Phys. Rev. Fluids, 5 (2020) · doi:10.1103/PhysRevFluids.5.113506 |
| [27] | Pathak, J.; Hunt, B.; Girvan, M.; Lu, Z.; Ott, E., Model-free prediction of large spatiotemporally chaotic systems from data: a reservoir computing approach, Phys. Rev. Lett., 120, 2 (2018) · doi:10.1103/PhysRevLett.120.024102 |
| [28] | Pyle, R.; Jovanovic, N.; Subramanian, D.; Palem, KV; Patel, AB, Domain-driven models yield better predictions at lower cost than reservoir computers in Lorenz systems, Phil. Trans. R. Soc. A, 379, 2194, 20200246 (2021) · doi:10.1098/rsta.2020.0246 |
| [29] | Reinbold, PA; Grigoriev, RO, Data-driven discovery of partial differential equation models with latent variables, Phys. Rev. E, 100, 2 (2019) · doi:10.1103/PhysRevE.100.022219 |
| [30] | Rudy, SH; Brunton, SL; Proctor, JL; Kutz, JN, Data-driven discovery of partial differential equations, Sci. Adv., 3, 4, 1602614 (2017) · doi:10.1126/sciadv.1602614 |
| [31] | Vlachas, PR; Pathak, J.; Hunt, BR; Sapsis, TP; Girvan, M.; Ott, E.; Koumoutsakos, P., Backpropagation algorithms and reservoir computing in recurrent neural networks for the forecasting of complex spatiotemporal dynamics, Neural Netw., 126, 191-217 (2020) · doi:10.1016/j.neunet.2020.02.016 |
| [32] | Xu, H., Chang, H., Zhang, D.: DL-PDE: deep-learning based data-driven discovery of partial differential equations from discrete and noisy data. arXiv preprint arXiv:1908.04463 (2019) · Zbl 1528.65093 |
| [33] | Yildiz, IB; Jaeger, H.; Kiebel, SJ, Re-visiting the echo state property, Neural Netw., 35, 1-9 (2012) · Zbl 1258.68129 · doi:10.1016/j.neunet.2012.07.005 |
| [34] | Zhang, Z., Liu, Y.: Robust data-driven discovery of partial differential equations under uncertainties. arXiv preprint arXiv:2102.06504 (2021) |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
