-
Dynamical-generative downscaling of climate model ensembles
Authors:
Ignacio Lopez-Gomez,
Zhong Yi Wan,
Leonardo Zepeda-Núñez,
Tapio Schneider,
John Anderson,
Fei Sha
Abstract:
Regional high-resolution climate projections are crucial for many applications, such as agriculture, hydrology, and natural hazard risk assessment. Dynamical downscaling, the state-of-the-art method to produce localized future climate information, involves running a regional climate model (RCM) driven by an Earth System Model (ESM), but it is too computationally expensive to apply to large climate…
▽ More
Regional high-resolution climate projections are crucial for many applications, such as agriculture, hydrology, and natural hazard risk assessment. Dynamical downscaling, the state-of-the-art method to produce localized future climate information, involves running a regional climate model (RCM) driven by an Earth System Model (ESM), but it is too computationally expensive to apply to large climate projection ensembles. We propose a novel approach combining dynamical downscaling with generative artificial intelligence to reduce the cost and improve the uncertainty estimates of downscaled climate projections. In our framework, an RCM dynamically downscales ESM output to an intermediate resolution, followed by a generative diffusion model that further refines the resolution to the target scale. This approach leverages the generalizability of physics-based models and the sampling efficiency of diffusion models, enabling the downscaling of large multi-model ensembles. We evaluate our method against dynamically-downscaled climate projections from the CMIP6 ensemble. Our results demonstrate its ability to provide more accurate uncertainty bounds on future regional climate than alternatives such as dynamical downscaling of smaller ensembles, or traditional empirical statistical downscaling methods. We also show that dynamical-generative downscaling results in significantly lower errors than bias correction and spatial disaggregation (BCSD), and captures more accurately the spectra and multivariate correlations of meteorological fields. These characteristics make the dynamical-generative framework a flexible, accurate, and efficient way to downscale large ensembles of climate projections, currently out of reach for pure dynamical downscaling.
△ Less
Submitted 2 October, 2024;
originally announced October 2024.
-
Generative AI for fast and accurate Statistical Computation of Fluids
Authors:
Roberto Molinaro,
Samuel Lanthaler,
Bogdan Raonić,
Tobias Rohner,
Victor Armegioiu,
Zhong Yi Wan,
Fei Sha,
Siddhartha Mishra,
Leonardo Zepeda-Núñez
Abstract:
We present a generative AI algorithm for addressing the challenging task of fast, accurate and robust statistical computation of three-dimensional turbulent fluid flows. Our algorithm, termed as GenCFD, is based on a conditional score-based diffusion model. Through extensive numerical experimentation with both incompressible and compressible fluid flows, we demonstrate that GenCFD provides very ac…
▽ More
We present a generative AI algorithm for addressing the challenging task of fast, accurate and robust statistical computation of three-dimensional turbulent fluid flows. Our algorithm, termed as GenCFD, is based on a conditional score-based diffusion model. Through extensive numerical experimentation with both incompressible and compressible fluid flows, we demonstrate that GenCFD provides very accurate approximation of statistical quantities of interest such as mean, variance, point pdfs, higher-order moments, while also generating high quality realistic samples of turbulent fluid flows and ensuring excellent spectral resolution. In contrast, ensembles of operator learning baselines which are trained to minimize mean (absolute) square errors regress to the mean flow. We present rigorous theoretical results uncovering the surprising mechanisms through which diffusion models accurately generate fluid flows. These mechanisms are illustrated with solvable toy models that exhibit the relevant features of turbulent fluid flows while being amenable to explicit analytical formulas.
△ Less
Submitted 26 September, 2024;
originally announced September 2024.
-
A probabilistic framework for learning non-intrusive corrections to long-time climate simulations from short-time training data
Authors:
Benedikt Barthel Sorensen,
Leonardo Zepeda-Núñez,
Ignacio Lopez-Gomez,
Zhong Yi Wan,
Rob Carver,
Fei Sha,
Themistoklis Sapsis
Abstract:
Chaotic systems, such as turbulent flows, are ubiquitous in science and engineering. However, their study remains a challenge due to the large range scales, and the strong interaction with other, often not fully understood, physics. As a consequence, the spatiotemporal resolution required for accurate simulation of these systems is typically computationally infeasible, particularly for application…
▽ More
Chaotic systems, such as turbulent flows, are ubiquitous in science and engineering. However, their study remains a challenge due to the large range scales, and the strong interaction with other, often not fully understood, physics. As a consequence, the spatiotemporal resolution required for accurate simulation of these systems is typically computationally infeasible, particularly for applications of long-term risk assessment, such as the quantification of extreme weather risk due to climate change. While data-driven modeling offers some promise of alleviating these obstacles, the scarcity of high-quality simulations results in limited available data to train such models, which is often compounded by the lack of stability for long-horizon simulations. As such, the computational, algorithmic, and data restrictions generally imply that the probability of rare extreme events is not accurately captured. In this work we present a general strategy for training neural network models to non-intrusively correct under-resolved long-time simulations of chaotic systems. The approach is based on training a post-processing correction operator on under-resolved simulations nudged towards a high-fidelity reference. This enables us to learn the dynamics of the underlying system directly, which allows us to use very little training data, even when the statistics thereof are far from converged. Additionally, through the use of probabilistic network architectures we are able to leverage the uncertainty due to the limited training data to further improve extrapolation capabilities. We apply our framework to severely under-resolved simulations of quasi-geostrophic flow and demonstrate its ability to accurately predict the anisotropic statistics over time horizons more than 30 times longer than the data seen in training.
△ Less
Submitted 2 August, 2024;
originally announced August 2024.
-
DySLIM: Dynamics Stable Learning by Invariant Measure for Chaotic Systems
Authors:
Yair Schiff,
Zhong Yi Wan,
Jeffrey B. Parker,
Stephan Hoyer,
Volodymyr Kuleshov,
Fei Sha,
Leonardo Zepeda-Núñez
Abstract:
Learning dynamics from dissipative chaotic systems is notoriously difficult due to their inherent instability, as formalized by their positive Lyapunov exponents, which exponentially amplify errors in the learned dynamics. However, many of these systems exhibit ergodicity and an attractor: a compact and highly complex manifold, to which trajectories converge in finite-time, that supports an invari…
▽ More
Learning dynamics from dissipative chaotic systems is notoriously difficult due to their inherent instability, as formalized by their positive Lyapunov exponents, which exponentially amplify errors in the learned dynamics. However, many of these systems exhibit ergodicity and an attractor: a compact and highly complex manifold, to which trajectories converge in finite-time, that supports an invariant measure, i.e., a probability distribution that is invariant under the action of the dynamics, which dictates the long-term statistical behavior of the system. In this work, we leverage this structure to propose a new framework that targets learning the invariant measure as well as the dynamics, in contrast with typical methods that only target the misfit between trajectories, which often leads to divergence as the trajectories' length increases. We use our framework to propose a tractable and sample efficient objective that can be used with any existing learning objectives. Our Dynamics Stable Learning by Invariant Measure (DySLIM) objective enables model training that achieves better point-wise tracking and long-term statistical accuracy relative to other learning objectives. By targeting the distribution with a scalable regularization term, we hope that this approach can be extended to more complex systems exhibiting slowly-variant distributions, such as weather and climate models.
△ Less
Submitted 5 June, 2024; v1 submitted 6 February, 2024;
originally announced February 2024.
-
Neural Ideal Large Eddy Simulation: Modeling Turbulence with Neural Stochastic Differential Equations
Authors:
Anudhyan Boral,
Zhong Yi Wan,
Leonardo Zepeda-Núñez,
James Lottes,
Qing Wang,
Yi-fan Chen,
John Roberts Anderson,
Fei Sha
Abstract:
We introduce a data-driven learning framework that assimilates two powerful ideas: ideal large eddy simulation (LES) from turbulence closure modeling and neural stochastic differential equations (SDE) for stochastic modeling. The ideal LES models the LES flow by treating each full-order trajectory as a random realization of the underlying dynamics, as such, the effect of small-scales is marginaliz…
▽ More
We introduce a data-driven learning framework that assimilates two powerful ideas: ideal large eddy simulation (LES) from turbulence closure modeling and neural stochastic differential equations (SDE) for stochastic modeling. The ideal LES models the LES flow by treating each full-order trajectory as a random realization of the underlying dynamics, as such, the effect of small-scales is marginalized to obtain the deterministic evolution of the LES state. However, ideal LES is analytically intractable. In our work, we use a latent neural SDE to model the evolution of the stochastic process and an encoder-decoder pair for transforming between the latent space and the desired ideal flow field. This stands in sharp contrast to other types of neural parameterization of closure models where each trajectory is treated as a deterministic realization of the dynamics. We show the effectiveness of our approach (niLES - neural ideal LES) on a challenging chaotic dynamical system: Kolmogorov flow at a Reynolds number of 20,000. Compared to competing methods, our method can handle non-uniform geometries using unstructured meshes seamlessly. In particular, niLES leads to trajectories with more accurate statistics and enhances stability, particularly for long-horizon rollouts.
△ Less
Submitted 1 June, 2023;
originally announced June 2023.
-
Debias Coarsely, Sample Conditionally: Statistical Downscaling through Optimal Transport and Probabilistic Diffusion Models
Authors:
Zhong Yi Wan,
Ricardo Baptista,
Yi-fan Chen,
John Anderson,
Anudhyan Boral,
Fei Sha,
Leonardo Zepeda-Núñez
Abstract:
We introduce a two-stage probabilistic framework for statistical downscaling using unpaired data. Statistical downscaling seeks a probabilistic map to transform low-resolution data from a biased coarse-grained numerical scheme to high-resolution data that is consistent with a high-fidelity scheme. Our framework tackles the problem by composing two transformations: (i) a debiasing step via an optim…
▽ More
We introduce a two-stage probabilistic framework for statistical downscaling using unpaired data. Statistical downscaling seeks a probabilistic map to transform low-resolution data from a biased coarse-grained numerical scheme to high-resolution data that is consistent with a high-fidelity scheme. Our framework tackles the problem by composing two transformations: (i) a debiasing step via an optimal transport map, and (ii) an upsampling step achieved by a probabilistic diffusion model with a posteriori conditional sampling. This approach characterizes a conditional distribution without needing paired data, and faithfully recovers relevant physical statistics from biased samples. We demonstrate the utility of the proposed approach on one- and two-dimensional fluid flow problems, which are representative of the core difficulties present in numerical simulations of weather and climate. Our method produces realistic high-resolution outputs from low-resolution inputs, by upsampling resolutions of 8x and 16x. Moreover, our procedure correctly matches the statistics of physical quantities, even when the low-frequency content of the inputs and outputs do not match, a crucial but difficult-to-satisfy assumption needed by current state-of-the-art alternatives. Code for this work is available at: https://github.com/google-research/swirl-dynamics/tree/main/swirl_dynamics/projects/probabilistic_diffusion.
△ Less
Submitted 30 October, 2023; v1 submitted 24 May, 2023;
originally announced May 2023.
-
Evolve Smoothly, Fit Consistently: Learning Smooth Latent Dynamics For Advection-Dominated Systems
Authors:
Zhong Yi Wan,
Leonardo Zepeda-Núñez,
Anudhyan Boral,
Fei Sha
Abstract:
We present a data-driven, space-time continuous framework to learn surrogate models for complex physical systems described by advection-dominated partial differential equations. Those systems have slow-decaying Kolmogorov n-width that hinders standard methods, including reduced order modeling, from producing high-fidelity simulations at low cost. In this work, we construct hypernetwork-based laten…
▽ More
We present a data-driven, space-time continuous framework to learn surrogate models for complex physical systems described by advection-dominated partial differential equations. Those systems have slow-decaying Kolmogorov n-width that hinders standard methods, including reduced order modeling, from producing high-fidelity simulations at low cost. In this work, we construct hypernetwork-based latent dynamical models directly on the parameter space of a compact representation network. We leverage the expressive power of the network and a specially designed consistency-inducing regularization to obtain latent trajectories that are both low-dimensional and smooth. These properties render our surrogate models highly efficient at inference time. We show the efficacy of our framework by learning models that generate accurate multi-step rollout predictions at much faster inference speed compared to competitors, for several challenging examples.
△ Less
Submitted 6 February, 2023; v1 submitted 24 January, 2023;
originally announced January 2023.
-
Bubbles in Turbulent Flows: Data-driven, kinematic models with memory terms
Authors:
Zhong Yi Wan,
Petr Karnakov,
Petros Koumoutsakos,
Themistoklis P. Sapsis
Abstract:
We present data driven kinematic models for the motion of bubbles in high-Re turbulent fluid flows based on recurrent neural networks with long-short term memory enhancements. The models extend empirical relations, such as Maxey-Riley (MR) and its variants, whose applicability is limited when either the bubble size is large or the flow is very complex. The recurrent neural networks are trained on…
▽ More
We present data driven kinematic models for the motion of bubbles in high-Re turbulent fluid flows based on recurrent neural networks with long-short term memory enhancements. The models extend empirical relations, such as Maxey-Riley (MR) and its variants, whose applicability is limited when either the bubble size is large or the flow is very complex. The recurrent neural networks are trained on the trajectories of bubbles obtained by Direct Numerical Simulations (DNS) of the Navier Stokes equations for a two-component incompressible flow model. Long short term memory components exploit the time history of the flow field that the bubbles have encountered along their trajectories and the networks are further augmented by imposing rotational invariance to their structure. We first train and validate the formulated model using DNS data for a turbulent Taylor-Green vortex. Then we examine the model predictive capabilities and its generalization to Reynolds numbers that are different from those of the training data on benchmark problems, including a steady (Hill's spherical vortex) and an unsteady (Gaussian vortex ring) flow field. We find that the predictions of the developed model are significantly improved compared with those obtained by the MR equation. Our results indicate that data-driven models with history terms are well suited in capturing the trajectories of bubbles in turbulent flows.
△ Less
Submitted 4 October, 2019;
originally announced October 2019.
-
Data-assisted reduced-order modeling of extreme events in complex dynamical systems
Authors:
Zhong Yi Wan,
Pantelis R. Vlachas,
Petros Koumoutsakos,
Themistoklis P. Sapsis
Abstract:
Dynamical systems with high intrinsic dimensionality are often characterized by extreme events having the form of rare transitions several standard deviations away from the mean. For such systems, order-reduction methods through projection of the governing equations have limited applicability due to the large intrinsic dimensionality of the underlying attractor but also the complexity of the trans…
▽ More
Dynamical systems with high intrinsic dimensionality are often characterized by extreme events having the form of rare transitions several standard deviations away from the mean. For such systems, order-reduction methods through projection of the governing equations have limited applicability due to the large intrinsic dimensionality of the underlying attractor but also the complexity of the transient events. An alternative approach is data-driven techniques that aim to quantify the dynamics of specific modes utilizing data-streams. Several of these approaches have improved performance by expanding the state representation using delayed coordinates. However, such strategies are limited in regions of the phase space where there is a small amount of data available, as is the case for extreme events. In this work, we develop a blended framework that integrates an imperfect model, obtained from projecting equations into a subspace that still contains crucial dynamical information, with data-streams through a recurrent neural network (RNN) architecture. In particular, we employ the long-short-term memory (LSTM), to model portions of the dynamics which cannot be accounted by the equations. The RNN is trained by analyzing the mismatch between the imperfect model and the data-streams, projected in the reduced-order space. In this way, the data-driven model improves the imperfect model in regions where data is available, while for locations where data is sparse the imperfect model still provides a baseline for the prediction of the system dynamics. We assess the developed framework on two challenging prototype systems exhibiting extreme events and show that the blended approach has improved performance compared with methods that use either data streams or the imperfect model alone. The improvement is more significant in regions associated with extreme events, where data is sparse.
△ Less
Submitted 30 April, 2018; v1 submitted 8 March, 2018;
originally announced March 2018.
-
Data-Driven Forecasting of High-Dimensional Chaotic Systems with Long Short-Term Memory Networks
Authors:
Pantelis R. Vlachas,
Wonmin Byeon,
Zhong Y. Wan,
Themistoklis P. Sapsis,
Petros Koumoutsakos
Abstract:
We introduce a data-driven forecasting method for high-dimensional chaotic systems using long short-term memory (LSTM) recurrent neural networks. The proposed LSTM neural networks perform inference of high-dimensional dynamical systems in their reduced order space and are shown to be an effective set of nonlinear approximators of their attractor. We demonstrate the forecasting performance of the L…
▽ More
We introduce a data-driven forecasting method for high-dimensional chaotic systems using long short-term memory (LSTM) recurrent neural networks. The proposed LSTM neural networks perform inference of high-dimensional dynamical systems in their reduced order space and are shown to be an effective set of nonlinear approximators of their attractor. We demonstrate the forecasting performance of the LSTM and compare it with Gaussian processes (GPs) in time series obtained from the Lorenz 96 system, the Kuramoto-Sivashinsky equation and a prototype climate model. The LSTM networks outperform the GPs in short-term forecasting accuracy in all applications considered. A hybrid architecture, extending the LSTM with a mean stochastic model (MSM-LSTM), is proposed to ensure convergence to the invariant measure. This novel hybrid method is fully data-driven and extends the forecasting capabilities of LSTM networks.
△ Less
Submitted 19 September, 2019; v1 submitted 21 February, 2018;
originally announced February 2018.
-
Reduced-space Gaussian Process Regression for Data-Driven Probabilistic Forecast of Chaotic Dynamical Systems
Authors:
Zhong Yi Wan,
Themistoklis P. Sapsis
Abstract:
We formulate a reduced-order strategy for efficiently forecasting complex high-dimensional dynamical systems entirely based on data streams. The first step of our method involves reconstructing the dynamics in a reduced-order subspace of choice using Gaussian Process Regression (GPR). GPR simultaneously allows for reconstruction of the vector field and more importantly, estimation of local uncerta…
▽ More
We formulate a reduced-order strategy for efficiently forecasting complex high-dimensional dynamical systems entirely based on data streams. The first step of our method involves reconstructing the dynamics in a reduced-order subspace of choice using Gaussian Process Regression (GPR). GPR simultaneously allows for reconstruction of the vector field and more importantly, estimation of local uncertainty. The latter is due to i) local interpolation error and ii) truncation of the high-dimensional phase space. This uncertainty component can be analytically quantified in terms of the GPR hyperparameters. In the second step we formulate stochastic models that explicitly take into account the reconstructed dynamics and their uncertainty. For regions of the attractor which are not sufficiently sampled for our GPR framework to be effective, an adaptive blended scheme is formulated to enforce correct statistical steady state properties, matching those of the real data. We examine the effectiveness of the proposed method to complex systems including the Lorenz 96, the Kuramoto-Sivashinsky, as well as a prototype climate model. We also study the performance of the proposed approach as the intrinsic dimensionality of the system attractor increases in highly turbulent regimes.
△ Less
Submitted 4 November, 2016;
originally announced November 2016.
-
Extend Special Relativity to the Superluminal Case
Authors:
Z. C. Tu,
Z. Y. Wan
Abstract:
First, we extend the special relativity into the superluminal case and put forward a superluminal theory of kinematics, in which we show that the temporal coordinate need exchanging with one of the spatial coordinates in a superluminal inertial frame, and that the coordinate transformations from any superluminal inertial frame to the rest frame (here rest just says in a relative sense) are the s…
▽ More
First, we extend the special relativity into the superluminal case and put forward a superluminal theory of kinematics, in which we show that the temporal coordinate need exchanging with one of the spatial coordinates in a superluminal inertial frame, and that the coordinate transformations from any superluminal inertial frame to the rest frame (here rest just says in a relative sense) are the same as the Lorentz transformations from some normal inertial frame to the rest frame. Consequently, the causality can not be violated. Secondly, we investigate the superluminal theory of dynamics and find that the total energy of any object moving at a speed of $v$ (faster than the speed of light in vacuum $c$) is equal to the total energy of that object moving at a speed of $u (u<c)$ provided that the product of two speeds satisfy $uv=c^{2}$. Lastly, we conjecture that this superluminal theory can give a novel interpretation to the essence of matter waves put forward by de Broglie.
△ Less
Submitted 22 July, 2009; v1 submitted 16 July, 2001;
originally announced July 2001.
