Abstract: Recently, some works have suggested methods to combine variational probabilistic inference with Monte Carlo sampling. One promising approach is via local optimal transport. In this approach, a gradient steepest descent method based on local optimal transport principles is formulated to transform deterministically point samples from an intermediate density to a posterior density. The local mappings that transform the intermediate densities are embedded in a reproducing kernel Hilbert space (RKHS). This variational mapping method requires the evaluation of the log-posterior density gradient and therefore the adjoint of the observational operator. In this work, we evaluate nonlinear observational mappings in the variational mapping method using two approximations that avoid the adjoint, an ensemble based approximation in which the gradient is approximated by the particle covariances in the state and observational spaces the so-called ensemble space and an RKHS approximation in which the observational mapping is embedded in an RKHS and the gradient is derived there. The approximations are evaluated for highly nonlinear observational operators and in a low-dimensional chaotic dynamical system. The RKHS approximation is shown to be highly successful and superior to the ensemble approximation.

Abstract: Ensemble optimal interpolation (EnOI) have been introduced to drastically reduce the computational cost of the ensemble Kalman filter (EnKF). The idea is to use a static (pre-selected) ensemble to parameterize the background covariance matrix, which avoids the costly integration step of the ensemble members with the dynamical model. To better represent the strong variability of the Red Sea circulation, we propose new adaptive EnOI schemes in which the ensemble members are adaptively selected at every assimilation cycle from a large dictionary of ocean states describing the variability of the Red Sea system. Those members would account for the strong eddy and seasonal variability of the Red Sea circulation and enforce climatological smoothness in the filter update. We implement and test different schemes to adaptively choose the ensemble members based on (i) the similarity to the forecast, or (ii) an Orthogonal Matching Pursuit (OMP) algorithm. Results of numerical experiments assimilating remote sensing data into a high-resolution MIT general circulation model (MITgcm) of the Red Sea will be presented to demonstrate the efficiency of the proposed approach.

Abstract: This paper demonstrates the use of machine learning techniques to study the uncertainty in numerical weather prediction models due to the interaction of multiple physical processes. We aim to address the following problems: 1)estimation of systematic model errors in output quantities of interest at future times and 2)identification of specific physical processes that contribute most to the forecast uncertainty in the quantity of interest under specified meteorological conditions. To address these problems, we employ simple machine learning algorithms and perform numerical experiments with Weather Research and Forecasting (WRF) model. The results demonstrate the potential of machine learning approaches to aid the study of model errors.

Abstract: A Weak Constraint Gaussian Process (WCGP) model is presented to integrate noisy inputs into the classical Gaussian Process predictive distribution. This follows a Data Assimilation approach i.e. by considering information provided by observed values of a noisy input in a time window. Due to the increased number of states processed from real applications and the time complexity of GP algorithms, the problem mandates a solution in a high performance computing environment. In this paper, parallelism is explored by defining the parallel WCGP model based on domain decomposition. Both a mathematical formulation of the model and a parallel algorithm are provided. We prove that the parallel implementation preserves the accuracy of the sequential one. The algorithm’s scalability is further proved to be O(p^2) where p is the number of processors.

Abstract: When the heat released by a flame is sufficiently in phase with the acoustic pressure, a self-excited thermoacoustic oscillation can arise. These nonlinear oscillations are one of the biggest challenges faced in the design of safe and reliable gas-turbines and rocket motors. In the worst-case scenario, uncontrolled thermoacoustic oscillations can shake an engine apart. Reduced-order thermoacoustic models, which are nonlinear and time-delayed, can only qualitatively predict thermoacoustic oscillations. To make reduced-order models quantitatively predictive, we develop a data assimilation framework for state estimation. We numerically estimate the most likely nonlinear state of a Galerkin-discretized time delayed model of a prototypical combustor. Data assimilation is an optimal blending of observations with previous system’s state estimates (background) to produce optimal initial conditions. A cost functional is defined to measure (i) the statistical distance between the model output and the measurements from experiments; and (ii) the distance between the model’s initial conditions and the background knowledge. Its minimum corresponds to the optimal state, which is computed by Lagrangian optimization with the aid of adjoint equations. We study the influence of the number of Galerkin modes, which are the natural acoustic modes of the duct, with which the model is discretized. We show that decomposing the measured pressure signal in a finite number of modes is an effective way to enhance the state estimation, especially when highly nonlinear modal interactions occur in the assimilation window. This work represents the first application of data assimilation to nonlinear thermoacoustics, which opens new possibilities for real time calibration of reduced-order models with experimental measurements.

Abstract: We discuss surrogate models based on machine learning as approximation to the solution of an ordinary differential equation. Neural networks and a multivariate linear regressor are assessed for this application. Both of them show a satisfactory performance for the considered case study of a damped perturbed harmonic oscillator. The interface of the surrogate model is designed to work similar to a solver of an ordinary differential equation, respectively a simulation unit. Computational demand and accuracy in terms of local and global error are discussed. Parameter studies are performed to discuss the sensitivity of the method and to tune the performance.

Abstract: Methods from learning theory are used in the state space of linear dynamical and control systems in order to estimate the system matrices and some relevant quantities such as a the topological entropy. The approach is illustrated via a series of numerical examples.

Abstract: Recent progress in machine learning has shown how to forecast and, to some extent, learn the dynamics of a model from its output, resorting in particular to neural networks and deep learning techniques. We will show how the same goal can be directly achieved using data assimilation techniques without leveraging on machine learning software libraries, with a view to high-dimensional models. The dynamics of a model are learned from its observation and an ordinary differential equation (ODE) representation of this model is inferred using a recursive nonlinear regression. Because the method is embedded in a Bayesian data assimilation framework, it can learn from partial and noisy observations of a state trajectory of the physical model. Moreover, a space-wise local representation of the ODE system is introduced and is key to deal with high-dimensional models. The method is illustrated on several chaotic discrete and continuous models of various dimensions, with or without noisy observations, with the goal to identify or improve the model dynamics, build a surrogate or reduced model, or produce forecasts from mere observations of the physical model. It has recently been suggested that neural network architectures could be interpreted as dynamical systems. Reciprocally, we show that our ODE representations are reminiscent of deep learning architectures. Furthermore, numerical analysis considerations on stability shed light on the assets and limitations of the method.

Abstract: In this work, we propose a physics-informed Echo State Networks (ESN) to predict the evolution of chaotic systems. Compared to conventional echo state networks, the physics-informed ESN are trained to solve supervised learning tasks while ensuring that their predictions do not violate the given physical laws. This is done by introducing an additional loss during the training of the ESN, which penalizes non-physical predictions by the ESN. The potential of this approach is demonstrated on the Lorenz system where the obtained predictability horizon of the physics-informed ESN was improved by up to nearly 2 Lyapunov times compared to conventional ESN without the need of additional training data. These results illustrate the potential of using machine learning tools combined with prior physical knowledge to improve the time-accurate prediction of chaotic dynamical systems.

Abstract: Recent use of satellite synthetic aperture radar (SAR) images in flood forecasting has allowed assimilation of spatially dense observations over large rural areas into flood forecasting models. This rich source of observational information has offered a valuable improvement in flood forecasting accuracy as the instruments are able to image day and night, and can see through clouds. However, in urban areas, the use of SAR data is limited due to building shadows and layover effects. Hence, in urban areas it is even more important to use observational data to constrain hydrodynamic flood models, due to the complexity of the landscape and interactions with buildings, sewers, rivers etc. To increase the amount of observation data available in urban areas, and to make use of abundance of technology in cities, our research is concentrating on using novel and easily available data from cities such as CCTV camera images. We have carried out an initial investigation into the impact of assimilating such data on flood forecasts. Our experiments used water level observations extracted from river camera images from four Farson Digital Ltd cameras, for a flood event near Tewkesbury, UK in 2012. We show that these data can improve flood forecast accuracy, especially as they capture the rising limb of the flood when satellite data is usually unavailable. However, in our initial experiments we used manual water level extraction and quality control for the observations, due to complications with the camera settings, image processing, and various digital terrain map resolutions and accuracies. Our next aim is to use machine learning to automatically extract water levels from CCTV images, with associated observation uncertainty. Machine learning will allow us to obtain and use real time water observations from images on a large scale, especially in complex systems such as cities, and we will discuss the potential of this approach.

Abstract: Ensemble Kalman filter (EnKF) has proven successful in assimilating observations of large-scale dynamical systems, such as the atmosphere, into computer simulations for better predictability. Due to the fact that a limited-size ensemble of model states is used, sampling errors accumulate, and manifest themselves as long-range spurious correlations, leading to filter divergence. This effect is alleviated in practice by applying covariance localization. This work investigates the possibility of using machine learning algorithms to automatically tune the parameters of the covariance localization step of ensemble filters. Numerical experiments carried out with the Lorenz96 model reveal the potential of the proposed machine learning approaches.