Complex dynamical systems, from climate to neuroscience, are often governed by equations that are either too expensive to solve or simply unknown. Deep neural networks have learned to predict such systems from data, but their black-box nature hampers data assimilation, a key technique that weaves observations into the model and estimates hidden variables. Ensemble assimilation methods require thousands of model runs and suffer from sampling errors. This work sits at the intersection of scientific machine learning and Koopman theory: neighboring approaches either fully linearize the dynamics, losing nonlinearity, or remain black boxes with inefficient assimilation.
CGKN builds a neural stochastic differential equation in three stages. First, an encoder maps unobserved variables into a latent space where the dynamics become conditionally linear: given the observed variables, the latent variables follow a linear equation with Gaussian noise. Then, neural networks approximate the four functions of this conditional Gaussian equation. Finally, a decoder returns the latent variables to the original space. The key idea is a generalization of Koopman theory: instead of full linearization, nonlinearity in the observed variables is preserved, but conditional linearity is introduced. This yields closed-form analytical formulas for the mean and covariance of the latent state, which are integrated with a standard ODE solver. Data assimilation becomes analytical and is incorporated into the loss function during training, eliminating the need for empirical ensemble tuning.
On all three test systems, CGKN achieved forecast accuracy comparable to the true model and the best black boxes, while performing data assimilation analytically. For the Burgers–Sivashinsky equation, the forecast error was 0.284 versus 0.246 for the true model, and the assimilation error was 0.728 versus 0.695. In the Lorenz-96 system with 40 variables, CGKN reached a forecast error of 0.064 and an assimilation error of 0.095, whereas the standard Koopman network gave 0.306 and 0.273, respectively. For El Niño, the model predicted sea surface temperature with a correlation above 0.5 at a 12-month horizon, outperforming the persistence forecast. In all cases, CGKN surpassed the standard Koopman network and the conditional Gaussian regression, and in data assimilation it also beat the DNN black box, for which analytical assimilation is impossible.
This work differs from neighboring approaches in that it does not sacrifice nonlinearity for analytical assimilation. The standard Koopman network fully linearizes the latent dynamics and loses the contribution of observed variables, while CG-Reg regression is limited to a predefined library of functions. CGKN trains nonlinear mappings with neural networks and preserves conditional linearity, offering both expressiveness and efficiency. This is the first demonstration that a conditional Gaussian structure can be embedded in deep learning and used for analytical data assimilation in strongly nonlinear turbulent systems.
The topic could evolve toward rigorous quantitative uncertainty quantification for the original unobserved variables. Currently, uncertainty is estimated via post-processing through residual analysis, but the nonlinear decoder complicates accurate mapping of covariance from the latent space. Multi-objective optimization methods that balance transformation quality, forecasting, and data assimilation are possible, as well as extensions to control problems for partially observed systems and inverse problems.
The method is applicable in climatology, geophysics, materials science, and neuroscience, where fast forecasts and estimation of hidden states from incomplete data are needed.
The next step will be to develop efficient uncertainty quantification methods for the original variables, for example via the Jacobian of the decoder or an unscented transform for weakly nonlinear cases.
This work directly addresses the long-standing problem of data assimilation in nonlinear systems: how to combine the expressiveness of deep models with computational efficiency. It proposes a path through structural constraints, avoiding the curse of dimensionality typical of ensemble methods.
This section is addressed to the author of the paper and their colleagues. Neighbouring works were found by meaning-based search across our archive, not by word overlap. We do not review: we show the field around the work.
What the work does
The authors propose the CGKN model, which transforms a nonlinear dynamical system into a conditionally Gaussian form using neural networks, enabling analytical formulas for data assimilation. The model is tested on three turbulent systems: the stochastic Burgers–Sivashinsky equation, the Lorenz-96 system, and El Niño. Results show forecast accuracy on par with the best black-box methods while radically accelerating data assimilation.
Where its strength is
The key strength of the work lies in preserving nonlinearity in the observed variables while keeping the hidden dynamics conditionally linear, which provides both the expressiveness of neural networks and the analytical efficiency of assimilation. This is the first demonstration of embedding a conditionally Gaussian structure into deep learning for strongly nonlinear turbulent systems.
Where it could go next
Worth trying
Why this may matter
If CGKN can be extended to a broader class of problems with incomplete and noisy data, it could provide a universal tool for fast data assimilation in Earth sciences, astrophysics, and engineering, where expensive ensemble methods currently dominate.
Nearby works in our archive
The archive is sparse around this work: the closest review is only 0.55, against a usual 0.64. It looks like few have drilled nearby.
The Burgers–Sivashinsky equation used in the tests describes not only turbulence but also flame front instability, and its stochastic version generates extreme events reminiscent of sudden gusts in turbulent flow.