Reservoir with a Correction: How to Predict Noisy Chaos ✎

Original: "Reservoir Computing with Error Correction: Long-term Behaviors of Stochastic Dynamical Systems"
arXiv:2305.00669 · 2023-05-01 · arXiv non-exclusive · independent analysis · 2 min
A hybrid of reservoir computing and normalizing flows enables, for the first time, long-term prediction of stochastic dynamical systems and reproduction of their rare regimes.

A translator who learns to correct their own slips works more accurately. In the world of forecasting noisy processes, a new method takes on this role: a reservoir computer that adds a random correction to its prediction at each step. Previously, such systems quickly went off track because noise accumulated errors, like a translator who doesn't notice their own slips, and by the end of the speech the meaning was lost. Now the error is modeled separately and turned into a tool.

A reservoir computer is a network with fixed internal connections that learns only at the output. It predicted chaos without noise well, but as soon as randomness was added, the forecasts diverged from reality within a few steps. The authors of the RC-NF method took a different approach: first, they trained the reservoir on a set of trajectories, and then, using a normalizing flow, they estimated the error it makes at each step. A normalizing flow is a generative model that transforms a simple distribution into a complex one, and here it captures the error distribution.

During prediction, at each step a random correction generated from this distribution is added to the reservoir output. Thus, the error stops accumulating, and instead of a single trajectory, the method produces a whole fan of possible paths.

Tests on seven systems showed that the method reproduces not only the average behavior but also rare events. For a double-well process, where the system sometimes jumps from one stable point to another, RC-NF reproduced the transition rate with an error of less than 0.001. For the stochastic Lorenz system, the Wasserstein distance, a measure of dissimilarity between distributions, was 0.0266 versus 0.0479 for a pure reservoir computer. The median of the maximal Lyapunov exponent, which characterizes chaoticity, was 2.5622, compared to the true value of 2.5557.

The key difference from previous approaches: the method does not require knowing the system's equations and does not select special trajectories. It learns only from data and works directly with internal noise.

A translator who hears their own slips and corrects them on the fly preserves the meaning of a long speech. Similarly, RC-NF maintains the statistics of the process over long intervals, opening the way to forecasts of climate fluctuations, biological rhythms, and financial series without knowing their equations.

The RC-NF method reproduces the transition rate in a double-well system to the third decimal place, comparable to the accuracy of direct numerical simulation.

Concepts
reservoir computing stochastic dynamical system Stochastic modeling computational fluid dynamics Dynamical Systems optimization probabilistic forecast bootstrap probabilistic model Markov chain regularisation statistical method normalizing flows linear regression nonlinear laplacian spectral analysis patterned fluid dynamics biological rhythm modeling Navier-Stokes equations Monte Carlo methods regularization methods