How to Predict Chaos When Everything Is Noisy ✛

Original: "Adaptive Nonlinear Vector Autoregression: Robust Forecasting for Noisy Chaotic Time Series"
arXiv:2507.08738 · 2025-07-11 · CC BY 4.0 · 2 min
A trainable neural network instead of rigid polynomials makes forecasting chaotic series accurate even at 30% noise.

Imagine a translator in a noisy hall, trying to catch a speech through the buzz of voices. The usual approach is to memorize a set of phrases and slot them into templates. But when the noise cranks up, the template falls apart. Adaptive nonlinear vector autoregression works differently: it learns to grasp the meaning of the speech, filtering out the interference.

Standard nonlinear vector autoregression builds nonlinear features using a rigid polynomial template. That's like a translator who only knows memorized phrases.

In the new model, nonlinear features are generated by a trainable multilayer perceptron. The input is a delayed embedding: several consecutive values of the series, shifted in time. The neural network transforms this vector into features, which are then combined with linear ones and fed to a readout layer. The readout matrix is trained jointly with the perceptron via gradient descent, not computed through matrix inversion. This sidesteps the bottleneck of the classical approach.

On the Lorenz-96 system with 100 variables, the standard model would need about 0.8 terabytes of memory, roughly 200 full-length movies. The adaptive version got by with tens of megabytes.

Experiments on three chaotic systems, Mackey-Glass, Lorenz-63, and Lorenz-96, showed the advantage of the adaptive approach. At 30% noise on the Mackey-Glass system, the forecast error over 100 steps was 0.051 versus 0.073 for the standard model. On Lorenz-96, only the adaptive version could complete training and forecasting: the standard one hit a memory wall, and echo-state networks hit a computational time wall. The trainable nonlinearity proved more robust than fixed polynomials.

A translator who learns to understand meaning, rather than memorizing phrases, copes with noise. Likewise, a model that generates features from data, not from a template, predicts chaos more accurately. This is a step from hand-crafted features to automatic pattern extraction, which is especially valuable when the physics of the process is only approximately known.

Storing the feature matrix for standard NVAR on the Lorenz-96 system would require about 0.8 terabytes of memory—roughly 200 full-length high-definition movies. The adaptive version got by with just tens of megabytes.

The idea of trainable nonlinearity for predicting chaos resonates with Isaac Asimov's Foundation: there, psychohistory predicted the future of large masses of people based on statistical regularities. Here, the model learns the laws of chaos from data, loosely echoing Hari Seldon's attempt to formalize the unpredictable.

Concepts
nonlinear laplacian spectral analysis neural network regression predictive learning neural networks reservoir computing Large Language Models linear regression probabilistic forecast Weakly Supervised Learning regularization methods nonlinear dynamics protein language model autoregressive model parameter random network regularisation transformers autoencoder robust machine learning Stochastic modeling Generative Adversarial Networks