main concepts scientists sections
SimplePopularAdvanced

How a neural network learns to hear chaos through noise ✛

Original: "Adaptive Nonlinear Vector Autoregression: Robust Forecasting for Noisy Chaotic Time Series"
· Azimov Sherkhon, Susana Lopez-Moreno, Eric Dolores-Cuenca, Sieun Lee, Sangil Kim
arXiv:2507.08738 · 2025-07-11 · CC BY 4.0 · 1 min
Machine Learning Artificial Intelligence Dynamical Systems
Instead of memorized phrases, it understands meaning: a trainable neural network keeps predictions accurate even under heavy noise.

Imagine an interpreter in a noisy hall, trying to make out a speech amid the buzz of voices. The usual approach is to memorize a set of phrases and slot them into a template. But when the noise grows, the template breaks. Adaptive nonlinear vector autoregression works differently: it learns to grasp the meaning of the speech while filtering out the interference.

Standard nonlinear vector autoregression builds nonlinear features using a rigid polynomial template. That is like an interpreter who knows only memorized phrases.

In the new model, a trainable neural network generates the nonlinear features. It takes several consecutive values of the series and transforms them into useful features. These features are then combined with the original ones and fed into a linear layer. Everything is trained jointly, without inverting large matrices. On the 100-variable Lorenz-96 system, the standard model would need about 0.8 terabytes of memory, while the adaptive version got by with tens of megabytes.

Experiments on three chaotic systems showed that the adaptive model is more accurate than the standard one, especially with noise up to 30%. On the Mackey–Glass system, the forecast error over 100 steps was 0.051, compared with 0.073 for the standard model. On the 100-dimensional system, only the adaptive version could complete training. An interpreter who learns to understand meaning rather than memorize phrases is not thrown off by noise.

Whole graph →

Storing the feature matrix for standard NVAR on the Lorenz-96 system would require about 0.8 terabytes of memory—roughly 200 full-length high-definition movies. The adaptive version got by with just tens of megabytes.

The idea of trainable nonlinearity for predicting chaos resonates with Isaac Asimov's Foundation: there, psychohistory predicted the future of large masses of people based on statistical regularities. Here, the model learns the laws of chaos from data, loosely echoing Hari Seldon's attempt to formalize the unpredictable.

Concepts
nonlinear laplacian spectral analysis neural network regression predictive learning neural networks reservoir computing Large Language Models linear regression probabilistic forecast Weakly Supervised Learning regularization methods nonlinear dynamics protein language model autoregressive model parameter random network regularisation transformers autoencoder robust machine learning Stochastic modeling Generative Adversarial Networks