main concepts scientists sections
SimplePopularAdvanced

How to Predict Chaos When Data Is Scarce?

Original: "Knowledge-based Deep Learning for Modeling Chaotic Systems"
· Zakaria Elabid, Tanujit Chakraborty, Abdenour Hadid
arXiv:2209.04259 · 2022-09-09 · CC BY 4.0 · 1 min
Machine Learning Computational Physics Statistical ML
A neural network first learns on artificial examples, then understands real extreme events.

Think of a translator who learned a language's grammar from textbook examples and now understands live speech with its accents and noise. That's how the new method KDL works: first, the neural network is trained on artificial data generated by the Liénard equation, then it is transferred to real time series where extreme events are rare and unpredictable.

Extreme events include giant rogue waves in the ocean, El Niño episodes, and sudden rainfall spikes. They are rare, so data are scarce, and their dynamics are nonlinear and sensitive to noise.

The secret of KDL lies in physical regularization: the network predicts not only the quantity itself but also its derivatives, and the loss function includes a term that enforces the differential equation. It's as if the translator didn't just memorize phrases but learned the grammar, enabling recognition of even unfamiliar constructions. On El Niño data, KDL achieved the lowest RMSE error (0.428 vs. 0.433 for LSTM with 20% training data) and the smallest physical residual (276.654 vs. 283.340). When the training fraction is increased to 80%, the advantage persists but narrows.

Whole graph →

The Liénard equation, used to generate synthetic data, was introduced by French physicist Alfred Liénard in 1928 to describe oscillatory circuits and has since become a classic in the theory of nonlinear oscillators. Interestingly, extreme events in such systems arise as rare but recurring amplitude spikes, similar to rogue waves in the ocean.

The idea of predicting chaos with hybrid knowledge echoes Isaac Asimov's novel "Foundation," where psychohistory combines statistics and deterministic laws to forecast the future of large systems. Unlike Asimov, here physics is embedded in a neural network rather than in equations for masses, but the spirit of the hybrid approach is similar.

\frac{d^2x}{dt^2} + \alpha x \frac{dx}{dt} + \gamma x + \beta x^3 = f \sin(\omega t)
Second-order equation where α, γ, β are nonlinearity and damping parameters, f and ω are amplitude and frequency of the external force
L_{phy} = RMSE\left(\frac{d^2y_{pred}}{dt^2} + \alpha y_{pred}\frac{dy_{pred}}{dt} + \gamma y_{pred} + \beta y_{pred}^3, f\sin(\omega t)\right)
Root mean square error between the left-hand side of the Liénard equation for predicted values and the external forcing
L_{phy} = RMSE\left(\frac{d^2y_{real}}{dt^2} + \alpha y_{real}\frac{dy_{real}}{dt} + \gamma y_{real} + \beta y_{real}^3, \frac{d^2y_{pred}}{dt^2} + \alpha y_{pred}\frac{dy_{pred}}{dt} + \gamma y_{pred} + \beta y_{pred}^3\right)
Comparison of the left-hand sides for real and predicted series, so that the forecast follows the same dynamics
Concepts
neural network physics modeling predictive learning neural networks Large Language Models chaos theory Dynamical Systems deep learning in medical imaging computational fluid dynamics complex system dynamics stochastic dynamical system complex systems quantum chaos training data mathematical model training forecast skill protein language model Stochastic modeling generalization probabilistic forecast computational reaction analysis