Taming Chaos with Physics: Forecasting Extreme Events from Sparse Data
Context
Forecasting chaotic systems, such as weather, epidemics, or ocean currents, remains a challenge: classical physics-based models require precise initial conditions and quickly diverge from reality, while purely data-driven approaches demand vast amounts of data. Extreme events compound the problem: they are rare, so data on them are sparse, and their dynamics are nonlinear and sensitive to noise. This work sits at the intersection of physics-informed machine learning and transfer learning: neighboring studies have shown that embedding conservation laws in neural networks helps model lake temperatures or espresso flows, but for chaotic systems with high-order derivatives, such methods have not worked.
Methods
The authors build the KDL architecture on an LSTM (long short-term memory) with three layers of 50 neurons each. First, they generate a synthetic time series from the Liénard equation with an external periodic forcing (parameters: f=0.2, α=0.45, γ=-0.5, ω=0.642, β=0.5) using the Runge-Kutta method. For each time step, they compute discrete first and second derivatives as differences between neighboring values. They then train the network to predict three quantities at once: x(t), dx/dt, and d²x/dt², minimizing the sum of the data error and a physical residual that requires the predicted values to satisfy the Liénard equation. After pretraining, the network weights are transferred to real data (El Niño, dengue fever, rainfall on Bjørnøya), where physical regularization compares the left-hand side of the equation for real and predicted series.
Results
On El Niño data, KDL achieved the lowest RMSE (0.428 versus 0.433 for LSTM with 20% training data) and the lowest physical residual (276.654 versus 283.340 for LSTM). When the training fraction is increased to 80%, the advantage persists but narrows. For dengue fever, transfer learning from synthetic data worsened the result, so the authors disabled pretraining and kept only physical regularization: KDL achieved the best physical residual at 40% and 60% data, though it lagged behind LSTM in RMSE. On Bjørnøya rainfall (15320 observations), KDL won on RMSE at 20%, 60%, and 80% training data, but lost on physical residual to reservoir computing (ESN). On average across all datasets and splits, KDL ranked first in RMSE (rank 1.12) and MAE (rank 1.29), outperforming LSTM, FFNN, CNN, and ESN.
Implications
This work shows that physical regularization and transfer learning from synthetic data allow deep networks to forecast chaotic extreme events even with few observations. Unlike neighboring studies, where physics is embedded via hard constraints or used to emulate PDEs, here derivatives are computed discretely and incorporated into the LSTM loss function, making the method applicable to real time series without exact boundary conditions. This expands the toolkit of physics-informed neural networks to problems with rare and irregular data, where classical PINNs fail.
Future development
The authors plan to extend KDL to multi-step forecasting, which is crucial for practical early warnings of extreme events. Adaptation to other types of nonlinear oscillators is possible, as is the inclusion of more complex physical laws, such as the Navier-Stokes equations for ocean currents. It is also promising to investigate when transfer learning from synthetic data helps and when it hurts (as in the dengue case), in order to build adaptive training strategies.
Impact
The method could improve early warning for climate anomalies (El Niño), infectious disease outbreaks, and extreme rainfall in regions with limited historical data. In engineering, it is applicable to forecasting rare failures in chaotic systems, for example in power grids.
Next steps
Test KDL on other datasets with extreme events, such as financial crashes or seismic activity. Investigate automatic selection of the regularization coefficient λ and the threshold at which transfer learning from synthetic data should be disabled.
Key open problems
This work directly addresses the long-standing problem of forecasting chaotic systems with limited data, where classical methods either require an exact model or a large volume of observations. It also touches on the issue of physical consistency of forecasts, which is important for trust in models in geophysics and epidemiology. An open question remains how universal the Liénard equation is in describing real extreme events and how to choose an appropriate physical model for a given dataset.
The Liénard equation, used to generate synthetic data, was introduced by French physicist Alfred Liénard in 1928 to describe oscillatory circuits and has since become a classic in the theory of nonlinear oscillators. Interestingly, extreme events in such systems arise as rare but recurring amplitude spikes, similar to rogue waves in the ocean.
The idea of predicting chaos with hybrid knowledge echoes Isaac Asimov's novel "Foundation," where psychohistory combines statistics and deterministic laws to forecast the future of large systems. Unlike Asimov, here physics is embedded in a neural network rather than in equations for masses, but the spirit of the hybrid approach is similar.
Key numbers
- El Niño RMSE at 20% data: 0.428 (KDL) vs 0.433 (LSTM)
- El Niño physical residual at 20% data: 276.654 (KDL) vs 283.340 (LSTM)
- Average rank of KDL in RMSE: 1.12 (first among 5 models)
- Number of observations in Bjørnøya rainfall data: 15320 (vs 1634 for El Niño)
- Nonlinearity parameter β in Liénard equation: 0.5