Trees Predict Chaos Without Tuning ✛ ✎

Original: "Tree-based Learning for High-Fidelity Prediction of Chaos"
arXiv:2403.13836 · 2024-03-12 · CC BY-NC-SA 4.0 · independent analysis · 2 min
TreeDOX replaces neural networks with an ensemble of decision trees and automatically sets its parameters from the training data.

An orchestra without a conductor: the musicians take their seats, tune their instruments, and begin to play in harmony because each one hears the common rhythm. That is how TreeDOX works, a method that predicts chaotic time series without requiring the user to choose parameters or fine-tune anything. Previously, forecasting chaos was like a rehearsal with a temperamental soloist: neural networks are accurate, but they need lengthy tuning, while simple models fall short in quality. Now an ensemble of decision trees handles it on its own.

Chaos is not disorder but a system where a tiny error quickly grows into a large one. An attractor is the shape toward which a system's trajectory tends, like a funnel gathering streams of water.

The secret of TreeDOX lies in explicit memory. Windows are formed from the time series: each next value is predicted from several previous ones. Then Extra Trees Regression is used twice, first to estimate feature importance, then to train on a reduced set. The window size and the number of features are determined automatically via mutual information and statistical tests. No knobs to turn.

On the Hénon map and the Lorenz system, TreeDOX reproduced chaotic attractors with a correlation dimension matching to the fourth decimal place. For the Kuramoto–Sivashinsky equation with 64 spatial points, the forecast held for about 12.5 Lyapunov times. On the real Southern Oscillation Index, the method showed lower error and higher mutual information than LSTM and NG-RC. Training time was 0.63 seconds versus 200 seconds for neural networks with Bayesian optimization.

Lyapunov time is the period over which the divergence between two nearby trajectories of a chaotic system grows by a factor of e (approximately 2.7). It is a measure of predictability.

Forecasting chaos used to be an art of tuning: an expert would adjust hyperparameters for each problem, spending hours and computational resources. TreeDOX shows that an ensemble of trees with automatic delay selection can compete without a single hand-tuned parameter. Chaos forecasting becomes a routine operation, accessible to anyone who can call a ready-made library.

The orchestra has played the concert without a conductor, and the hall applauds. Trees that tune themselves predict climate indices, biomedical signals, and financial series. The method can still evolve: parallelization on a GPU would speed up training by tens of times, and adaptation to non-stationary data would allow the delay window to change over time. But already TreeDOX makes chaos forecasting as simple as calling a ready-made library.

For the Southern Oscillation Index, TreeDOX trained in 0.63 seconds, whereas tuning an LSTM via Bayesian optimization would take about 200 seconds.

Concepts
random forest xgboost boosted decision tree predictive learning neural networks retrospective forecast training data pattern recognition training probabilistic forecast autoregressive model transfer learning regularization methods self-supervised learning forecast reservoir computing