main concepts scientists sections
SimplePopularAdvanced

Puppet Masters of Prediction: How to Find the Few That Control the Future ✎

Original: "Optimal model-free prediction from multivariate time series"
· Jakob Runge, Reik V. Donner, Jürgen Kurths
arXiv:1506.05822 · 2015-06-18 · arXiv non-exclusive · independent analysis · 1 min
Statistical ML stat.ME
To predict the future, you need to find a few key causes, not track every indicator at once.

A puppeteer pulls the strings, and the marionette dances. In forecasting the future from time series, the role of puppeteers is played by the few variables that truly govern the system. The other indicators merely mimic their movements. The task is to find these puppeteers and watch only their fingers, not the whole stage.

But how to distinguish a real cause from a chance coincidence? Standard methods often grab everything in sight and include extra indicators that do not help the forecast. Or they miss cases where only two variables together provide the needed information. The scientists proposed first to find the direct causes of the target variable using a special algorithm, and then to test all their combinations. Such an exhaustive search is feasible because the number of causes is small.

Direct causes are what directly influence the outcome. For example, for the temperature in a room, these are the radiator and the open window, but not the weather outside, which acts through the window.

On synthetic data, the method yields nearly zero error. For El Niño forecasting, it improves accuracy compared with the old approach. Previously, it was unclear how to find the best set for prediction without an enormous search. Now it is clear: first find the puppeteers, then watch how they move the marionette.

Whole graph →

The PC algorithm underlying causal selection was proposed in 1991 and named after the initials of its creators: Peter Spirtes and Clark Glymour. Since then, it has become a cornerstone of causal inference, and here it is applied for the first time to accelerate optimal prediction.

The idea of selecting causal drivers echoes the 'psychohistory' from Isaac Asimov's Foundation series, where mathematicians predict the future of society by identifying key driving forces. Instead of a galactic empire, here it is a climate index, but the principle is the same: find the few variables that truly govern the system.

H(Y_{t+1}) = I(Y_{t+1}; X^-_{t+1}) + H(Y_{t+1} | X^-_{t+1})
The total uncertainty of the future value Y_{t+1} consists of the information that can be extracted from the entire past X^-_{t+1} plus the residual entropy that cannot be predicted.
I(Y_{t+1}; X^-_{t+1}) = I(Y_{t+1}; P_{Y_{t+1}}) + I(Y_{t+1}; X^-_{t+1} \setminus P_{Y_{t+1}} | P_{Y_{t+1}})
The mutual information between the future value and the entire past equals the mutual information with the causal parents P_{Y_{t+1}}, because the second term is zero by the Markov property.
SRMSE = \sqrt{\frac{1}{|T_{test}|} \sum_{t \in T_{test}} \frac{(Y_{t+h} - \hat{Y}_{t+h})^2}{\sigma^2_Y}}
The forecast error is normalized by the variance of the target variable, allowing comparison of forecast quality across different series.
Concepts
model selection probabilistic forecast ensemble forecast prediction optimization problem probabilistic model maximum likelihood random forest spring predictability barrier predictive learning neural networks overfitting prediction accuracy training data Stochastic modeling profile likelihood Multivariate ENSO Index optimization optimal transport granger causality information criterion