The puppet master pulls the strings, and the marionette dances. In forecasting the future from time series, the puppet masters are the few variables that truly control the system. The rest merely echo their movements, creating an illusion of complexity. The challenge is to find these puppet masters and watch only their fingers, not the whole stage.
But how do you tell a real cause from a random coincidence? Classical variable selection methods often grab at everything: they include redundant indicators that correlate with the target but carry no new information. Or, conversely, they miss synergistic combinations, where only the joint action of two variables provides predictive power. The authors propose first identifying the causal parents of the target variable using a modified PC algorithm, then enumerating all their subsets. Since causal parents are usually few, the enumeration becomes feasible.
The key idea rests on information theory. All predictive information about the future value of the target variable is contained in its causal parents. This follows from the Markov property: the future depends on the past only through the present state. Therefore, once the causal parents are found, the remaining variables can be ignored. To estimate mutual information, the authors use a nearest-neighbor method with the maximum norm, which is robust to high dimensionality.
On synthetic nonlinear processes, the method achieves a median relative error of zero, and in 90% of realizations the error is below 0.1. This is substantially better than schemes based on mutual information or direct selection. For the El Niño index, a linear model on optimal predictors reduces forecast error by 0.05–0.1 compared to the nearest-neighbor method at horizons up to 5 months.
Previously, it was unclear how to find a globally optimal predictor set without enumerating exponentially many combinations. Now it is clear that the search can be restricted to causal drivers, which are identified in polynomial time. This approach applies to climatology, economics, and other fields where data come as multivariate time series. The puppet masters are found, and the marionette is predictable.
The PC algorithm underlying causal selection was proposed in 1991 and named after the initials of its creators: Peter Spirtes and Clark Glymour. Since then, it has become a cornerstone of causal inference, and here it is applied for the first time to accelerate optimal prediction.
The idea of selecting causal drivers echoes the 'psychohistory' from Isaac Asimov's Foundation series, where mathematicians predict the future of society by identifying key driving forces. Instead of a galactic empire, here it is a climate index, but the principle is the same: find the few variables that truly govern the system.