Scientists have long used coarse-grained models to explain the behavior of fine-grained systems. Temperature serves as a macro-variable for the kinetic energy of molecules, and in climate models it becomes a micro-variable among hundreds of measurements. Choosing suitable variables has long been a matter of intuition, but now algorithms can do this automatically. The work sits alongside research on learning disentangled representations and causal inference, but tackles the problem for two datasets at once.
The author builds a causal autoencoder from two stochastic neural networks trained simultaneously. Each network compresses its input into a narrow bottleneck of a few neurons, then predicts the other dataset from it. An additional output predicts the current bottleneck of the second network, forcing the variables into a linear relationship. After training, the variables pass through variational autoencoders that seek monotonic transformations minimizing the dependence between the predictor and the residual in an additive noise model.
On synthetic data with 64-dimensional variables, the algorithm reliably found exactly two macro-variables in each set, as intended by the generative model. For the second pair of variables, the additive noise model test gave an HSIC score of 0.32 in one direction and 3.15 in the reverse, correctly identifying the cause. On climate data about El Niño, the algorithm extracted two patterns corresponding to the warm pool and the cold tongue. No causal link between these variables was found, consistent with the weekly resolution of the data.
The work offers a way to automatically construct continuous macro-variables that preserve all mutual information between two datasets. Unlike neighboring approaches that yield categorical clusters, these variables are continuous and allow testing causal relationships. This opens the door to more detailed analysis of complex systems where causal patterns lack a natural numeric scale.
The causal autoencoder architecture can be extended to recurrent or convolutional networks to handle time series and images. Another direction concerns the theoretical justification of the link between mutual information in observational data and causality measures from interventional distributions. It is also worth exploring other constraints on functions between variables to obtain more interpretable models.
The method could prove useful in climatology, neuroscience, and economics, where high-level models must be built from multivariate observations.
Test the approach on more complex synthetic data with common causes and on time series, and compare with other causal inference methods.
The work touches on the fundamental problem of variable selection for causal models, which philosophers of science are only beginning to systematize. It remains an open question how well Woodward's criteria hold in practice and how to assess the possibility of intervening on discovered variables.
The algorithm discovered two known types of El Niño even though it was not told about the phenomenon.
Searching for hidden causal variables echoes Isaac Asimov's heroes trying to derive the laws of psychohistory from the chaos of galactic history.