How to Find the Signal in a Data Flood ✎

Original: "Encoding Causal Macrovariables"
· Benedikt Höltgen
arXiv:2111.14724 · 2021-11-29 · CC BY-NC-ND 4.0 · independent analysis · 1 min
A neural network learns to compress data into a few numbers that explain the relationship between different measurements.

When you look at a thermometer, you see a single number: the temperature. But behind that number hides an unimaginable crowd of molecules, each moving at its own speed. Temperature is a macrovariable, that is, a coarse-grained characteristic that stands in for many fine details. Scientists have long sought such variables to explain complex systems.

The new algorithm does this automatically. It consists of two neural networks that learn simultaneously. The first network compresses ocean temperature data into a few numbers. The second does the same with air pressure data. Then each network tries to predict the full dataset of the other from its own compressed numbers. This trick is like an interpreter who listens to a long conversation and produces a short note, from which another person can reconstruct the entire meaning.

The algorithm found two known types of El Niño, even though it was never told that this phenomenon exists.

On synthetic data with 64 dimensions, the algorithm found exactly two macrovariables, as intended. It also correctly identified which variable is the cause and which is the effect. Previously, finding such variables was a matter of a scientist's intuition. Now the neural network itself finds variables that preserve all the mutual information between two datasets. This will help extract meaningful patterns in climatology, neuroscience, and other sciences.

The algorithm discovered two known types of El Niño even though it was not told about the phenomenon.

Searching for hidden causal variables echoes Isaac Asimov's heroes trying to derive the laws of psychohistory from the chaos of galactic history.

L[p(x|x)] = I(X; X) - β·I(X; Y)
Balance between compressing data and preserving information about the other variable
loss_netX = d1(Y, Ŷ) + β·DKL(N(0,1)||q(X|X)) + γ·d2(Y, Ŷ)
Sum of prediction errors, regularization, and bottleneck matching
Concepts
forecast granger causality Multivariate ENSO Index neural networks El Niño evidential neural networks seasonal cycle parameter interannual variability ERSST natural variability predictive model probabilistic model equatorial wave propagation probabilistic forecast seasonal forcing El Niño index labeled data composite analysis statistical analysis