In a tropical rainforest, the dry season gives way to the rainy season, and with the weather, the rules of life change: some animals go into hibernation, while others come out of hiding. Similarly, in complex systems, from climate to economics, the links between processes are not set in stone. They depend on a hidden "regime" that itself switches from time to time. The new work offers a way to learn both things simultaneously without any labels.
The method works as follows: first, it hypothesizes which regime the system is in at each moment, then it checks how well these regimes explain the data. It alternately refines two things: the division of time into regimes and the causal links within each regime. On real data of El Niño and Indian rainfall, the method, without external information, identified two seasons: a summer regime with a strong negative link (coefficient −0.4) and a winter regime with almost no link. This means that the warm current in the Pacific Ocean indeed suppresses Indian rainfall in summer, but in winter its influence vanishes. The algorithm, without any hints, split the year into two seasons corresponding to the summer monsoon and winter, even though it was given only the time series of El Niño and rainfall.
Returning to the tropical rainforest image: we now have a tool that not only detects that in the rainy season some links strengthen and others weaken, but also finds these seasons from the data on its own. The work shows that one can simultaneously learn hidden regimes and causal structures without requiring prior knowledge of the regimes. This extends the applicability of causal analysis to a broad class of non-stationary systems, from climate to economics and neuroscience.
In the climate example, the method, without any hints, split the year into two seasons corresponding to the summer monsoon and winter, even though it was given only the time series of El Niño and rainfall.
The idea of hidden regimes governing a system's behavior evokes Clifford Simak's 'Way Station', where different states of reality replace one another, and the observer sees only their consequences.