An orchestra without a conductor plays a complex piece. The listener hears the overall music but wants to understand who leads the melody and who merely echoes it in unison. Similarly, it is hard to make sense of a complex system where many processes influence each other at once, be it climate, brain, or finance. We need to know where the direct link between two processes is, and where the collective effect lies.
Until now, scientists either ignored all other processes and looked only at the pair, or they considered all of them at once but could not tell which ones mattered and which were just noise. The new method does not search all subsets; it looks only for those that maximally weaken or maximally strengthen the information transfer between the two processes of interest. The difference between these extremes and the usual pairwise value shows how much of the data flow is due to the pure pair and how much to collective effects.
On climate data, where the influence of sea surface temperature in the tropical Pacific (El Niño) on the atmospheric pressure difference between Tahiti and Darwin (the Southern Oscillation) was analyzed, the method showed that about 20% of the information flow is due to collective effects. This means that even in such a tight pair as El Niño and the Southern Oscillation, a fifth of the information is not transmitted directly but through other climate indices, such as Atlantic Ocean temperature. Now we have a way to read the orchestral score and understand which instruments carry the melody, which merely echo it, and which create a harmony that exists in no single part.
El Niño and the Southern Oscillation are so tightly linked that they are combined into the acronym ENSO, but the proposed method shows that even in such a pair, about a fifth of the information is transmitted not directly but through other climate indices, such as the temperature of the Northern Tropical Atlantic or the Southern Tropical Atlantic.