A hundred neural networks look at ocean temperatures and try to guess whether El Niño will arrive in two years. Each network gives its own answer, and if the answers diverge widely, the forecast is unreliable. That's how an ensemble works: a group of neural networks that learn independently of one another. In a new study, scientists showed that the ensemble's disagreement is the most honest signal that the climate has shifted and the network has stepped outside its knowledge.
Previously, the prevailing view was that the key was the system's own randomness, since the ocean is chaotic. But as the climate began to change, it turned out that this measure can be misleading: a network becomes overly confident in unfamiliar situations. The disagreement among the hundred networks, by contrast, grows and warns of an error. It's like a hundred forecasters arguing about tomorrow's weather: if they agree, you can trust the forecast; if they don't, better bring an umbrella for both sun and rain.
This approach can already be used in El Niño forecasting centers: monitor the ensemble's disagreement and flag in advance when a forecast is unreliable. That's especially important now, as the climate changes and old patterns stop holding.
The ensemble of 100 neural networks in this work is ten times larger than typical ENSO forecasting ensembles, and this size is exactly what allowed the two types of uncertainty to be reliably separated. Interestingly, for short forecasts (up to 6 months), aleatoric uncertainty does increase slightly under shift, but this signal is weaker than that of epistemic uncertainty.