This work sits at the intersection of machine learning and climate science, alongside research on uncertainty in quantum calculations, neural network chemistry, and El Niño prediction. It is already known that probabilistic neural networks can estimate aleatoric uncertainty and that ensembles improve calibration, but a detailed analysis of the mechanisms under climate shift was lacking. This article fills that gap: it is the first to systematically compare the behavior of aleatoric and epistemic uncertainty for ENSO under anthropogenic warming.
Data from 100 ensemble members of the CESM2-LE climate model from 1850 to 2100 were used. For each month, anomalies of sea surface temperature, heat content of the upper 300 meters, and zonal wind stress were computed. A ResNet-18 neural network with additional layers takes the last three months of anomalies and predicts the ENSO category up to 24 months ahead. Three ensembles of 100 components each were trained: on the pre-industrial period (1850–1949), the modern period (1850–2024), and the scenario period (1850–2098). Aleatoric uncertainty was estimated as the average entropy of component predictions, while epistemic uncertainty was the variance of probabilities across components.
When the input data distribution shifts due to climate change, forecast quality degrades, but epistemic uncertainty (ensemble disagreement) steadily increases starting around 2040, and this increase correlates with worsening error. Aleatoric uncertainty, in contrast, even decreases for long horizons (more than 10 months), although forecasts become worse: the model becomes overconfident in unfamiliar conditions. The ensemble's improvement over the average component grows with epistemic uncertainty, meaning the ensemble is especially valuable precisely under shift. For short horizons (up to 6 months), aleatoric uncertainty does increase slightly, but its signal is weaker and less reliable.
This work shows that in tasks with high inherent randomness, such as ENSO forecasting, relying only on aleatoric uncertainty is dangerous: under distribution shift it can mislead. Epistemic uncertainty, measured by ensemble disagreement, serves as a reliable indicator that the model has moved beyond its training data. This changes practice: for climate forecasts under climate change, both types of uncertainty must be tracked, not just the confidence of a single network. Compared to neighboring works that focused on calibration or aleatoric uncertainty, this is the first to show that the epistemic component is the key signal of degradation.
The topic may develop toward cheaper methods for estimating epistemic uncertainty without a full ensemble, for example via ensemble distillation into a single network or evidential deep learning. It is also promising to investigate how these results transfer to other climate variables and regions, as well as to real observational data, not just model data. Adaptive ensembles that automatically increase the number of components upon detecting distribution shift may also emerge.
The results directly affect operational seasonal ENSO forecasting in a changing climate, helping to assess forecast reliability. They also apply to other climate forecasting tasks where uncertainty matters, such as precipitation or extreme event forecasting.
The next step is to test whether the advantage of epistemic uncertainty holds with real observational data and lighter architectures. It is also important to investigate whether the ensemble can be automatically calibrated upon detecting shift, to maintain forecast reliability in real time.
In climate forecasting, the problem of uncertainty estimation under non-stationary climate remains unsolved: traditional methods assume stability of statistical relationships, which is violated by anthropogenic warming. This work directly links the growth of epistemic uncertainty to climate shift and shows that aleatoric uncertainty cannot serve as a substitute. This raises the need for new metrics and methods that are robust to changes in input distribution.
This section is addressed to the author of the paper and their colleagues. Neighbouring works were found by meaning-based search across our archive, not by word overlap. We do not review: we show the field around the work.
What the work does
The authors train ensembles of 100 probabilistic neural networks on CESM2-LE data to forecast ENSO and split uncertainty into aleatoric and epistemic components. They show that under a climate shift, epistemic uncertainty grows and correlates with error, while aleatoric uncertainty behaves paradoxically, decreasing for long forecast horizons.
Where its strength is
A systematic comparison of how the two types of uncertainty behave under a distribution shift for a stochastic climate system offers a concrete tool for monitoring forecast reliability.
Where it could go next
Worth trying
Why this may matter
The practical value could lie in building early warning systems for forecast unreliability under climate change, which is critical for planning in agriculture, energy, and risk management.
Nearby works in our archive
The archive is sparse around this work: the closest review is only 0.54, against a usual 0.64. It looks like few have drilled nearby.
The ensemble of 100 neural networks in this work is ten times larger than typical ENSO forecasting ensembles, and this size is exactly what allowed the two types of uncertainty to be reliably separated. Interestingly, for short forecasts (up to 6 months), aleatoric uncertainty does increase slightly under shift, but this signal is weaker than that of epistemic uncertainty.