Selecting the Best El Niño Predictors from a Model Ensemble ✛
A team of forty forecasters prepares an El Niño forecast two years ahead. Each works with the same model but slightly different initial conditions and settings, so their outputs diverge. Typically, forecasting centers average all forty opinions, assuming each member is equally useful. But in any team, some players hit the target more often, while others consistently miss.
The study authors decided to test whether the best can be identified in advance. They took an ensemble of 40 forecasts of ocean surface temperature in the El Niño region and compared each with actual observations from 1986–2017. For each forecast lead–season pair, members were ranked by two metrics: amplitude accuracy (how close the predicted event strength is) and phase matching (how precisely the timing is predicted). The top five by each metric were combined with weights proportional to their presence in both lists.
The result was stronger than expected. At short leads, the gain is modest: correlation increases by +0.02, error drops by 0.14 °C. But by month 23, the gap becomes dramatic: correlation increases by +0.43, that is, by 172%, and error decreases by 22.5%. Progress is especially notable in the transition seasons September–November and December–February, when accurately predicting the onset and maturity of the event is crucial. Statistical tests confirm that at leads longer than 12 months, more than 85% of lead–season pairs show significant improvement in correlation.
Previously, it was unclear how much spread in forecast quality exists within a single ensemble and whether it could be exploited. Now it is shown that in a set of 40 members, there is always a subgroup of 5–10 that predicts El Niño noticeably better than the ensemble average up to a 23-month lead, with the gain increasing with lead time. This is achieved by simply comparing each forecast with observations and selecting the best by two metrics.
The method can be embedded in operational weather and climate forecasting systems as a cheap way to improve long-term predictions without changing models. The next step is to learn to identify strong members in real time, when future observations are not yet known. This requires reliable methods for estimating forecast error without access to verification, as well as adaptive schemes that update member weights as new data arrive. If such tools can be created, the retrospective gain will turn into an operational advantage for forecasting centers. So a team of forty forecasters can become even more accurate if the coach only puts proven players on the field.
At a 23-month lead, the selected five members achieve a correlation 0.43 higher than the ensemble average, as if the forecast became almost one and a half times more 'in sync' with reality. And the computational cost does not increase by a single ruble.