Why the February Monsoon Forecast Flatters to Deceive ✎

Original: "Sensitivity of Indian summer monsoon rainfall forecast skill of CFSv2 model to initial conditions and the role of model biases"
arXiv:2101.07465 · 2021-01-19 · arXiv non-exclusive · independent analysis · 3 min
The CFSv2 model shows the best skill for forecasting the Indian monsoon when started in February, but that success is a fluke caused by a wrong La Niña prediction in 1983.

Context

Forecasting the Indian summer monsoon is critical for India's agriculture and economy, but modeling this phenomenon is complicated by many interacting processes. In the archive, this work sits alongside studies on weather and climate forecasting using machine learning and analysis of large-scale atmospheric structure, yet this article stands apart: it focuses on the classic dynamical model CFSv2 and the diagnosis of its systematic errors. Neighboring works focus on neural network approaches or statistical methods, whereas here the physics of errors in a specific forecasting system is dissected.

Methods

The researchers analyzed 124 nine-month retrospective forecasts from the CFSv2 model (NCEP version) with initial conditions from January 1 (4-month lead, L4) to May 31 (0-month lead, L0) over 1982–2010. Additionally, experimental retrospective forecasts were run on the current model version (CFSv2-CSIR) with five optimal initial conditions from late April to early May. For validation, they used IMD, GPCP, HadISST, OISST data and the ERA5 reanalysis. They evaluated the correlation of interannual variability, mean error, bias, RMSE, and spatial characteristics of precipitation climatology.

Results

The correlation between forecast and observations for L3 (February) was 0.44, higher than for L1/L0 (0.35), but when 1983 is excluded, the L3 correlation drops to 0.40, while for later starts it reaches 0.42. The L3 success in 1983 is explained by a wrong La Niña forecast: the model predicted cooling in the equatorial Pacific, whereas in reality temperatures were near normal, and the excess rainfall was driven by positive EQUINOO. Other metrics (mean error, bias, RMSE) favor April–May starts. The model shows an overly strong ENSO–rainfall link: the correlation between the NINO3.4 index and rainfall reaches -0.85 in L3 versus -0.44 in observations. The model also wrongly links EQUINOO to ENSO (correlation 0.83 in L3 versus 0.14 in observations) and even flips the sign of the EQUINOO–rainfall relationship (in the model -0.77, in observations +0.54).

Implications

This work shows that assessing forecast skill using a single metric (correlation) can be misleading due to chance coincidences in individual years. Unlike neighboring studies that apply novel machine learning methods to weather or climate forecasting, this article provides a detailed diagnosis of the physical mechanisms behind errors in an existing operational model. It underscores the need to use multiple metrics and to analyze specific events, not just average scores.

Future development

The topic could evolve toward improving atmospheric physics in CFSv2, especially convection parameterizations and ocean–atmosphere coupling, to reduce the cold bias in the central Pacific and weaken the excessive ENSO–rainfall link. Also promising is investigating the causes of climate drift at long lead times and testing the model with prescribed observed sea surface temperatures.

Impact

The results are important for operational weather and climate forecasting centers, especially the India Meteorological Department and the Ministry of Earth Sciences of India, which use CFSv2 for seasonal monsoon forecasts. They are also useful for model developers aiming to improve tropical climate predictability.

Next steps

Next steps include experiments with the atmospheric component of the model forced by observed sea surface temperatures to separate atmospheric errors from oceanic ones. It is also necessary to check how well the ocean model reproduces the main modes of variability under realistic atmospheric forcing.

Key open problems

This work directly addresses the fundamental problem of seasonal predictability: why models struggle to reproduce the independent influence of the Equatorial Indian Ocean Oscillation on the monsoon, and how to reduce systematic errors that limit the useful forecast lead time.

In 1983, the CFSv2 model predicted a La Niña that never happened, but this erroneous forecast coincidentally matched the real excess rainfall, artificially inflating its skill score.

Key numbers

  • L3 (February) correlation: 0.44
  • L3 correlation without 1983: 0.40
  • CFSv2-CSIR correlation without 1983: 0.42
  • Model ENSO-rainfall correlation (L3): -0.85
  • Observed ENSO-rainfall correlation: -0.44
Concepts
monsoon onset spring predictability barrier interannual variability forecast skill monsoon Indian Ocean Dipole Multivariate ENSO Index Forecast lead Jin-Timmermann model ENSO tropospheric biennial oscillation