Neural Network Predicts Ocean Temperature Three Months Ahead ✛ ✎

Original: "CAS-Canglong: A skillful 3D Transformer model for sub-seasonal to seasonal global sea surface temperature prediction"
arXiv:2409.05369 · 2024-09-09 · arXiv non-exclusive · independent analysis · 4 min
Deep learning has for the first time outperformed physical models in forecasting sea surface temperature on timescales from two weeks to a season.

Context

Forecasting on subseasonal to seasonal timescales (from two weeks to three months) remains a weak point in meteorology: the atmosphere forgets its initial conditions within a month, while the ocean has not yet accumulated enough energy to dictate the weather. General-purpose physical models require enormous computational resources and suffer from systematic errors in describing ocean–atmosphere interactions. Around this challenge, studies have emerged using machine learning to predict El Niño, solar flares, and droughts, but none has tackled the global sea surface temperature forecast on subseasonal timescales. CAS-Canglong joins this line of work but stands out in its target: instead of individual indices or regions, the model predicts the entire ocean temperature field at once.

Methods

The authors compiled 63 years of monthly ERA5 reanalysis data at a resolution of 0.25°×0.25° (1440×721 grid points). As predictors, they used eight variables: sea level pressure, zonal and meridional wind at 850 and 500 hPa, geopotential heights, latent heat flux, and net solar radiation. The model is built on the Swin Transformer architecture with windowed multi-head attention. The input tensor has dimensions of 16 months × 8 variables × 721 latitudes × 1440 longitudes. The output is sea surface temperature for the next one, two, or three months. Training was performed on four A100 GPUs with 80 GB of memory each. The loss function combined mean squared error with a penalty for low correlation (below 0.5) at long lead times. To reduce uncertainty, an ensemble of several models with different initializations was trained.

Results

On the independent test for 2016–2022, CAS-Canglong achieved a correlation of 0.91 for the Niño 3.4 index forecast at one month lead and 0.57 at three months. Compared with 22 physical models from the ECMWF S2S and NMME ensembles, the error decreased by 13.7±3.0% in the southern Pacific Ocean and by up to 77.1±0.6% in the Arctic Oscillation region. On average across seven key regions, the improvement ranged from 28% to 40%. The model correctly predicted 85% of El Niño and La Niña events three months ahead. In the southern oceans, correlation reached 0.9 at one month and 0.7 at three months. For the warm pool of the Indo-Pacific region, the model estimated an area of 5.93×10⁷ km² against the actual 6.17×10⁷ km², an error of 3.8%. In real-time mode for 2024, the model predicted the transition from El Niño to La Niña earlier than other models.

Implications

This work differs from neighboring studies in that it is the first to apply a three-dimensional transformer to the global ocean temperature field on subseasonal timescales. Previous approaches either used LSTM for local points, predicted individual indices, or operated on shorter or longer timescales. CAS-Canglong shows that attending to spatiotemporal relationships in reanalysis data can circumvent the limitations of physical models without explicitly describing physics. This is not just an improvement in metrics but a paradigm shift: instead of expensive ensemble calculations, one can train a single neural network and obtain a forecast in seconds.

Future development

The model already supports forecasts up to 12 months, although only the first three were evaluated in the paper. The architecture allows transfer learning to other variables: precipitation, soil moisture, wind. The next natural step is forecasting extreme events on land associated with ocean temperature anomalies, such as droughts in the Amazon or heatwaves in western North America. Expansion to seasonal forecasting of sea ice and ocean biogeochemical parameters is also possible.

Impact

Improved ocean temperature forecasting directly affects warnings of droughts, floods, and tropical cyclones months before the event. This is important for agriculture, water resource management, and insurance.

Next steps

Test the model on independent data from other reanalyses and on real-time forecasts over several years. Integrate ocean temperature forecasts into models for predicting precipitation on land.

Key open problems

In Earth sciences, the problem of predictability on subseasonal timescales remains unsolved: how long does memory of initial conditions persist in the ocean–atmosphere system? This work shows that deep learning can extract this memory from data better than physical models, but the question of physical interpretability of such forecasts remains open.

The knowledge machine's view

This section is addressed to the author of the paper and their colleagues. Neighbouring works were found by meaning-based search across our archive, not by word overlap. We do not review: we show the field around the work.

What the work does

The authors compiled 63 years of ERA5 reanalysis, trained a three-dimensional Swin transformer on eight atmospheric variables, and showed that the model predicts global sea surface temperature 1–3 months ahead more accurately than physical ensembles in seven key regions, including the Niño 3.4 index and the warm pool area.

Where its strength is

This is the first application of a three-dimensional transformer to the global ocean temperature field on a subseasonal scale, allowing it to capture spatiotemporal relationships without explicit physics and yielding a gain in accuracy over expensive physical models.

Where it could go next

Worth trying

  • Try training the model on higher-resolution data or adding oceanic variables to see if accuracy improves in the tropics.
  • Test transfer learning: fine-tune CAS-Canglong on data from another reanalysis or climate model to assess robustness.
  • Use an ensemble of different architectures (e.g., adding convolutional layers) to reduce uncertainty in long-term forecasts.

Why this may matter

Improved subseasonal sea surface temperature forecasts directly enhance early warnings for droughts, floods, and heatwaves, which is especially important for vulnerable regions.

The model's name, Canglong, refers to a mythical dragon from ancient Chinese mythology that controls rain. The model was trained on data from 1959 onward; over that period, ocean temperature in the tropics has risen by about 0.5 °C.

In Kim Stanley Robinson's novel 'The Ministry for the Future', heat in India becomes a trigger for global change. Accurate ocean temperature forecasts months ahead could provide time to prepare for such events.

L_{fmse} = \frac{1}{T_{out}} \sum_{t=1}^{T_{out}} (X_{t,l} - \hat{X}_{t,l})^2
Penalty for deviation of predicted temperature from actual, averaged over all forecast months
L_{corr} = \frac{1}{T_{out}} \sum_{l=2}^{T_{out}} \max\{0, 0.5 - \frac{\sum_{t=1}^{T} (x_{t,l} - \bar{X}_l)(\hat{x}_{t,l} - \bar{\hat{X}}_l)}{\sqrt{\sum_{t=1}^{T} (x_{t,l} - \bar{X}_l)^2} \sqrt{\sum_{t=1}^{T} (\hat{x}_{t,l} - \bar{\hat{X}}_l)^2}}\}
Additional penalty if the correlation between forecast and observations falls below 0.5 at lead times of two months or more
RMSE = \sqrt{\frac{\sum_{i=1}^{N_{lon}} \sum_{j=1}^{N_{lat}} W(i) (\hat{X}_{i,j,l} - X_{i,j,l})^2}{N_{lon} \times N_{lat}}}
Forecast error weighted by cosine of latitude to account for varying grid cell sizes

Key numbers

  • accuracy improvement: from 13.7% to 77.1%
  • Niño 3.4 correlation at 1 month: 0.91
  • Niño 3.4 correlation at 3 months: 0.57
  • El Niño prediction accuracy at 3 months: 85%
  • grid resolution: 0.25°×0.25°
Concepts
sea surface temperature anomaly sea surface temperature NOAA CPC ENSO spring predictability barrier forecast El Niño La Niña NOAA PMEL / TAO ERSST Niño 4 Jin-Timmermann model OISST v2.1 forecast skill zebiak cane model IRI Indian Ocean Dipole North Atlantic Oscillation Southern Oscillation ocean-atmosphere coupling