SEA-ViT: How a Neural Network Learns to Read Currents ✛

Original: "SEA-ViT: Sea Surface Currents Forecasting Using Vision Transformer and GRU-Based Spatio-Temporal Covariance Modeling"
· Teerapong Panboonyuen
arXiv:2409.16313 · 2024-09-14 · CC BY 4.0 · 1 min
A neural network predicts ocean currents like an interpreter who listens to the whole room and remembers the entire conversation.

An interpreter in a large hall doesn't just listen to words. They follow all conversations at once, remember who said what earlier, and anticipate what comes next. SEA-ViT works the same way, except instead of voices, it listens to the ocean.

Ocean currents resemble a conversation: water doesn't move on its own but is driven by wind, tides, and distant climate events. Previously, computers processed each point separately, as if an interpreter heard only one person. SEA-ViT uses two techniques. One layer sees the entire current field at once, like an interpreter scanning the room. Another layer remembers what happened yesterday and a week ago, because it reads data both forward and backward in time.

ENSO is a climate seesaw: warm El Niño and cold La Niña alternate. It influences how currents behave.

When El Niño arrives, water off the coast of Thailand moves differently. SEA-ViT gets a hint about this, like an interpreter told that everyone in the hall will speak faster today. On such days, errors are penalized more strictly, so the model learns to be attentive precisely when it matters most.

Now SEA-ViT can predict currents in the Gulf of Thailand and the Andaman Sea. Sailors and fishers will get a forecast that hears the whole ocean, not just a single point near the shore. And then the interpreter in the meeting hall becomes not just a comparison but real work: SEA-ViT translates the language of the sea into the language of safe routes.

The model is named SEA-ViT, simultaneously referencing the sea and the Vision Transformer architecture. The author thanks himself in the acknowledgments section, a rarity in scientific papers.

u' = \frac{u - \mu_u}{\sigma_u}
Z-score standardization to zero mean and unit variance
\text{Attention}(Q, K, V) = \text{softmax}\left(\frac{QK^T}{\sqrt{d_k}}\right)V
Weighted sum of values V, where weights are determined by similarity between queries Q and keys K
L = \frac{1}{N}\sum_{i=1}^{N}\left((U_i - \hat{U}_i)^2 + (V_i - \hat{V}_i)^2\right) + \lambda \cdot \text{ENSO\_weight} \cdot \left((U_i - \hat{U}_i)^2 + (V_i - \hat{V}_i)^2\right)
Mean squared error plus a penalty for errors during El Niño and La Niña periods
Concepts
sea surface temperature OISST v2.1 OISST NRT forecast IRI ocean current gps free navigation navigation Multivariate ENSO Index hydrodynamic simulation