A meteorologist looks at a map where clouds gather in flocks, like birds before a storm. Each spot on the radar could be a future downpour or just noise. Earth system forecasting is like trying to assemble a puzzle with thousands of pieces that keep changing shape.

For years, forecasters relied on numerical models that grind through the physics equations of the atmosphere and ocean. These machines are accurate but slow and expensive. Satellites and radars have accumulated mountains of data over decades, and neural networks have already learned to predict rain from sky images. But older architectures, such as convolutional and recurrent networks, are too rigidly tied to the data's shape. The Earth system is chaotic: a butterfly flapping its wings in Brazil could trigger a tornado in Texas.

The authors of Earthformer found a way to tame this greed. They slice the input into cubes, blocks of size time × height × width. Inside each cube, the neural network performs self-attention, weighing the importance of points relative to each other. Cubes do not see each other directly, but a small set of global vectors links them. These vectors gather a summary from all cubes and send it back, like a courier delivering news across neighborhoods.

The architecture is hierarchical: first, convolutional layers compress the data, then it is expanded again. The decoder produces the forecast not step by step but all at once, using positional embeddings. On the synthetic MovingMNIST dataset, where digits move across the screen, Earthformer achieves an MSE of 41.79, and on N-body MNIST, where digits obey the gravity of three bodies, it reaches 14.82. This beats six baseline models.

Global vectors provide a stable improvement without increasing computation: parameters grow from 6.6 million to 7.6 million, while GFLOPS barely change. The hierarchy reduces the error on MovingMNIST from 48.04 to 46.91. Qualitatively, the model tracks digit trajectories in N-body MNIST more accurately, whereas baseline methods blur the image or confuse direction.

Previously, it was unclear how to apply transformers to three-dimensional Earth system forecasting data without drowning in computation. Now it is clear: cutting into cubes and using global vectors allow the transformer to outperform convolutional and recurrent networks on precipitation and El Niño tasks. The model can be used for operational forecasting of precipitation and ocean temperature, and as a basis for hybrid physics-neural network systems.

The meteorologist looks at the map again, but now attention cubes assemble the world picture piece by piece, like a mosaic where each fragment knows about the whole.