A weather forecaster looks at a map where clouds gather in flocks, like birds before a storm. Each spot on the radar could be a future downpour or just noise. Earth system forecasting is like trying to assemble a puzzle from thousands of pieces that constantly change shape.
Previously, weather forecasting relied on numerical models that solve the physics equations of the atmosphere. They are accurate but slow and expensive. Neural networks have already learned to predict rain from sky images, but older architectures are too rigidly tied to the shape of the data. And weather is chaotic: a small change in one place can trigger a storm elsewhere.
The new model Earthformer cuts data into cubes, blocks of size time × height × width. Inside each cube, the neural network weighs the importance of points to each other. The cubes do not see each other directly, but a small set of global vectors links them. These vectors gather a summary from all cubes and send it back, like a courier delivering news across neighborhoods.
In tests with moving digits, Earthformer makes fewer errors than six other models. On real rainfall data, it predicts precipitation more accurately than Rainformer and UNet. For forecasting El Niño, the warm current that alters weather across the planet, the model also shows better results. The global vectors provide a boost without slowing down computations.
It was previously unclear how to apply transformers to three-dimensional weather forecasting data without drowning in computations. Now it is clear: cutting into cubes and using global vectors allows the neural network to work more accurately and faster. The model can be used for operational forecasting of precipitation and ocean temperature.
The forecaster looks at the map again, but now attention cubes assemble the world picture piece by piece, like a mosaic where each fragment knows about the whole.
The N-body MNIST dataset makes digits move according to the laws of three-body gravity, a chaotic system where the slightest change in initial velocity leads to a completely different trajectory. The model must guess where the digit '0' will fly, even though the exact solution is unstable.
The idea of predicting a chaotic system from limited observations echoes Liu Cixin's 'The Three-Body Problem', where a civilization tries to compute the motion of a planet in a system of three suns. Earthformer, like the novel's protagonists, confronts fundamental unpredictability but learns to give the best possible forecast.