Peeking Inside the Mind of a Weather AI express
A forecaster issues a week-long forecast in seconds but cannot explain a single decision. That is roughly how modern AI weather models work: they already match classical numerical methods, yet their internal logic is hidden behind millions of parameters.
A team of researchers has built a tool that looks inside such a 'black box.' They took GraphCast, a graph neural network and one of the leading systems, and learned to spy on its hidden layers. The tool extracts the model's internal data representations (a kind of 'thoughts') and breaks them down into simple components using principal component analysis. This method highlights the strongest patterns — much like sorting photos by brightness and contrast.
In initial tests, they found directions that resemble things meteorologists know well: waves in the mid-latitude atmosphere and the distribution of water vapor. These are not random patterns. The model genuinely captures physical processes; it just records them in its own internal language. The tool helps translate that language into human terms.
These are only first steps, but they matter. If we learn to read the 'thoughts' of weather neural networks, we can not only check them for errors but also better understand the atmosphere itself. After all, the model might see connections that escape classical equations.
Original abstract
as written by the authors arXiv:2604.20467
Artificial Intelligence (AI) weather models are improving rapidly, and their forecasts are already competitive with long-established traditional Numerical Weather Prediction (NWP). To build confidence in this new methodology, it is critical that we understand how these predictions are generated. This is a huge challenge as these AI weather models remain largely black boxes. In other areas of Machine Learning (ML), mechanistic interpretability has emerged as a framework for understanding ML predictions by analysing the building blocks responsible for them. Here we present an open-source, highly adaptable tool which incorporates concepts from mechanistic interpretability. The tool organises internal latent representations from the model processor and allows for initial analyses, including cosine similarity and Principal Component Analysis (PCA), enabling the user to identify directions in latent space potentially associated with meteorological features. Applying our tool to the graph neural network GraphCast, we present preliminary case studies for mid-latitude synoptic-scale waves and specific humidity. These demonstrate the tool's ability to identify linear combinations of latent channels that appear to correspond to interpretable features.
GraphCast is a neural network that represents the atmosphere not as a grid but as a graph, where nodes are points on the globe and edges are connections between them. This approach allows it to capture long-range interactions more efficiently, such as how a storm in the Atlantic affects weather in Europe.