📄 Showing the "Simple" version — "Advanced" is not ready yet. Add it to favorites to help prioritize it. Ask for the full read

Inside a cell, work is in full swing: thousands of genes switch on and off like lights on a huge display. Neural networks trained to read this display have learned to guess what will happen if you 'turn off' one gene or add a drug. But how exactly they do it is a mystery. Scientists decided to look inside and check: does the model's 'attention' (which genes it looks at more closely) really reflect real connections in the cell?

They took two well-known models — scGPT and Geneformer — and ran blood cell and retinal cell data through them. It turned out that attention does build structure: in the early layers of the model, you can see connections between proteins that physically touch each other, and in later layers, connections between genes that regulate each other. Pretty, like a subway map. But when these 'maps' were used to predict a cell's response to perturbation, they didn't help. A simple method that just looks at which genes tend to change together gave a score of 0.81–0.88 on the accuracy scale, while the 'smart' attention connections gave only 0.70. If you remove these connections altogether, predictions don't get worse.

It's like a translator who makes detailed margin notes at a meeting but translates worse than a dictionary. The scientists proposed a way to 'retrain' attention — CSSI, which helps the model better reconstruct real gene networks (up to 1.85 times more accurately). But even after that, simple methods remain stronger. For science, this is an important lesson: don't trust pretty pictures; verify usefulness with numbers.

Original abstract

as written by the authors arXiv:2602.17532

We present a systematic evaluation framework - thirty-seven analyses, 153 statistical tests, four cell types, two perturbation modalities - for assessing mechanistic interpretability in single-cell foundation models. Applying this framework to scGPT and Geneformer, we find that attention patterns encode structured biological information with layer-specific organisation - protein-protein interactions in early layers, transcriptional regulation in late layers - but this structure provides no incremental value for perturbation prediction: trivial gene-level baselines outperform both attention and correlation edges (AUROC 0.81-0.88 versus 0.70), pairwise edge scores add zero predictive contribution, and causal ablation of regulatory heads produces no degradation. These findings generalise from K562 to RPE1 cells; the attention-correlation relationship is context-dependent, but gene-level dominance is universal. Cell-State Stratified Interpretability (CSSI) addresses an attention-specific scaling failure, improving GRN recovery up to 1.85x. The framework establishes reusable quality-control standards for the field.