This work stands apart: its nearest neighbors are express cards on AI for materials science, smart grids, and detectors, but there are no direct competitors in high-energy physics. However, the thematic group of the card—prostate_mri, medical_image_segmentation, dimensionality_reduction, weakly_supervised_learning—points to a common direction: using transformers and sparse representations for complex data analysis. In particle physics, transformers have already shown success in jet tagging, but existing models such as ParT require a large number of parameters and do not always effectively filter out soft radiation.
IAFormer replaces the standard dot product of queries and keys with a learnable pairwise interaction matrix between particles, which depends on Lorentz-invariant quantities (pair mass, angular distance, transverse momentum). Attention is computed as the difference between two copies of this matrix with a learnable coefficient β that suppresses the contribution of irrelevant pairs. The network has 12 layers with 16 attention heads and is trained on the public top tagging and quark-gluon tagging datasets, as well as JetClass. For interpretation, attention maps and CKA similarity are used.
On the top tagging task, IAFormer achieves an accuracy of 0.942 and an AUC of 0.9870 with 211 thousand parameters, an order of magnitude fewer than ParT (2.14 million). On quark-gluon tagging, it achieves an accuracy of 0.844 and an AUC of 0.9172 with 171 thousand parameters. On JetClass, the model with 890 thousand parameters shows competitive results. The number of FLOPs is reduced from 300 million (standard transformer) to 38 million. Analysis of β shows that its value increases in the early layers and decreases in the later ones, which is interpreted as building robust collective variables and then discarding redundant information.
Unlike neighboring works, which either use transformers for other domains or do not focus on sparse attention in particle physics, IAFormer is the first to systematically demonstrate that dynamic sparse attention based on pairwise interactions can radically reduce network size without sacrificing accuracy. This changes the perception of the required model complexity for jet tagging and paves the way for lighter and more interpretable architectures.
The topic could evolve toward further parameter reduction and studying the physical meaning of the coefficient β as a measure of effective degrees of freedom separating signal from background. The architecture could also be adapted to other classification tasks in high-energy physics, including identification of rare processes and searches for new physics.
The method will impact data analysis at the Large Hadron Collider, especially in the ATLAS and CMS experiments, where fast and accurate jet classification is crucial. It may also be useful in related fields that require processing unordered sets of objects with pairwise interactions.
Next steps include hyperparameter optimization and regularization of β for training stability, as well as testing on the full JetClass set of 100 million events. It would be interesting to investigate how sparse attention behaves in the presence of pile-up and multiple interactions.
This work directly addresses the unresolved problem of efficiently representing the hierarchical structure of hadronic jets under limited computational resources. It shows that physically motivated sparsity can replace redundant parameterization, which is important for the scalability of deep learning methods in high-energy physics.
This section is addressed to the author of the paper and their colleagues. Neighbouring works were found by meaning-based search across our archive, not by word overlap. We do not review: we show the field around the work.
What the work does
The paper introduces IAFormer, a transformer model for classifying hadronic jets that uses a learnable pairwise interaction matrix and dynamic sparse attention. The model achieves accuracy on par with the best existing methods while requiring significantly fewer parameters and FLOPs on top tagging, quark-gluon tagging, and JetClass tasks. Interpretability analysis shows that the network highlights physically meaningful structures.
Where its strength is
The strength of the work lies in demonstrating that sparse attention based on Lorentz-invariant pairwise interactions can radically reduce model size without sacrificing accuracy, paving the way for lightweight and interpretable architectures in particle physics.
Where it could go next
Worth trying
Nearby works in our archive
The archive is sparse around this work: the closest review is only 0.54, against a usual 0.64. It looks like few have drilled nearby.
The number of FLOPs for IAFormer (38 million) is about 8 times smaller than that of a standard transformer (300 million), yet the classification quality is even higher. It is like a translator who ignores background noise and works faster and more accurately than a colleague who listens to everything.