We introduce IAFormer, a Transformer-based architecture that seamlessly incorporates pairwise particle interactions through a dynamic sparse attention mechanism. Unlike its predecessors, the attention matrix is built from predefined Lorentz-invariant pairwise features, slashing the parameter count. A second innovation, 'differential attention,' lets the model dynamically rank relevant particle tokens and cut computational waste on low-yield signals. The upshot? Complexity drops by an order of magnitude versus Particle Transformer, with no hit to accuracy: IAFormer achieves top-tier results on top-quark and quark-gluon jet classification benchmarks. Interpretability studies reveal layer-by-layer extraction of physically rich information via sparse attention, yielding an output that shrugs off statistical noise. These findings champion sparse attention in Transformer-based particle analysis as a way to shrink networks while boosting performance.
A head chef in a crowded kitchen doesn’t monitor every pot: he glances only at what’s boiling, sizzling, or about to burn. Similarly, IAFormer — a neural network for analyzing particle collisions — doesn’t waste time on all possible particle pairs. Its 'lazy' attention dynamically picks out only the combinations that really matter, much like the Standard Model describes only the key interactions, ignoring the noise.
But IAFormer’s main trick is that it uses special numbers that are independent of particle speed — like a recipe that works the same whether the kitchen is still or racing on a train. These invariants, tied to the speed of light, drastically reduce the network’s parameters, making the model transparent. Physicists can see how it filters out random spikes layer by layer and accumulates an understanding of physics.
Amazingly, the model is so lightweight that it runs on a laptop, processing data faster than the collisions occur.
🎯 The top quark is the heaviest known elementary particle, with a mass roughly equal to that of a whole tungsten atom, but it decays in 10^−25 seconds, not having time to bind with anything.