A large neural network like Whisper masters hundreds of languages and resembles the standard model of the universe: everything is balanced. But training on a rare language disrupts this equilibrium — it turns into a black hole, pulling in all training resources. The network starts forgetting the other languages.
The Unified Gradient Projection (UGP) method prevents this skew. It acts like a gravitational field, evenly distributing 'attention' across languages. The gradients — the directions of network tuning — don’t tilt in favor of one language, and the spacetime curvature of knowledge remains uniform. The entropy of forgetting — a measure of disorder — approaches zero.
Amazingly, the human brain solves a similar problem during sleep: as it processes daily experiences, it evenly strengthens all memories so nothing is lost. UGP does for neural networks what sleep does for humans.
When learning a new language, the human brain can temporarily 'switch off' an old one — this is known as language interference. Neural networks suffer from a similar effect, but unlike us, they forget forever without special techniques.
The perfect assistant C-3PO from Star Wars knows millions of languages and never mixes them up — perhaps it has a built-in version of UGP.