Why neural networks are like black holes: a lesson from 'bald' models ⚡ экспресс
Scientists figured out why the LoRA method—an elegant way to train large language models by changing only a tiny fraction of parameters—works so well. They examined the internal "entanglements" (similar to quantum ones) and found a mysterious "entanglement valley". But just like a black hole hides its secrets, the model's external behavior remains simple—that's the secret to its effectiveness.
Fine-tuning a neural network is like trying to alter a black hole without touching its event horizon: you change only a tiny fraction of parameters. Researchers looked inside and saw stunning patterns—strong links between pieces of the model, akin to quantum entanglement. By measuring entropy (a measure of disorder), they discovered an 'entanglement valley'—a sharp drop in orderliness that depends on the training method.
In practice, this means: it doesn't matter what patterns exist inside—only the external behavior counts. So we can develop lightweight AI adaptation methods without diving into the details.
🎯 The 'no-hair' theorem: a black hole is described solely by mass, charge, and spin. All other information about consumed matter is erased.