How a Neural Network Learns to Speak Like a Human express
A language model is a multi-layered structure where each layer processes words in its own way. At the start of training, layers are chaotic, but gradually they organize into two teams. The first team works simply: its internal geometry smooths out, becoming less tangled. The second remains complex and intricate.
Scientists compared the model with fMRI data from people who listened to or read text. The similarity to the brain appeared unevenly: in temporal areas (responsible for the sound and meaning of words) it was fast and stable, while in frontal areas (responsible for building phrases) it came later and gradually. Moreover, for larger models, this connection is stronger.
It turns out that learning language is a path from chaos to order. The brain may do the same: smoothing some pathways of information processing while leaving others complex for flexibility.
Original abstract
as written by the authors arXiv:2602.07539
How large language models (LLMs) align with the neural representation and computation of human language is a central question in cognitive science. Using representational geometry as a mechanistic lens, we addressed this by tracking entropy, curvature, and fMRI encoding scores throughout Pythia (70M-1B) training. We identified a geometric modularization where layers self-organize into stable low- and high-complexity clusters. The low-complexity module, characterized by reduced entropy and curvature, consistently better predicted human language network activity. This alignment followed heterogeneous spatial-temporal trajectories: rapid and stable in temporal regions (AntTemp, PostTemp), but delayed and dynamic in frontal areas (IFG, IFGorb). Crucially, reduced curvature remained a robust predictor of model-brain alignment even after controlling for training progress, an effect that strengthened with model scale. These results links training-driven geometric reorganization to temporal-frontal functional specialization, suggesting that representational smoothing facilitates neural-like linguistic processing.
The layers of a language model can be compared to the floors of a building: lower floors break words into sounds and letters, upper floors build meaning from them. The study showed that 'simple' floors are closer to the entrance, and 'complex' ones are closer to the exit.
In the movie 'Her,' the artificial intelligence Samantha evolves and begins to understand people better than they understand themselves. This research is a step toward understanding how a machine could reach such a level.