You're telling a friend a story, and their brain seems to tune into your wavelength — the same regions light up in sync with your words. Scientists recorded this 'neural synchronization' using magnetoencephalography (a device that picks up the brain's magnetic fields) and decided to test whether a computer language model could replicate this dance of meaning.
They took stories told by real people and asked both human listeners and five versions of language models (from small to giant ones like GPT-4) to rate each significant word along ten axes of meaning: motion, time, emotion, sociality, and so on. Then they built a mathematical 'bridge' between human and model ratings to see how accurately the model predicts where a listener's brain will light up in unison with the storyteller.
The result was surprising: models do capture a significant portion of this neural synchronization, and the larger the model, the better the match. But when it comes to words related to intentions, emotions, or social roles, the computer 'listener' starts to go off-key. Its map of meaning remains flatter — with fewer shades of affect and volition.
This difference is especially noticeable with verbs like 'want,' 'fear,' or 'promise.' For the human brain, they carry a powerful social charge, while the model sees only a statistical shadow of these words. Yet the very fact that a soulless algorithm can partially reproduce the living pattern of shared understanding makes you wonder: perhaps a common language is not just biology but also a structure that can be measured and even improved.
The magnetoencephalography used in the experiment is so sensitive that it can detect magnetic fields a billion times weaker than Earth's — roughly like hearing a whisper from the other end of a football field.
In the novel 'The Difference Engine' by William Gibson and Bruce Sterling, a mechanical 'engine' tries to analyze human speech, but the soulless automaton cannot grasp irony and pain, much like modern AI models in this experiment.