Neurons Speak Plain Language express
An artist paints a picture, glancing at the audience in the hall. If the viewer smiles, the artist adds more red and rounded lines to the canvas; if the viewer frowns, the artist removes blue and sharpens the angles. In much the same way, neuroscientists "interrogate" neurons in the visual cortex, except instead of paints, they use hundreds of thousands of photographs, and instead of a smile, an electrical impulse.
For the very first visual area (V1), scientists long had a convenient model: neurons there respond to simple features — line orientation, stripe frequency. But in the neighboring area V4, where more complex images are assembled, no such clear language existed. The authors of the new study decided that the most ordinary, human language could serve as a universal one. They took digital copies of macaque brain areas (V1 and V4), "showed" them many images, and for each neuron selected those that elicited the strongest and weakest responses. Then they fed these images to a vision-language model, which produced a short caption — a hypothesis about what that neuron likes or dislikes.
To test the hypotheses, the scientists synthesized new, previously unseen images based on the same verbal descriptions and again "showed" them to the digital neurons. The result was striking: in area V4, images created from "favorite" descriptions made 96.1% of neurons produce a strong response (above the 95th percentile of natural stimuli), while "disliked" images suppressed 97.6% of neurons below the 5th percentile. Random images yielded only about 10% such hits. For V1, activation worked just as accurately, but suppression was harder to capture in words — apparently, what simple cells "dislike" is less well-defined.
This approach resembles a game of "telephone" between brain, image, and text, but with scientific rigor. It allows not just picking stimuli at random, but purposefully "interrogating" each neuron in a language we understand. In the long run, such a closed loop, from images to words and back to synthesized images, could become the basis for automated scientific discovery, where an algorithm itself formulates and tests hypotheses about brain function. Thus, an artist who started by guessing the viewer's mood gains a dictionary of their preferences, and paints a picture that lights up neurons from the first second.
In macaque area V4, researchers found neurons that respond to combinations of color and texture — for example, a red object on a wooden surface. This is surprisingly similar to how artists describe their works.
The idea of "reading thoughts" of neurons through language models echoes Ted Chiang's story "Story of Your Life," where deciphering an alien language changes the perception of reality.