A teacher in class explains the difference between 'dog' and 'tog': put your hand on your throat and feel the vibration. A speech recognition machine does much the same thing, except inside it's not a throat but long lists of numbers. It turns out these numbers can be sorted onto shelves, like alphabet blocks in a child's room.
Scientists took a model trained on 96 languages and found directions inside it that correspond to individual sound features. For example, there's a 'voicing' direction. If you take the vector for 'p' and add this vector, you get 'b'. If you add not the whole vector but half of it, you get a sound that's half-voiced, as if the speaker started to whisper but didn't finish.
This discovery has practical value. If we understand how sounds are organized inside the model, we can better tune speech recognition for rare languages with few recordings. Or make voice assistants less likely to confuse 'cot' and 'got' in accented speech. The model has essentially been handed a phonetic atlas, and now it can travel through it deliberately, not by feel.
as written by the authors arXiv:2602.18899
Self-supervised speech models (S3Ms) are known to encode rich phonetic information, yet how this information is structured remains underexplored. We conduct a comprehensive study across 96 languages to analyze the underlying structure of S3M representations, with particular attention to phonological vectors. We first show that there exist linear directions within the model's representation space that correspond to phonological features. We further demonstrate that the scale of these phonological vectors correlate to the degree of acoustic realization of their corresponding phonological features in a continuous manner. For example, the difference between [d] and [t] yields a voicing vector: adding this vector to [p] produces [b], while scaling it results in a continuum of voicing. Together, these findings indicate that S3Ms encode speech using phonologically interpretable and compositional vectors, demonstrating phonological vector arithmetic. All code and interactive demos are available at https://github.com/juice500ml/phonetic-arithmetic .
The study found that if you take the difference between 'd' and 't' and add it to 'p', you get 'b'. It's like subtracting 'tom' from 'dom' and getting the instruction 'make it voiced'.