In the 1980s, the DARPA speech recognition challenge settled the debate between statistical methods and hand-written rules, setting a common standard for the entire field. For speech neural interfaces that translate brain activity into words, no such common benchmark has existed until now: each system uses its own vocabulary, recording method, and type of speech. Comparing them is as difficult as comparing scales that measure in kilograms and pounds without knowing the conversion rate. The new OVMI metric offers such a rate: it estimates how much information about the intended word the decoder conveys, considering not only accuracy but also which words are supported at all.

OVMI works like a scale with two pans. On one pan, you place decoding accuracy: the probability that the system correctly recognizes a word. On the other, lexical coverage: the proportion of words from a reference distribution that the system can potentially output. The reference distribution is the probabilities of words the user wants to say in real life, such as word frequencies in everyday speech. The product of these two quantities gives OVMI. If the vocabulary is small but accuracy is high, the scale shows little: the system cannot convey most of what is intended. If the vocabulary is huge but accuracy is low, the scale also tips down: errors eat away information. Only a balance of both pans gives an honest measure of communication.

Applying OVMI to 11 existing systems revealed a surprising picture. Invasive interfaces with a large vocabulary of 125,000 words convey between 72% and 93.7% of the information in the reference distribution, whereas systems with 50 words convey only between 2.4% and 6.7%. But the choice of reference distribution changes everything: the same 50-word Willett system conveys 6.4% for general speech, but 40.4% for clinical communication, where only basic phrases like "painful" or "thirsty" are needed. This shows that the usefulness of an interface depends on who uses it and in what situation.

OVMI allows us to see the progress of the entire field, not just individual benchmarks. It turns out that the growth of invasive systems has been driven mainly by expanding vocabulary, not by improving accuracy. This is an important takeaway for developers: instead of endlessly tuning accuracy on a fixed word set, you can expand the vocabulary, and the overall benefit to the user will increase. The metric also shows that non-invasive decoders currently convey only 0.3–3.6% of the information, meaning they are at the very beginning of the journey.

The OVMI scale does not replace other metrics, but it provides a common yardstick for comparing any systems. Just as the DARPA challenge in the 1980s standardized speech recognition, OVMI could become the standard for reporting in speech neural interfaces. In the future, it could be extended to contextual information, where the probability of a word depends on previous words. Then the scale would become even more accurate, weighing not only individual words but also entire phrases that a person wants to say.