Why Brain-Computer Interfaces Are Hard to Compare and How to Fix That ✛
A store scale shows weight, but if you put apples on one side and a watermelon on the other, the comparison is unfair. The same goes for speech brain-computer interfaces: some systems know 50 words, others 125,000, and their accuracy varies. The new OVMI metric works like a scale that weighs both characteristics together: how many words the system knows and how accurately it recognizes them. This allows any systems to be compared on a single scale.
When OVMI was applied to 11 existing systems, it turned out that systems with large vocabularies convey up to 93.7% of the information, while those with 50 words convey only 2.4–6.7% for ordinary speech. But for a person who only needs to say "yes" or "no," a small vocabulary might be better. The metric also showed that choosing words by OVMI improves accuracy by 16.3% compared to standard frequency-based selection.
In the 1980s, DARPA organized a speech recognition competition that settled the debate between statistical methods and hand-crafted rules. OVMI could play a similar role for speech neurointerfaces.