Automatic speech judges are algorithms that evaluate pronunciation and emotions. However, they often work like a food critic who delivers a verdict after reading the menu but not tasting the dish.
Experiments showed that simply swapping two audio recordings makes the AI judge change its score, even though the sound remains the same. It latches onto the order of presentation, not the actual quality. Relying on the standard protocol of testing, rather than the true sound, such systems cannot be considered fair. To trust them in the future, every model must be checked for a tendency to cheat — otherwise, the evaluation will remain a fiction.
A neural network can give different scores to the same recording just by swapping the order of samples.