A new system analyzes a short interview to recognize human doubt by combining text, voice, and facial expressions. But the main key to uncertainty turned out to be the dark matter of speech — pauses and filler words that standard speech systems erase like a black hole. It is these gaps, usually considered junk, that provided the most independent nonverbal signal.
Although language remains the strongest channel, the silence between words betrays doubt more accurately than intonation. Researchers also noticed that attempts to account for contradictory signals — for example, a smile with a sad voice — only confused the system. Simple threshold adjustment and breaking the signal into components allowed bypassing the standard model, accurately detecting doubt even on unfamiliar data. This approach will be useful in support services and emotional state assessment.
About 40% of spontaneous speech consists of pauses and fillers, which speech systems discard, losing valuable clues about uncertainty.