How to fish out a friend's voice from a noisy conversation express
A human effortlessly picks out the desired voice from the clamor of a party. For a computer, the same task is still at the edge of its abilities. Previous programs trained on sterile recordings, free of echo or the clinking of appliances. But a real conversation is like a turbulent stream: speakers interrupt, move away from the microphone, and in the background a coffee maker hisses.
The new REAL-TSE Challenge casts a line into these murky waters for the first time. Participants are given a set of spectrograms — kind of sound photographs — recorded in a lively environment. From just a couple of seconds of a voice, the algorithm must fish out that person's speech from the polyphonic mess. Naturalness is also evaluated: will the voice turn into a screech? Two tracks: online — like a stream, and offline — with analysis of the entire recording.
The method is based on decomposing sound into simple waves — a discovery by Fourier, now two centuries old. Modern systems, like fishermen armed with the knowledge of how to untangle knots of frequencies, grapple with high entropy — a measure of disorder in the sound mixture. And although the perfect catch is still far off, it's precisely such 'field' trials that push the technology out of quiet labs and into our noisy world.
The brain performs this trick without practice: the 'cocktail party effect' lets us pick out a familiar voice from dozens of strangers — without the slightest strain.