How to fish out a friend's voice from a noisy conversation express
A human effortlessly picks out the desired voice from the clamor of a party. For a computer, the same task is still at the edge of its abilities. Previous programs trained on sterile recordings, free of echo or the clinking of appliances. But a real conversation is like a turbulent stream: speakers interrupt, move away from the microphone, and in the background a coffee maker hisses.
The new REAL-TSE Challenge casts a line into these murky waters for the first time. Participants are given a set of spectrograms — kind of sound photographs — recorded in a lively environment. From just a couple of seconds of a voice, the algorithm must fish out that person's speech from the polyphonic mess. Naturalness is also evaluated: will the voice turn into a screech? Two tracks: online — like a stream, and offline — with analysis of the entire recording.
The method is based on decomposing sound into simple waves — a discovery by Fourier, now two centuries old. Modern systems, like fishermen armed with the knowledge of how to untangle knots of frequencies, grapple with high entropy — a measure of disorder in the sound mixture. And although the perfect catch is still far off, it's precisely such 'field' trials that push the technology out of quiet labs and into our noisy world.
Original abstract
as written by the authors arXiv:2607.15198
We introduce the REAL-TSE Challenge, an IEEE SLT 2026 satellite challenge on target speaker extraction~(TSE) from real conversational recordings. Given a multi-speaker mixture and one or more enrollment utterances from a target speaker, participating systems must recover only the target speech. Unlike simulated read-speech benchmarks, REAL-TSE evaluates Mandarin and English recordings that contain natural overlap, reverberation, noise, channel mismatch, and conversational dynamics. The challenge defines two complementary tracks: an Online track for low-latency streaming extraction and an Offline track for full-context processing. Systems are evaluated with Token Error Rate (TER), Speaker Similarity (SpkSim), DNSMOS, and target-speaker activity F1. This overview paper describes the task definition, datasets, baselines, evaluation protocol, submitted systems, condition-wise findings, and lessons for future real-world TSE benchmarks.
The brain performs this trick without practice: the 'cocktail party effect' lets us pick out a familiar voice from dozens of strangers — without the slightest strain.