cs.SD cs.SD

In September 2024, the concert hall in Linz had more than just an orchestra on stage. Next to the musicians, a setup was generating pairs of entangled photons: a laser with a wavelength of 405 nm passed through a nonlinear crystal and split into two photons at 810 nm, which flew off to two measurement stations, Alice and Bob.

An ornithologist in the jungle hears a trill, but the bird is hidden in the foliage. It would be great to get its portrait right away — at least from a photo database. Usually, that requires thousands of pairs of “song recording + photo of the same bird,” and collecting them is expensive and time-consuming. The authors took a detour: they used two ready-made models.

The bronze plates of a gamelan do not sound according to a single tuning standard: each orchestra on Java keeps its own tuning, and two neighboring ensembles may play the same melody in slightly different intervals.

A teacher in class explains the difference between 'dog' and 'tog': put your hand on your throat and feel the vibration. A speech recognition machine does much the same thing, except inside it's not a throat but long lists of numbers. It turns out these numbers can be sorted onto shelves, like alphabet blocks in a child's room.

The gut talks to the doctor, but very quietly. Gurgles and rumbles last fractions of a second and easily get lost among the sounds of the lungs and heart. A stethoscope catches only a fraction, so the diagnosis depends on the experience and hearing of the individual doctor.

A large neural network like Whisper masters hundreds of languages and resembles the standard model of the universe: everything is balanced. But training on a rare language disrupts this equilibrium — it turns into a black hole, pulling in all training resources. The network starts forgetting the other languages.

A sound signal is conveniently represented as a complex number: its modulus (amplitude) reflects loudness, while its argument (phase) indicates the wave's position in time: its instantaneous shift.

Models that turn text into audio are good at recreating single sounds, but get confused when the script has several in a row. For example, the script might call for a ring, then footsteps, then a door slam. The generator might shuffle events, so the listener hears footsteps before the ring.

A conductor holds not a baton but a brush, and each stroke gives birth to color and form. New technology works the same way: it transforms a song's mood into a panoramic video that surrounds the listener. First, the algorithm decomposes the track into emotions, similar to how light is analyzed by spectrum.

Automatic speech judges are algorithms that evaluate pronunciation and emotions. However, they often work like a food critic who delivers a verdict after reading the menu but not tasting the dish.

A human effortlessly picks out the desired voice from the clamor of a party. For a computer, the same task is still at the edge of its abilities. Previous programs trained on sterile recordings, free of echo or the clinking of appliances.

An AI predicting room acoustics is like a student taking an exam with a cheat sheet. If the test points are the same as the training ones, the model simply memorizes the acoustic 'fingerprint' of each position — like a student recognizing a question by the dog-eared corner.

A magnetoencephalograph, a helmet-like device with hundreds of sensors, picks up the weak magnetic fields generated by active neurons. When a person listens to speech, these signals contain imprints of the heard words. But extracting them is tricky: signals from thousands of sources merge into a tangled mess.