benchmark

In the 1980s, the DARPA speech recognition challenge settled the debate between statistical methods and hand-written rules, setting a common standard for the entire field.

At night on a volcano summit, six telescopes stare at the same speck of light. Each catches its own copy of the wave, and if you combine their signals, you get an image that no single instrument could produce alone. But there's a catch: light captured by different stations cannot simply be added like photographs.

A protein engineer is like a chef trying to make a dish that is tasty, healthy, and cheap all at once. Change one ingredient, and you risk ruining another. In the past, computer assistants could only evaluate protein properties one at a time, missing how they are connected.

Biologists are increasingly trusting computers to read the "user manuals" of living cells. Six recent AI models (Nicheformer, CellPLM, scGPT-spatial, GenePT, scELMo, and Novae) are like six translators working from different dialects of the same cellular language.

Reconstructing a full picture from fragments is like discerning an orchestra's melody in a noisy hall. Such challenges arise in photometry, where a star's brightness is recovered from faint twinkling. Traditional methods need precise noise information, which is almost always unknown.

A human effortlessly picks out the desired voice from the clamor of a party. For a computer, the same task is still at the edge of its abilities. Previous programs trained on sterile recordings, free of echo or the clinking of appliances.

Models that turn text into audio are good at recreating single sounds, but get confused when the script has several in a row. For example, the script might call for a ring, then footsteps, then a door slam. The generator might shuffle events, so the listener hears footsteps before the ring.

Automatic speech judges are algorithms that evaluate pronunciation and emotions. However, they often work like a food critic who delivers a verdict after reading the menu but not tasting the dish.

In boiling soup, molecules collide chaotically and react. But if you measure how their concentrations change, patterns emerge. This is how carbon chemical networks work — mixtures of molecules that spontaneously 'cook' information, turning each reaction into an elementary computation. Previously they were used only for modeling nature.

At the heart of every neutron star lies a symphony — an intricate interplay of quantum fields and relativistic gravity.

The brain communicates in short electrical clicks. Previously, to learn this language, thousands of examples with translations from the experimenter were needed. MOJO takes a different path: like learning a language through immersion, it first listens to the 'stream of consciousness' without any hints and finds recurring patterns on its own.

Assessing image quality is a job for a meticulous inspector with a toolbox. The IQA-T1 neural network acts exactly like that: it doesn't just stare at the picture, but applies either a spectroscope or a light meter.

Human hearing is wired so that the brain prioritizes familiar patterns — a friend's voice, the patter of rain, a car horn. But a new system works on a different principle: it registers any sound burst, even one never encountered before.

A cook who learns to cook from hand-drawn pictures risks ruining every dish. The same trap awaits computer vision: models need thousands of labeled images, but collecting them manually is expensive and tedious. Synthetic frames solve the problem, but only if they look real.

Code-writing assistants often behave like a chef who, to boil a single egg, rereads the whole cookbook. When asked to make a tiny fix, they reopen dozens of files and retrace long-familiar connections. This is unnecessary work — energy going to waste.

The float in a toilet tank is an ideal controller: sensing the water level, it instantly brings the system into equilibrium without looking back at the past. Engineers applied this principle to neural networks.

Evolution flows like water. In a wide river, changes are smooth and slow, while a mountain stream churns with every stone. Population size and mutation rate set the channel: broad and calm, or narrow and rapid-filled.

An electron microscope is a giant pipe organ: instead of pipes (electromagnetic lenses, instead of air) a stream of accelerated electrons. It once merely painted mute frescoes of the atomic world, condemning scientists to grueling decipherment.

Decades of radio astronomy have listened to space, hoping to catch narrowband radio transmissions — the technogenic 'voice' of distant exoplanets. But our own technological evolution reveals a paradox: loud analog signals, like 1960s TV broadcasts, are just a brief squawk of a newborn civilization.

A quantum computer is like a finicky musical instrument: the slightest change in temperature, humidity, or string tuning, and the melody sounds different.

A child builds a simple game in Scratch: a character appears in the center and instantly jumps to a corner. The first move happens between frames — no one sees it. Removing that command is safe. But how does an automatic optimization optimizer know that? Today, the Scratch repository holds tens of millions of projects written by young authors.

Eight hundred participants, firms and workers, enter the market. Each holds a strict preference list in mind. The challenge: pair up so that no one wants a do-over. In mathematics, this is called a game theory with a ban on repeat alliances, and its roots trace back to the work of John von Neumann and Oskar Morgenstern.

Every week, doctors ask AI assistants millions of questions at the bedside. But these tools are usually tested on exam-style problems, not on what clinicians actually ask in practice. A team of researchers set out to fix that. They collected 620 real questions from physicians across 30 specialties — from cardiology to oncology.

Studying distant exoplanets, scientists look for signs of life — for instance, water and carbon — and signs of technology: radio signals, megastructures. But between these clues there is a yawning gap. If intelligence hasn't created machines or they've vanished, we won't notice it.

Air flowing over a wing or turbine blade behaves like a string: it starts smooth and layered: that's the laminar regime. But as speed increases or external ripples appear, the string breaks into chaotic vibration — turbulence. Predicting the exact moment of breakdown is crucial: it affects drag, heating, and noise.

An engineer tunes an antenna: pull one segment, the frequency drifts; change the angle, the gain drops. Developing therapeutic peptides (short protein chains) works much the same way.

You buy a new kitchen scale, but instead of exact weight, it only shows deviations from some "norm." To find out how much an apple weighs, you first have to weigh dozens of other apples and calculate the average.

A lab sends a robot a list of a thousand candidate molecules for a new drug. The predictor robot, trained on old data, gives each one a 'fit or no fit' score. But when the list comes from an unfamiliar area, a different virus or a rare tumor, the robot starts to get confused. Its confidence stays high, while actual hits drop to zero.

Engineers are building a car. Some assemble an exact replica of the engine, every piston and spark plug, but the car doesn't move: there's no steering wheel or wheels. Others build a fast self-driving go-kart, but its engine is made of playdough and doesn't resemble a real one. This is roughly the situation in brain science.

You're trying a dish at a restaurant, but you haven't seen the menu. You ask the waiter questions — too salty? what spices?—and from the answers you try to guess what kind of chef prepared it.

The brain is like a multi-story building: lower floors process simple signals, while upper floors assemble them into complex perceptions. Scientists have long suspected that this order is tied to how neurons work together. But exactly how has remained a puzzle.

Your hand rests on the table, and two short vibrations arrive at your wrist — one after the other. You need to say which is stronger. The task seems simple, but the answer depends on which vibration came first. If the first is weak and the second is strong, the difference feels larger than when they are swapped.

Astronomers occasionally spot interstellar objects — rocky chunks or comets from other star systems. Most look ordinary, but some puzzle us with their shape, speed, or shine. To stop guessing, scientists applied the Loeb scale: from 0 (plain rock) to 5 (technosignature — an artificial signal from space).

A lab assistant carefully follows every step of an experiment, but at the end, writes in the report what they wanted to write before they even started. That's how modern language models behave when given scientific research tasks.

A city map: intersections are nodes, streets are connections. For an algorithm to understand the structure, each node needs a label. The classical approach — standard model — assigns simple tags, like house numbers.

A drop of blood from a fingertip is like a photograph. One is just a snapshot, but collect them over years and you get a movie of your health.

The best AI assistant for scientists scored worse on this exam than a C student on a pop quiz. Just 21% correct answers, and that's the ceiling for the four strongest models. The problems for the new test were written not by programmers but by graduate students: people who deal with real science every day. The test is called COMPOSITE-STEM.

A translator who has never been in a meeting translates literally, and loses half the meaning. They don't know that the boss's nod means "do it like last time," and the phrase "well, we'll see" is a polite refusal. That's how a language model that writes code from a scientific paper works: it can read the equations, but not the unspoken agreements.

When chemists calculate the behavior of molecules on a computer, they use approximate methods because exact calculations are too complex. For light atoms, there are well-established benchmarks, but for heavy atoms, where relativistic effects matter, such benchmarks did not exist. It's like checking a ruler against another ruler rather than against a standard meter.

The atmosphere is a chaotic system, and a detailed global forecast requires monstrous computing power. StretchCast sidesteps this by working like the human eye: it creates 'sharp vision' over a selected region with an ultra-fine grid, while seeing the rest of the world blurry, but sharp enough to capture global connections.