information theory

Write down a recipe, then check it an hour later: if the proportions are off by even a gram, the taste is gone. Chemical reactions in open systems behave much the same way: the slightest difference in initial concentrations grows over time into completely different states.

An architect reinforces the foundation but leaves a door open for remodeling: that's how living systems balance reliability against renewal. A cell copies hereditary information with a fidelity any scribe would envy, yet it can also allow changes that help it survive under new conditions.

A piston in a cylinder with gas usually moves when the gas pushes on it. But there is a piston that moves when the space in front of it is empty. Every five seconds, it checks the space near its wall, and if no one is there, it takes a step inward, compressing the gas without direct mechanical force.

A fussy thermometer shows different temperatures depending on who is looking at it. A black hole behaves much the same way: its thermal properties, entropy and temperature, depend on how we describe the information hidden inside.

A bacterium swims toward food. A chemical signal guides it forward, but with each movement the signal changes. A loop emerges: sensation, action, new sensation. The authors turned this loop into a measurable quantity.

An ordinary road map with traffic is a heap of numbers. Sending them all as is takes too long. But you can reduce the volume without losing the gist. Researchers found a clever solution: first redraw the map so that streets become points, and intersections become bridges between these points.

In 2019, the world saw the first image of a black hole's shadow — an orange doughnut on a black background, becoming a symbol of scientific progress. But this image wasn't captured by a camera: it was born from mathematical alchemy, where data from eight radio telescopes, which gathered only crumbs of signal, merged into a single picture.

A search party combs a foggy field. At first, people spread out wide: a high 'temperature' of search means a great variety of attempts. Upon finding a trace, they close in. But the speed of convergence is critical — narrow the circle too early, and the best find stays aside. The coordinator assesses the spread of searchers and commands when to speed up or slow down.

Imagine an experienced chef during rush hour: suddenly a stove fails or a key ingredient runs out. Instead of panicking, he instantly reworks the recipes — speeding up some dishes, slowing down others, reallocating resources on the fly. This principle of dynamic adaptation underpins a new generation of ultra-reliable networks.

Code-writing assistants often behave like a chef who, to boil a single egg, rereads the whole cookbook. When asked to make a tiny fix, they reopen dozens of files and retrace long-familiar connections. This is unnecessary work — energy going to waste.

The standard way of digitizing speech: taking volume readings thousands of times per second. It's like tasting broth with a teaspoon: each sip is watery, with almost no detail. The data stream swells, though the substance is minuscule. Scientists proposed first "simmering down" the audio signal.

The orchestra plays without a conductor. We know the overall volume: that's the basic reproduction number R0, the spectral radius of the epidemic. But the score itself, the contact matrix between people of different ages, holds the melody: who spreads the infection and who takes the main blow.

A scientific paper is not just a collection of results: it's an imprint of research handwriting. Some scholars trace strict bibliometric lines, others add experimental flourishes, and still others inscribe content-analytic curlicues. Over 30 years, three top journals of library and information science accumulated 5281 such 'signatures' from 81 countries. In 1990, the average paper was a single-country autograph.

In the world of logic, there is a bridge. It spans the chasm between a rule's admissibility and its derivability from axioms. As long as rules involve only a finite number of premises, the bridge holds firm — for classical logic, anything that doesn't violate truth is already contained in the syntax. But add infinite premises, and the supports start to tremble. The bridge, alas, is not all-powerful.

We're used to thinking that noticing only confirmations of our guesses is a mistake. But recent calculations, inspired by wave behavior, flip that view. It turns out that the habit of seeking the familiar is the best way to reduce information noise (scientists call it entropy).

Life is a constant dance between order and chaos. Back in the mid-20th century, Erwin Schrödinger noted that living things feed not so much on matter as on "negative entropy"—the ability to maintain their own order by increasing disorder around them.

Erasing even a single bit means releasing heat. This discovery by Rolf Landauer hinted that information is neither material nor illusory. Imagine that every event casts a shadow: that is information, the collection of all unrealized possibilities.

Creating an alloy from several metals is like cooking a soup from mysterious ingredients: guessing the final texture without trial and error is nearly impossible. Conventional models often get it wrong when there are more than three different kinds of atoms.

Entropy is a measure of disorder. Two shuffled decks of cards, dumped into one pile, won’t become more chaotic than the sum of their individual disorders. This is the rule of subadditivity: the whole is no more chaotic than its parts.

Two simultaneous interpreters sit in the same booth. The first translates from English into Russian but swallows half the words. The second goes from Russian into French, and also loses half. If you put them together, you'd expect a mess: the losses would add up, and only a quarter of the meaning would reach the listener. But what if they start correcting each other?

A voice recorder captures a city square. You hear the hum of a crowd, footsteps, snatches of conversation, the rumble of traffic. Somewhere in that sonic stew, a street musician plays a simple tune on a flute. Hearing it isn't easy: there's just too much mixed together.

In 1948, engineer Claude Shannon laid the foundations of information theory. The Nyquist–Shannon theorem gave the first answer: to avoid losing a signal, you need at least twice as many measurements as the fastest feature in the signal. In 2006, the method of compressed sensing relaxed this requirement.

Spread a handful of coins on the table. Each coin is a possible outcome: heads, tails, or something else. If we know how chaotic this set is (its entropy), which combinations of probabilities will we encounter more often? It turns out the answer depends on how much chaos we allow.

A thermostat keeps a room at 20 degrees. Open a window in winter, and it kicks on the heat to restore equilibrium. The brain does something similar: it constantly predicts what will happen a moment ahead and corrects itself when wrong. That's how it learns.

If you write down a protein recipe on a piece of paper and send it to a print shop, a typo in a single letter can ruin the whole batch.

A broken guitar string doesn't just go silent. It still vibrates for a while, and by the character of that vibration, a luthier can tell what kind of string it was — thick or thin, tightly wound or loose. In much the same way, physicists listen to the "vibration" of quantum systems when they want to understand their structure.

A magnetic film divided into tiny arrow-shaped cells behaves like a crowd in a square. While it's cold, all arrows point in the same direction — perfect order. Heat the film, and the arrows start darting around like pedestrians who have lost their purpose. Somewhere between these extremes, there is a moment when the system stops being uniform but hasn't yet dissolved into random noise.

Erasing a single bit of information isn't free. In 1961, Rolf Landauer showed that at room temperature, it takes at least about 3 zeptojoules of energy — a tiny but nonzero amount. This limit was considered as ironclad as the conservation of energy. But what if you could give the system a little nudge?

Imagine an office where employees exchange notes. When there are few tasks, everyone works independently. But when an urgent order arrives, some departments start actively messaging others, and the overall work becomes more coordinated. That's roughly how the brain behaves when we try to hold several things in mind at once.

A translator who jumps at every rustle in the hall turns a one-hour meeting into two. It's the same in epidemic control: contact tracing teams try to reconstruct the chain of infections, but something always slips through. Someone forgets who they met, someone doesn't answer the phone, someone never even comes to the attention of health workers.

According to the laws of physics, nothing can outrun light, otherwise for some observers the effect would occur before the cause. Artur Ekert and Andrzej Dragan showed that this situation creates an uncertainty resembling quantum indeterminacy.

In 1989, John Wheeler proposed a bold idea: "it from bit," meaning everything that exists arises from information. For a long time, this remained a beautiful metaphor, but now physicists have found a concrete mechanism that turns bits into particles.

When entropy reaches its maximum, the play is over. The curtain falls, the actors leave, the scenery freezes. But the stage remains. So it is with time: the heat death of the universe erases the direction 'past—future', but not time itself.

An orchestra without a conductor plays a complex piece. The listener hears the overall music but wants to understand who leads the melody and who merely echoes it in unison.

A flashlight beam in fog reaches some objects and fades before touching others. So it is with El Niño forecasts: for events in the central Pacific, the light reaches almost two years ahead, but for eastern events it gets lost in random wind gusts within six months.

Winning the lottery changes the winner's life, but for the statistics of all players it means almost nothing: the probability of winning is negligible, and the average winnings per ticket remain tiny.

In a climate trial, the judge doesn't listen to who shouts loudest. He watches who whispers to whom. Nine witnesses, from carbon dioxide to cosmic rays, testified from 1984 to 2005. Their statements were recorded every month. The judge doesn't trust mere coincidences, because two witnesses might echo each other without knowing who started first.