Reinforcement Learning

Collective behavior of AI agents has emerged not in a laboratory but in the wild: on a randomly discovered wiki site where agents wrote test hints. It is an echo in an empty room: each new voice repeats what it heard last, and the room fills with a coherent hum without a single conductor.

A piston in a cylinder with gas usually moves when the gas pushes on it. But there is a piston that moves when the space in front of it is empty. Every five seconds, it checks the space near its wall, and if no one is there, it takes a step inward, compressing the gas without direct mechanical force.

Picture a flock of birds gathering around a feeder that no one built. Each bird searches for food on its own, yet they all end up together. Usually, such behavior is explained by social instinct: birds want to be near their kin. But what if the flock is simply a side effect of greed?

Picture a flock of birds gathering around a feeder that no one built. Each bird searches for food on its own, yet they all end up together. Usually, such behavior is explained by social instinct: birds want to be near their kin. But what if the flock is simply a side effect of greed?

A banknote taped to the finish line makes your hand fly faster. In the lab, that banknote was replaced with a 50-pence coin, and the finish line with a target on a screen. The researchers recruited 28 young people and 28 older adults, seated them at a robot handle, and asked them to steer a cursor toward the target. Sometimes they were paid for speed and accuracy, sometimes not.

In the ICU, the decision of how much fluid to give a sepsis patient and what dose of a vasopressor to administer is made every hour. A mistake in either direction can be fatal. But how do you know which decision would have been best? You could take thousands of patient records and ask a program to "replay" them, trying different actions. That's exactly what the authors did.

A lone fish sees a shadow overhead. Should it flee? If it's just a leaf, it wastes energy and gives away its position to a predator. If it's a heron, hesitation is fatal. That's the eternal trade-off of the solitary individual: quick reactions mean many false alarms, while caution means risking being eaten. In a school, it's different.

Hot water sometimes freezes faster than warm water: this odd fact is known as the Mpemba effect. Now something similar has been found in the quantum world: a state with weak entanglement (a special quantum connection between particles) can beat a more entangled one to the finish line.

Give your back a mild electric zap, and your leg starts to live its own life. That's exactly what researchers discovered when they tested how transcutaneous spinal cord stimulation affects healthy people. Fourteen volunteers tried to move their ankles precisely while a robotic trainer measured errors. It turned out that after brief stimulation, the sense of foot position worsens — errors increase.

Holding in mind the exact position of a pen as it's slowly moved is a routine task for the brain. For artificial neural networks, it's nearly impossible. Standard recurrent networks like GRU and LSTM tend to break continuous motion into rigid steps: the pen doesn't glide smoothly but jumps from point to point.

A neuron receives thousands of signals on its branching processes — the dendrites. Some signals push the cell toward firing (excitation), others hold it back (inhibition). For a long time, it was thought that the neuron simply adds up all the 'fors' and 'againsts.' But in real dendrites, inhibition works differently: it can not subtract but divide the excitatory signal.

A tour with a single robot is a tedious monologue with no one to exchange glances with. Researchers turned that one-man show into a lively stage: they added a hologram sidekick to a real robot guide. This tandem chats, argues, jokes, drawing visitors into conversation instead of just spouting facts.

Two gardeners haggle over water from a magical well. One proposes how many buckets to take, the other can agree or refuse. If the deal goes through, the water decreases; if not, the well gradually refills. This haggling repeats hundreds of times, and the gardeners' habits sometimes change randomly, as if they're trying new tactics.

A twitch of the robot's camera and it drops the part: its neural network is thrown off. But if you peek at its signals, they resemble constellations in a galaxy: when things go smoothly, they cluster in one sector, and just before an error, they abruptly shift to another. The boundary between them is as clear as a safe channel through reefs.

The robot juggler catches and throws balls in precise rhythm with a human, like two пульсара sending pulses in unison. Juggling three balls is a classic cascade, governed by a steady beat, like the uniform rotation of a cosmic lighthouse. Break the rhythm and the pattern crumbles.

When an unmanned aerial vehicle learns from an experienced pilot, it memorizes not individual maneuvers but entire safe corridors — the trajectories along which the human mentor moved. This approach is called imitation learning: the machine mimics expert demonstrations.

In calm water, the route is clear. But in turbulent flows, a robot needs a different strategy. Typical approaches demand an accurate map of currents, but in reality, you can't get one. Nature does it differently: fish feel pressure and vortices right against their bodies.

Chatting with living neurons is like tuning an old radio: it's not enough to find the right frequency: you've got to aim the antenna too.

Traditionally, machine learning algorithms operate in a static environment, like water flowing along a pre-cut channel without changing its shape. The new method flips this paradigm: it lets an agent simultaneously master actions and deliberately modify the very environment it inhabits.

Cooking a complex dish requires a clear recipe: first chop, then sauté, add spices at just the right moment. Chemical transformations also follow step-by-step recipes — mechanisms where atoms of carbon and hydrogen rearrange in a strict order. Previously, neural networks often invented nonexistent steps, like a whimsical cook swapping salt for sugar.

Power systems with solar panels and wind turbines are like a chess game: cloud cover and load changes are the opponent's moves, impossible to foresee. To maintain balance, operators used to run through dozens of scenarios (uncertainty) — that took too much time and computing power.

A robot swarm is like a cloud of fireflies: each moves on instinct to wherever is freer, and chaos turns into order. For a long time, it was a mystery how coherence emerges without communication or commands. The answer came with the creation of the Response Map — a tool that, like an X-ray, illuminates the hidden logic of the machine brain.

A driver on a slippery road doesn't know exactly how the car will respond to the steering wheel, and learns on the go, leaving room for error. A smart control algorithm behaves in a similar way.

When you type 'salad' and the keyboard suggests 'Caesar,' behind the scenes millions of phones are jointly learning to predict words. They share not your messages, but only improvements to the common algorithm, like a coach who sends athletes only the tweaks to the plan: 'add 5 push-ups to yesterday’s routine.' Information entropy helps send only unexpected, valuable updates.

A wolf pack switches from chasing prey to defending its den in an instant. Engineers have instilled this same versatility into an algorithm for a robot swarm. The key is a method proposed by John von Neumann: instead of chasing the target, the robots calculate its future moves and block escape routes, like beaters cutting off trails.

The railway overhead contact line resembles a galaxy: energy flows through it like gravity, with trains becoming planets. The more trains draw current simultaneously, the stronger the voltage fluctuations — gravitational waves ripple through the system, threatening blackouts.

Imagine a fleet of many ships, resembling a galaxy — a vast stellar system where every element follows the overall dynamics. From time to time, one ship breaks down, exploding like a supernova — a catastrophic event that can disrupt the integrity of the entire formation. If the damaged unit isn't replaced in time, the fleet loses combat capability.

Complex machinery, from drones to surgical robots, sometimes behaves like a wild mustang: full of disorder and whims, where the slightest touch triggers a storm. Instead of rote memorization of formulas, the new method observes the system like a horse breaker.

When a glass of water is full, pouring more is pointless waste. Cloud gaming often does the same with video data: encoders push the stream to maximum, even though the eye stops noticing improvements long before the limit. This overconsumption means extra traffic and entropy in the network — useless noise clogging the channel.

A runner loses on both a smooth track and sticky sand. Seems hopeless. But if the surface underfoot starts changing every few seconds (track, then sand) the runner suddenly stops falling behind. He doesn't get faster; the environment just doesn't get a chance to 'crush' his weakness. That's exactly what the authors found when studying a simple predator–prey model.

In embryonic development, a cell resembles a musician in a vast orchestra: it must play its part, following a conductor — the gradient of signaling molecules. But the conductor is unreliable: morphogen concentrations constantly fluctuate. How then does a cellist not veer into a violin melody?

Recommendation systems that decide which post you’ll see next run on deep neural networks made up of thousands of operations — intricate computation Graphs (blueprints of operations on data). These systems are living, ever-evolving Dynamical Systems: the slightest architecture change cascades through the entire graph.

A tiny dust particle in a supersaturated solution sprouts into a large crystal in seconds. Neighboring particles remain invisible. A similar process was recently discovered in communities of language agents.

The brain doesn't learn every task from scratch: it transfers experience from one activity to another. But how do you know which tasks are truly related? Neuroscientists came up with a clever trick: let an artificial network "peek" at brain activity during one task and try to predict activity during another. If it works well, the tasks are close.

We're used to thinking that noticing only confirmations of our guesses is a mistake. But recent calculations, inspired by wave behavior, flip that view. It turns out that the habit of seeking the familiar is the best way to reduce information noise (scientists call it entropy).

A person reaches out to a stranger across town. That stranger will never return the favor: they'll never meet. So why help? Biologists call this indirect reciprocity: kindness comes back not from the person you helped, but from someone else, down the line. In a new study, two variants of this idea were pitted against each other.

In the watery environment, vortices batter the mechanical fish. The AI agent, like an experimental chef, ran simulations, tried out 'recipes' of movements, and improved the program. Each experiment was like spectroscopy: the agent broke down behavior into elements, uncovering patterns. At first, the fish only paddled in one direction.

Understanding the photoelectric effect — where light knocks electrons out of a material — is like learning to cook without a recipe. That’s where digital tools help: games provide intuition, and simulations provide precision.

A supermarket cashier at rush hour doesn't memorize every scanned item: they keep the running total in their head and watch the queue. The brain allocates its resources similarly: the expensive, precise cortex (responsible for deliberation) and the simpler, economical subcortical nuclei (where habits live) work in tandem.

A photographer shoots a night forest, trying to catch fireflies in the frame. The flashes are rare and faint, while around there's the noise of wind and distant lights.