math

Markov chain

A Markov chain is a stochastic process where future states depend only on the present state, not on past events.
24 related articles

Description

A Markov chain is a stochastic model describing a sequence of states in which the probability of transitioning to the next state depends solely on the current state (Markov property). Formally: P(X_{n+1}=x | X_n=x_n,...,X_0=x_0) = P(X_{n+1}=x | X_n=x_n). Chains can be discrete or continuous in time, with a finite or countable number of states. Main tools: transition probability matrix, stationary distribution, Chapman–Kolmogorov equations.

History

Andrey Andreyevich Markov introduced chains in 1906 while studying the law of large numbers for dependent quantities. He was motivated by a desire to refute Nekrasov's claim that independence is necessary for statistical laws. Later, in the 1930s, Kolmogorov developed the general theory of Markov processes. In the mid-20th century, Markov chains became a central tool in operations research, physics (Ising model), and computer science (text generation, PageRank).

How it works

Mathematically, a Markov chain is defined by an initial probability vector π^{(0)} and a stochastic matrix P. Then the distribution after n steps: π^{(n)} = π^{(0)} P^n. The memoryless property means that to predict the future, it is sufficient to know only the current state. The long-term behavior is determined by the spectral decomposition of the matrix P: if there exists a unique left eigenvector with eigenvalue 1 (the stationary distribution), then π^{(n)} → π as n→∞. Chapman–Kolmogorov equations: P^{(n+m)} = P^{(n)} P^{(m)}, expressing the composition of transitions. For continuous time: dπ/dt = πQ, where Q is the transition intensity matrix.

In practice

Markov chains model queues in stores, search engine algorithms (Google's PageRank), random walks of stock prices, and text generation by artificial intelligence.

Fun fact

The first practical application of Markov chains was the analysis of the alternation of vowels and consonants in "Eugene Onegin" — Markov showed that it is not random and predicted future letters of the text.

Concept graph →

Related articles · 34

Chaos in Chemistry: How Information Loss Reveals Kolmogorov-Sinai Entropy
Write down a recipe, then check it an hour later: if the proportions are off by even a gram, the taste is gone. Chemical reactions in open systems behave much the same way: the slightest difference in initial concentrations grows over time into completely different states.
Physics and Society2026-08-142 minarXiv:2608.14398express
When being together is a choice
Two people agree to meet, but each has their own things to do. One hesitates: what if the other doesn't come? And decides not to go. The other thinks the same. The meeting doesn't happen. That's the simple mechanism the authors dissect: a joint endeavor requires everyone to invest time, but the payoff depends on whether the partner participates. In the model, each person keeps a to-do list ordered by importance.
Physics and Society2026-08-082 minarXiv:2608.08343express
How a Disease Retraces Its Own Steps
A mail carrier delivers letters along one-way streets. As long as he only drives forward, the route is simple and predictable. But if U-turns are suddenly allowed, he'll start doubling back, crossing his own tracks, and the delivery pattern gets tangled. That's roughly how an infection behaves when people can get sick more than once.
A Bias That Needs No Teacher
In a warehouse, there are boxes of apples and pears. There are many apples and few pears. But when the stock clerk says "a fruit with smooth skin," we are somehow more likely to recall the rare pear than the common apple. This is the inverse base-rate effect — a persistent glitch in how the brain resolves ambiguity between categories.
Exoplanets2026-08-051 minarXiv:2608.05054express
Martian Weather, Earthly Playbook
Meteorologists have grown used to trusting neural networks: the GraphCast model predicts Earth's weather days ahead by drawing on global snapshots of the atmosphere. But this 'brain' can be transplanted to another planet — where the air is thin and the Sun warms differently.
Biological Physics2026-08-042 minarXiv:2608.03000express
How the Brain Switches Between Sleep Modes
Every night, the brain cycles through several sleep stages — from drowsiness to deep sleep and back. Some transitions happen abruptly, like a light switch flipping, while others are smooth, like a slow slide down a slope.
Applied Statistics2026-08-032 minarXiv:2608.02896express
Thoughts and Actions: What Teenagers' Transitions Say About Risk
Each year is a new page. From ages 9 to 13, a child's life changes rapidly, and so do their inner experiences. A team of researchers took data from the large ABCD Study survey, in which 11,864 adolescents reported over several consecutive years on their thoughts and actions — whether they had thoughts of suicide, whether they engaged in actions, and whether they harmed themselves without such thoughts.
Applied Statistics2026-08-032 minarXiv:2608.02896express
Thoughts and Actions: What Teenagers' Transitions Say About Risk
Each year is a new page. From ages 9 to 13, a child's life changes rapidly, and so do their inner experiences. A team of researchers took data from the large ABCD Study survey, in which 11,864 adolescents reported over several consecutive years on their thoughts and actions — whether they had thoughts of suicide, whether they engaged in actions, and whether they harmed themselves without such thoughts.
q-bio.SC2026-08-022 minarXiv:2608.00995express
Mail Inside a Neuron: How Molecules Find Their Recipients
In the long extensions of a neuron, the dendrites, tiny couriers are constantly bustling about. These are messenger RNA molecules carrying instructions for protein assembly. Their journey starts at the cell body and ends at synapses — contacts with other neurons.
math.NT2026-08-012 minarXiv:2608.00615express
Numbers Hide Patterns
In 1998, mathematician David Boyd noticed something strange. He was studying the Mahler measure of polynomials, a quantity that gauges the "complexity" of their roots, and found that for a family of curves, it always equals a special rational number times the derivative of an L-function.
q-bio.PE2026-07-292 minarXiv:2607.27496express
The Backbone of Food Webs: How to Save Ecosystems in a Changing Climate
A web where lions eat zebras and hyenas finish the leftovers is not just a list of dinners. It's a survival map for an entire region. When the climate shifts, animals move, and the map tears. To understand where it will tear and where it will hold, you need to find its unchanging 'skeleton'—the connections that repeat across different places.
Physics and Society2026-07-262 minarXiv:2607.23457express
How Van Gogh's Paintings Gave Away His Approaching Breakdown
In 1888, Vincent van Gogh cut off his earlobe. Biographers and psychiatrists have been trying to explain this act ever since. But mathematicians approached the mystery differently: they measured how the artist's feelings changed in his letters and how the 'nervousness' of his brushstrokes grew, and they saw a clear signal of impending trouble. A swing that gets pushed usually swings and quickly calms down.
Mesoscale2026-07-262 minarXiv:2607.23498express
Knock, knock, who's there? Majorana
An interpreter gets distracted by every rustle in the hall, and a one-hour meeting stretches into two. That's roughly how scientists have spent years searching for special states in tiny electrical chains: they watched the total current but missed how the signal was corrupted by noise.
Statistical Mech2026-07-192 minarXiv:2607.17072express
Chemistry Under the Metronome
A kitchen mixer runs at a single speed, but start jabbing the button and the mixture whips up differently. Chemical reactions behave much the same way when the rates of their individual steps are not constant but pulse over time.
q-bio.QM2026-07-181 minarXiv:2607.17000express
Lab Results Tell a Story: A New Approach to Predicting Kidney Failure
A car's speedometer shows the speed right now, but to know whether the car will reach its destination in a year, a single glance at the gauge isn't enough. That's roughly how conventional formulas for kidney failure risk work: they rely on one blood test and miss how values have been shifting over months and years.
How the Brain Conducts Attention: A Meditation Model
The meditating brain is an orchestra, and attention is the conductor. When you're focused on your breath, all instruments play in harmony. But suddenly, a thought-cello plays a false note: that's mind wandering. An experienced conductor doesn't panic: they instantly catch the dissonance and gently guide the orchestra back to the main melody — the breath.
eess.SY2026-07-151 minarXiv:2607.13513express
Smart Shower: Taming an Unpredictable System
You're standing in the shower: you turn the tap, and the water is either freezing or scalding. The pipes have a mind of their own — the pressure jumps unpredictably. The new controller learns on the go: it doesn't know the plumbing layout, but within minutes it adapts and gently guides the temperature to comfort, without breaching safety limits.
eess.SY2026-07-141 minarXiv:2607.12343express
How a Smart Algorithm Learns Not to Crash
A driver on a slippery road doesn't know exactly how the car will respond to the steering wheel, and learns on the go, leaving room for error. A smart control algorithm behaves in a similar way.
q-bio.PE2026-07-131 minarXiv:2607.11520express
How Chance Drives Evolution
Evolution flows like water. In a wide river, changes are smooth and slow, while a mountain stream churns with every stone. Population size and mutation rate set the channel: broad and calm, or narrow and rapid-filled.
Physics and Society2026-06-122 minarXiv:2606.14818express
A Pedestrian Who Looks One Step Ahead
A crowd at a metro station during rush hour. Some hurry toward the escalator, others slow down at the departure board, still others weave around a column. Each step is not merely a reaction to what's right in front of your nose. A person estimates in advance where a neighbor will be a second from now and adjusts their route accordingly. Physicists call this forward-looking behavior anticipatory.
Quantum Physics2026-06-071 minarXiv:2606.08595express
Time Flows Through Quantum Clocks
Time in physics is like a conductor setting the tempo for the whole orchestra. But in the quantum world, there's no single baton: each particle can live by its own clock. Usually, we simplify the picture and assume time flows the same for everyone.
Statistical Mech2026-05-262 minarXiv:2605.27635express
How to Measure Chaos in a System Without Knowing Its Rules
A busy intersection without traffic lights. Cars drive, brake, and change lanes, and from the outside, it looks like complete chaos. But if you watch for a while, you can notice a rhythm: the flow thickens and thins. In physics, this rhythm of disorder is called entropy: it shows how far a system is from calm equilibrium, like water in a boiling kettle.
Chemical Physics2026-05-242 minarXiv:2605.24895express
NMR Radar for Invisible Molecules
Many chemical transformations resemble assembling furniture in complete darkness: you see the starting parts and the finished cabinet, but the assembly itself remains a mystery. The intermediate states, like awkward positions of the craftsman's hands, vanish too quickly to be discerned.
Quantum Physics2026-05-151 minarXiv:2605.16148express
How Internal Noise Destroys Quantum Magic
In a football stadium, the crowd's emotions arise from the chatter of individual fans. If mood could be both joyful and sorrowful at once, you'd have a quantum superposition. But the murmur of thousands of voices instantly forces that blend into a single definite state.
How the Brain Learns to Look Ahead
A melody plays, and the brain doesn't just listen note by note. It works like a conductor who already knows the score: each next note is expected because the previous ones still echo in memory. But how do neurons hold on to that echo?
Peeking Inside the Mind of a Weather AI
A forecaster issues a week-long forecast in seconds but cannot explain a single decision. That is roughly how modern AI weather models work: they already match classical numerical methods, yet their internal logic is hidden behind millions of parameters.
Weather Forecast: Sharp Focus
The atmosphere is a chaotic system, and a detailed global forecast requires monstrous computing power. StretchCast sidesteps this by working like the human eye: it creates 'sharp vision' over a selected region with an ultra-fine grid, while seeing the rest of the world blurry, but sharp enough to capture global connections.
Fluid Dynamics2026-03-141 minarXiv:2603.13996express
How Flow in a Reactor Splits into Islands
When you stir sugar into a glass, the water seems to move every which way. But inside a chemical reactor with an impeller, the liquid isn't just churning: it forms hidden "islands" that barely exchange their contents. Scientists found a way to see these islands by following the paths of tiny particles.
Math Physics2026-02-221 minarXiv:2602.19120express
Order of Operations Changes Quantum Memory
Two people read the same book, but one turns the page first and then looks at the text, while the other does the opposite. It would seem the result is the same. But in the quantum world, the order of operations changes everything. This work examines models that output sequences of symbols, like a ticket machine.
physics.data-an2026-02-211 minarXiv:2602.18949express
A Forecast That Remembers the Cycle
A solar panel on a roof produces little current in the morning, a lot at noon, and stays silent at night. Wind doesn't blow randomly either: it's often stronger in the morning and evening than during the day. Energy providers need to know in advance how much energy will be available in an hour or three, so they can switch on backup plants or store electricity in batteries in time.
Statistical Mech2026-01-111 minarXiv:2601.07096express
A Search Where the Team Changes on the Fly
A group of people searches for a lost item in a large park. Some get tired and leave, others join along the way. The number of searchers is constantly changing. How does this affect the search time? Until now, science had no clear answer. In a new study, a general model of such a process has been built. In it, independent participants randomly enter and leave the search.
Death and Rebirth of Quantum Echo: A Hierarchy of Temporal Correlations in a Qubit
Strike a gong and the sound fades in layers: the low hum lingers longer, the high overtones vanish first, leaving silence behind. So too a qubit, living its quantum life, leaves not one but three echoes of its own past.
stat.ME2023-11-193 minarXiv:2311.11445
Hidden Ocean Regimes: How to Catch Non-Integer Delays
The ocean doesn't always stick to the same schedule. Sometimes the warm El Niño current strengthens, sometimes it weakens, and each regime dictates its own reaction speed to the past. These hidden regimes are like shifts in currents that remember their path with different depths.
Reservoir with a Correction: How to Predict Noisy Chaos
A translator who learns to correct their own slips works more accurately. In the world of forecasting noisy processes, a new method takes on this role: a reservoir computer that adds a random correction to its prediction at each step.