The Network Finds Its Own Balance, No Thermometer Needed ✛

Original: "Adaptive self-organized criticality in deep neural networks"
· Simon Vock, Christian Meisel
arXiv:2608.28431v1 · 2026-08-28 · CC BY 4.0 · 2 min
Neurons adjust their connections based on their own activity, and the whole network settles at the edge between dying out and exploding.

A chain of people passes a message in a whisper. If each person speaks too quietly, the message fades halfway down the line. If too loudly, by the end it turns into a cacophony. But there is a mode where the whisper reaches the last person without distortion. Scientists call it the critical point, and the brain seems to operate right there.

The critical point is like the freezing temperature of water. A bit colder and you get ice, a bit warmer and you get liquid. A network at this point neither explodes nor fades out.

Previously, it was thought that the only way to bring a network to a critical state was from the outside, by tuning global parameters. But in the new work, each neuron gets a simple rule: if it is too active, weaken the incoming connection; if too passive, strengthen it. It is as if each person in the chain decides on their own whether to speak louder or softer, based only on their own voice.

The scientists tested this on a network with nine layers of a thousand neurons each. They ran it from different starting states, ranging from near silence to strong noise. After twenty cycles of self-tuning, all networks arrived at the same regime: activity neither decayed nor grew, and connection strengths leveled off. The network found the critical point by itself, just as water finds its own level.

But when digit recognition training was added to the self-tuning, competition began. Ordinary training pulls the network toward solving the task, while self-tuning pulls it toward criticality. These two forces interfered with each other: the stronger the self-regulating rule, the worse the network recognized digits. So the thermostat without a thermometer works, but it does not play well with the task.

The homeostasis rule is so simple it can be stated in one line: if a neuron is too active, weaken its incoming connections; if too passive, strengthen them. And that is enough for the network to find the critical point on its own.

\sigma = \frac{a_{out}}{a_{in}}
σ=1 means activity is preserved as it propagates through the network
\lambda_0 = \frac{1}{L} \log(\Lambda_0)
λ0=0 corresponds to the critical point, where perturbations neither grow nor decay
w'_{ij} = \begin{cases} (|w_{ij}| - c) \cdot \operatorname{sgn}(w_{ij}) & \text{если } a_i > T \\ (|w_{ij}| + c) \cdot \operatorname{sgn}(w_{ij}) & \text{если } a_i < T \end{cases}
An active neuron weakens its incoming connection, an inactive one strengthens it
Scientists
Emile Clapeyron
Concepts
deep neural network neural networks in spatial mapping stochastic self organization Dynamical Systems ai self correction Generative Adversarial Networks deep learning in medical imaging random network learning biological stochastic dynamics molecular dynamics acceleration information storage real time control self-supervised learning Post-transcriptional regulation Natural Language Processing predictive learning neural networks stochastic dynamical system cellular automaton