Holding in mind the exact position of a pen as it's slowly moved is a routine task for the brain. For artificial neural networks, it's nearly impossible. Standard recurrent networks like GRU and LSTM tend to break continuous motion into rigid steps: the pen doesn't glide smoothly but jumps from point to point. The network seems to remember not a line but a few discrete frames.

A group of researchers proposed a network with division — the Recurrent Divisive Normalization Network (RDNN). Its secret lies not in subtracting inhibitory signals but in dividing by them. When one neuron is active, it doesn't just silence its neighbor; it proportionally reduces the neighbor's output. This is like adjusting volume: in a noisy room, we don't subtract noise from speech; we divide: we speak louder relative to the background. This mechanism is widespread in the cerebral cortex but rarely used in artificial networks.

Mathematical analysis showed that division creates a slow manifold in the network — a smooth path along which the memory state can change without jumps. When the network is trained with backpropagation, division automatically scales the learning steps: in highly active regions, parameter updates are dampened. This compresses the effective rank of the network — the dynamics naturally squeeze into a narrow, low-dimensional channel, avoiding overfitting and chaos.

In experiments on working memory tasks, RDNN held continuous signals tens of times more accurately than standard networks. And when division was replaced with subtraction, the network immediately began to fragment the smooth signal into steps. It seems biology chose this mechanism for a reason — division turned out to be mathematically necessary for storing continuous quantities in a changing world.