concept

neural network memory architecture

Structural and geometric organization of memory and information storage in neural networks.
47 related articles

Description

Neural network memory architecture refers to the structural and geometric organization of how information is stored and retrieved within artificial neural networks. Unlike traditional computer memory with separate storage and processing units, neural networks embed memories directly in their connection weights and activation patterns. The architecture determines how much information can be stored, how quickly it can be accessed, and how resistant it is to interference or forgetting.

History

The concept of distributed memory in neural networks was formalized by David Rumelhart and Geoffrey Hinton in the 1980s, building on earlier work by Teuvo Kohonen on associative memory. The long short-term memory (LSTM) architecture, introduced by Hochreiter and Schmidhuber in 1997, addressed the vanishing gradient problem and enabled longer-term memory storage.

How it works

In feedforward networks, memories are distributed across the weights connecting layers, with each weight contributing to many different memories—a form of content-addressable memory. Recurrent networks add temporal dynamics, allowing information to persist in hidden states over time. More recent architectures like transformers use attention mechanisms to create dynamic memory pathways, where the network can selectively focus on relevant parts of the input sequence. The geometric organization of these representations in high-dimensional space is crucial: similar concepts cluster together, and the network's ability to separate distinct memories depends on the dimensionality and topology of this space.

In practice

Memory architectures are critical for applications requiring sequential processing, such as speech recognition, machine translation, and time-series prediction, as well as for continual learning systems that must adapt without forgetting.

Fun fact

The human brain's memory capacity is estimated at around 2.5 petabytes, equivalent to about 3 million hours of television shows, while a state-of-the-art transformer model with 175 billion parameters stores roughly 350 gigabytes of information.

Concept graph →

Related articles · 47

Genomic Niches of the Aging Brain
The brain is like a city where every cell is a resident, and its fate depends on its neighbors. It used to be thought that the aging of each cell was its own private business: DNA damage accumulated, so the cell wore out on its own. But a new study shows that even a cell's ability to repair its DNA depends on who lives next door.
Layer Memory: How MXene Remembers Being Flipped Through
A stack of pages that has been flipped through many times remembers it: some sheets lie flat, others stick up, and creases remain here and there. Layered materials behave much the same way when ions enter them.
Triple Connections: How the Brain Links Regions in Threes
A jazz trio plays in the dim light of a club. The pianist leads the melody, the bassist holds the rhythm, and the drummer sprinkles out rolls. If you listen only to pairs of musicians, you can catch how the pianist adjusts to the bass or the bass to the drums.
COFITAGE: a portrait of the aging brain in five dimensions
In 2023, in Belgium, 101 people aged 50 to 69 spent several days in a lab where their brains were scanned, their sleep was recorded, and they were asked to solve memory and attention tasks. That is how the COFITAGE dataset was born, describing each participant from five angles: molecular, structural, physiological, genetic and cognitive.
q-bio.BM2026-08-191 minarXiv:2608.18604express
How Protein Engineers Learn from Mistakes
A protein engineer is like a chef trying to make a dish that is tasty, healthy, and cheap all at once. Change one ingredient, and you risk ruining another. In the past, computer assistants could only evaluate protein properties one at a time, missing how they are connected.
Physics and Society2026-08-142 minarXiv:2608.14398express
When being together is a choice
Two people agree to meet, but each has their own things to do. One hesitates: what if the other doesn't come? And decides not to go. The other thinks the same. The meeting doesn't happen. That's the simple mechanism the authors dissect: a joint endeavor requires everyone to invest time, but the payoff depends on whether the partner participates. In the model, each person keeps a to-do list ordered by importance.
q-bio.SC2026-08-022 minarXiv:2608.00995express
Mail Inside a Neuron: How Molecules Find Their Recipients
In the long extensions of a neuron, the dendrites, tiny couriers are constantly bustling about. These are messenger RNA molecules carrying instructions for protein assembly. Their journey starts at the cell body and ends at synapses — contacts with other neurons.
q-bio.BM2026-07-202 minarXiv:2607.17753express
Scout proteins find genes by their energy trail
A protein slides along the DNA strand like a key along a keyring, searching for its lock. One wrong turn and a gene stays switched off. To know where a protein will grip firmly, you need to measure the strength of that grip — the binding energy.
A living organ in a Petri dish reveals loops
A city at night: lights on in some windows, dark in others. If you connect with lines the windows that light up simultaneously, you get a web of connections. Researchers studied such a web in tiny brain organoids — living clusters of nerve cells grown in the lab.
Signal Processing2026-07-152 minarXiv:2607.13710express
Cooking Without Delays: How Networks Achieve Ultra-Reliability
Imagine an experienced chef during rush hour: suddenly a stove fails or a key ingredient runs out. Instead of panicking, he instantly reworks the recipes — speeding up some dishes, slowing down others, reallocating resources on the fly. This principle of dynamic adaptation underpins a new generation of ultra-reliable networks.
Memory Without Thoughts: How the Brain Remembers After Silence
After a stone is thrown, the circles on water vanish, but the liquid retains microscopic oscillations. The slightest gust, and ripples appear again. Neural networks behave similarly. When electrical signals quiet down, the memory of them remains in the contact points — synapses.
Robotics2026-07-141 minarXiv:2607.12861express
Invisible Map of a Robot Swarm
A robot swarm is like a cloud of fireflies: each moves on instinct to wherever is freer, and chaos turns into order. For a long time, it was a mystery how coherence emerges without communication or commands. The answer came with the creation of the Response Map — a tool that, like an X-ray, illuminates the hidden logic of the machine brain.
Optics2026-07-112 minarXiv:2607.10295express
AI Finds a Recipe for Light Circuits
Building a light chip is like a game where you have to guide a beam through a maze of tiny mirrors and beam splitters. Each element acts like a railroad switch: it either lets light pass straight through or directs it to an adjacent track. The goal is to produce a specific light pattern at the output while using as few 'switches' as possible. Usually, engineers spend years finding the shortest route for each task.
Weight Geometry: The Conductor of Memory in Complex Systems
Why do some systems, from gene networks to air travel, preserve imprints of the past longer than others? For a long time, network science blamed topology: the number of connections, modularity, spectrum. But two networks with the same skeleton can remember and forget in completely different ways.
Machine Learning2026-06-061 minarXiv:2606.08287express
A neural network that senses stress in parts of any shape
Designing machines is like cooking: you need to know exactly where the 'dough' will tear. Old computer simulations check every point on a part for hours, like measuring the entire pot to make sure the handle doesn't fall off.
Medical Physics2026-06-051 minarXiv:2606.07851express
How to See Capillaries in a Living Brain
Photographing a delicate spiderweb trembling in the wind: that's roughly what it takes to look into the blood vessels of a living brain. Ultrasound localization microscopy (ULM) turns ordinary ultrasound into a tool capable of seeing individual capillaries, but only if you get the settings just right.
The Brain Saves Memory Like a Cashier at Rush Hour
A supermarket cashier at rush hour doesn't memorize every scanned item: they keep the running total in their head and watch the queue. The brain allocates its resources similarly: the expensive, precise cortex (responsible for deliberation) and the simpler, economical subcortical nuclei (where habits live) work in tandem.
Mesoscale2026-05-301 minarXiv:2606.00711express
Memory for a quantum computer at minus 271 degrees
An ordinary field-effect transistor is like a faucet with a very sensitive handle: turn it slightly, and water flows. In the cold depths of a quantum computer, where everything is chilled to near absolute zero, such a faucet must open strictly on command.
When the Mind Honestly Says 'I Don't Know'
A person looks at a door lock and sees a keyhole. They know that a key is inserted and turned. But if instead of a keyhole there's a touch panel, the familiar frame breaks. The person doesn't just make a mistake: they realize that the usual approach won't work and can say, "There's no place for a key here; a different approach is needed." Such an honest stop with an explanation is rare for machines.
Memory Lies Not in Cell Count but in Cell Order
A person hunting for keys mentally runs through places where they might be. A bird that tucked hundreds of seeds into crevices in autumn retrieves them flawlessly in winter. How can a brain hold so many addresses when all birds have roughly the same number of neurons? The answer lies in geometry.
Biological Physics2026-05-012 minarXiv:2605.00813express
Word Mosaic: How Physics Explains the Game of Scrabble
After a game of Scrabble, the board looks like a mosaic: tiles are scattered but form a connected pattern — islands of words in a sea of empty cells. Such a pattern can be described mathematically, but usually that requires complex rules.
How the Brain Balances on the Edge
The brain is like a multi-story building: lower floors process simple signals, while upper floors assemble them into complex perceptions. Scientists have long suspected that this order is tied to how neurons work together. But exactly how has remained a puzzle.
Medical Physics2026-04-162 minarXiv:2604.15478express
Tissue as a Maze: Why FLASH Radiation Acts Unevenly
Imagine a city where streets are not straight but winding, with dead ends and hidden passages. A bike courier will take much longer to deliver a package than in a city with a regular grid of roads. Similarly, molecules inside living tissue: their path depends on how "tangled" the tissue structure is. Scientists built a model that describes the movement of molecules in such a "tangled" environment.
q-bio.PE2026-04-121 minarXiv:2604.10571express
Evolution by the Same Rules
Engineers improving neural networks act like breeders: they try a variant, check the result, and keep the best. They collected 935 such trials from 161 papers and measured how often a code tweak turns out beneficial, harmful, or neutral. The picture is familiar to any biologist: 68% of changes degrade performance, 19% do nothing, and only 13% yield a gain.
Materials2026-04-061 minarXiv:2604.04388express
Memory That Glows
An ordinary flash drive stores bits in electron traps, but you can only read them through a separate wire and amplifier. In the new development, the same piece of the semimetal tellurium plays both the role of storage and the light bulb.
Medical Physics2026-03-271 minarXiv:2603.27003express
Heart by Contour: How AI Builds 3D Shape
The heart is not a perfect sphere; its shape changes with every beat. To build an accurate 3D copy for a computer model, doctors typically take dozens of MRI slices. But that's like assembling a watermelon from thin slices: the fewer slices, the more guesswork. A new approach teaches a neural network to understand the heart's shape from sparse contours.
Why Neurons Can't Be Friends with Everyone
A neuron in the brain is like a person handing out business cards at a crowded conference. At first, making acquaintances is easy, but as the address book fills up, each new contact requires more effort: there's little room left. The authors based their model on this 'crowding' rule: the more incoming connections a neuron already has, the harder it is to form another one.
Disordered Systems2026-03-151 minarXiv:2603.14534express
A Powder That Remembers Motion
A handful of sand on a table is just a pile. But if you shake the grains just right, the pile starts behaving like a memory: it remembers which motion follows which. In the new experiment, particles were moved cyclically, and after training, they reproduced the desired pattern even without a full command. The secret lies in the contacts between particles.
Physics and Society2026-03-071 minarXiv:2603.07114express
Brain Clocks Don't Depend on Network Size
Deep in the brain lies a tiny region called the suprachiasmatic nucleus. It's the conductor that sets the rhythm for the whole body: when to sleep, when to wake, when to release hormones. Scientists took the real connection map of this region from a mouse and made copies — one bigger, one smaller.
Medical Physics2026-03-042 minarXiv:2603.03832express
A Heart in Your Hand: Seeing Arrhythmia Without Surgery
The heart doesn't contract all at once but as a wave: an electrical signal travels through the muscle, and behind it, like ripples spreading on water, a contraction rolls through. In arrhythmia, this wave goes awry: it stalls in one spot, loops in another, and the pump starts working in a disorganized way.
The Neuron City: Why Some Brains Think Fast and Others Slow
A mouse brain contains about 70 million neurons, and the connections between them are far from evenly distributed: some cells have only a couple of dozen contacts, while others have thousands. It's like a city with quiet alleys and bustling squares. But until now, it was unclear how this unevenness affects the brain's actual function.
Applied Physics2026-03-012 minarXiv:2603.00995express
Antenna Grows in Volume
Two lanterns on a distant shore at night merge into a single blob of light. Likewise, a conventional flat antenna cannot tell apart two radio transmitters placed too close together: their signals mix, and the receiver sees only a jumbled mess. Engineers found a way out: they made the antenna not flat but like a stack of transparent plates.
Machine Learning2026-03-011 minarXiv:2603.02273express
Alzheimer's Genes: A Fresh Look Through Attention
Alzheimer's disease is like a tangled ball of yarn: thousands of genes are intertwined, and it's hard to tell which ones snap first. Previously, scientists looked for important genes by simply counting how many connections each node in the network had. But that approach is like judging a person by the number of acquaintances they have — the most sociable isn't always the most influential.
Machine Learning2026-02-282 minarXiv:2603.00786express
How the Brain Rewires Itself in Alzheimer's Disease
The brain isn't one big computer: it's more like an orchestra, where different groups of musicians play their parts. Neural networks for memory, attention, or emotions are constantly talking to each other. In Alzheimer's disease, this orchestra starts to go off-key: some groups fall silent, others play out of sync.
The Brain's Wiring: How White Matter Orchestrates Thought
The brain is like a sprawling city: districts-regions communicate via streets-neural pathways. If traffic lights at an intersection fail, traffic grinds to a halt. Similarly, in the brain, when bundles of nerve fibers (white matter) malfunction, distant regions stop working in concert, and thought stalls.
Disordered Systems2026-02-201 minarXiv:2602.18335express
Chaos in a Crystal Learns to Think
A crystal that inside resembles a sponge with a labyrinth of pores turns out to be capable of performing computations. The material, named DUT-8, consists of metal nodes and organic linkers forming an intricate lattice. Under the influence of different gases, this lattice slightly changes shape, like a house of cards that rearranges with a gentle breath.
How a Neural Network Learns to Speak Like a Human
A language model is a multi-layered structure where each layer processes words in its own way. At the start of training, layers are chaotic, but gradually they organize into two teams. The first team works simply: its internal geometry smooths out, becoming less tangled. The second remains complex and intricate.
How the Brain Passes Thoughts
Imagine an office where employees exchange notes. When there are few tasks, everyone works independently. But when an urgent order arrives, some departments start actively messaging others, and the overall work becomes more coordinated. That's roughly how the brain behaves when we try to hold several things in mind at once.
q-bio.MN2026-01-311 minarXiv:2602.00586express
Teaching a Neural Network to Read Textbooks
Proteins in a cell communicate like people in a bustling city: some pass signals, others put on the brakes, and still others just hang around. A graph neural network sees the map of these connections, who's linked to whom, but to it, they're just lines.
Sleep as Brain Cleaning: Why Memory Gets Stronger at Night
Over the course of a day, the brain accumulates impressions the way a room accumulates stuff. At night, during deep sleep, it doesn't just rest: it tidies up: it sorts through the day's events and decides what to keep and what to toss. Scientists built a computer model of the brain that learns from a few examples during the day and 'replays' them at night, as if dreaming.