method

Large Language Models

Large language models are neural network methods that learn to generate and predict human-like text from vast datasets.
111 related articles

Description

Modern large language models are neural networks with a 'transformer' architecture, featuring anywhere from hundreds of millions to hundreds of billions of parameters. Each parameter is a number that influences how the signal from one word is passed to another. The model doesn't store words but converts them into vectors of numbers—mathematical points in a high-dimensional space of meanings. Pre-training is carried out on an autocomplete task: the model fills in missing words or predicts the next word in a sequence. Afterwards, it can be fine-tuned on specialized examples to hold a dialogue, solve problems, or write code.

History

The history began with statistical n-gram models (1950s), then neural network language models (Bengio et al., 2000s). The turning point was the paper Attention Is All You Need (2017), which proposed the transformer architecture. In 2018, OpenAI released GPT-1 with 117 million parameters, and in 2020, GPT-3 with 175 billion parameters demonstrated that a model without explicit training could translate, program, and reason simply by receiving instructions in natural language. Since then, the race has focused on both scaling up models and improving efficiency (Mixture-of-Experts, Chinchilla scaling laws).

How it works

The model receives a context window of n tokens as input. Each token is mapped to an embedding and encoded by self-attention layers, creating contextualized representations. At the output, a linear classifier predicts the probability distribution of the next token. Training minimizes the cross-entropy between the prediction and the true token. During generation, text is produced by sequential sampling of tokens (usually with top-p or top-k nucleus sampling). Key techniques: pre-training on unlabeled text, then instruction-based or RLHF fine-tuning to align with usefulness and safety. The attention mechanism computes similarity weights between queries and keys for each pair of tokens, allowing it to focus on long-range dependencies.

In practice

Such models write news, compose poems, generate programming code, answer customer questions in chats, and help doctors draft discharge summaries from medical histories.

Fun fact

GPT-3 can write an essay indistinguishable from a human-written one, as judged by a committee, in 2 seconds—the time it takes a person just to open a text editor.

Concept graph →

Related articles · 142

Epidemics as Dialogue: How Language Models Teach Epidemiologists to Listen to Human Behavior
In March 2020, France emptied out: people stopped going to work, school, and cafés, and the streets filled with silence. Epidemiologists stared at the graphs and understood that the virus was not spreading on its own, but through handshakes, conversations, and metro rides. But how could they predict how many contacts a person would keep if the government announced new restrictions?
Epidemics as Dialogue: How Language Models Teach Epidemiologists to Listen to Human Behavior
In March 2020, France emptied out: people stopped going to work, school, and cafés, and the streets filled with silence. Epidemiologists stared at the graphs and understood that the virus was not spreading on its own, but through handshakes, conversations, and metro rides. But how could they predict how many contacts a person would keep if the government announced new restrictions?
Chemical Physics2026-08-211 minarXiv:2608.21322express
Counting Protein Jitters in Seconds
A protein in a cell doesn't hang motionless like a mannequin in a store window. Its atoms are constantly jiggling, stretching, and bending, like a plucked guitar string that keeps vibrating after you strum it. To understand how a protein works, you need to know how tightly each part holds onto its neighbors.
Monroe: A Molecular Model with In-Context Probabilistic Inference
An interpreter gets distracted by every rustle in the hall, and a one-hour meeting stretches into two. Similarly, molecular foundation models learn the language of molecules from vast corpora, but each new conversation requires fine-tuning. Monroe changes the rules: it doesn't learn a new language but instantly finds common meaning from a few examples.
Machine Learning2026-08-171 minarXiv:2608.16273express
A Digital Twin for an Entire Country's Health
A medical record is like a long diary: doctor visits, diagnoses, prescriptions, lab results. A human doctor reads it cover to cover to understand what's going on with the patient. Now, a machine reads this diary, and does it in seconds. The developers created a model called Foresight-E. It was trained on de-identified records of about 61 million people: that's nearly the entire population of England.
Materials2026-08-161 minarXiv:2608.15928express
AI hunts for materials like a head chef
Hunting for a new material is like cooking without a recipe: you try, try again, and often throw things out. Previously, scientists manually flipped through papers and tested compositions one by one. Now there's a "robot chef" that reads the literature itself, mixes components, and checks what came out.
Galaxies2026-08-091 minarXiv:2608.08410express
Five Thousand Giants on the Radio Sky
A radio telescope, like a sensitive ear, picks up the whispers of distant galaxies. Hidden among that noise are giants — giant radio sources. These are clouds of charged particles ejected from the vicinity of supermassive black holes.
Physics and Society2026-08-021 minarXiv:2608.00939express
AI Freezes in the Heat
In an oven, as temperature rises, molecules move faster and disorder increases. With language models like ChatGPT, it's the opposite. When developers increase the 'temperature' parameter, the model gets access to a wider choice of words, but its long texts suddenly start looping. A team of researchers tested 11 different AIs by generating 12,000 text continuations.
Machine Learning2026-08-022 minarXiv:2608.01481express
Reading Thoughts from Magnetic Fields
A magnetoencephalograph, a helmet-like device with hundreds of sensors, picks up the weak magnetic fields generated by active neurons. When a person listens to speech, these signals contain imprints of the heard words. But extracting them is tricky: signals from thousands of sources merge into a tangled mess.
How Ideas Learn from Each Other
The human mind is like a construction set that builds itself. New ideas don't die with their creators: they are passed on, mixed, and selected. This process is called cultural evolution: useful inventions stick, useless ones are forgotten. That's how a ratchet works — a mechanism that only turns forward. But what exactly happens in the head when one thought displaces another?
Galaxies2026-07-242 minarXiv:2607.22812
Galaxies2026-07-242 minarXiv:2607.22835
Archipelago in the Ocean of Data: Red Points Without Color Boundaries
Two hundred forty-two thousand three hundred twenty-seven galaxies — like grains of sand on a beach. But astronomers turned this chaos into a map: the UMAP algorithm (a machine learning method) took from each galaxy eleven parameters — seven colors from optical to mid-infrared (that's photometry), magnitude…
AI Learns to Create Viruses
By 2025, large language models like GPT are no longer just chatting: they're helping scientists run experiments. But the same tool could fall into the wrong hands. A team of researchers built a digital proving ground to find out.
Stellar2026-07-193 minarXiv:2607.17348
SpecFANN: Lightning-Fast Sketch of a Star's Spectrum
An artist makes a quick charcoal sketch, and the face in the portrait is already recognizable. Astrophysicists face a similar challenge: the spectrum of a star is its individual portrait, revealing temperature, composition, and gravity.
q-bio.GN2026-07-192 minarXiv:2607.17227express
Cellular Atlases: Who Reads the Language of Tissue Best
Biologists are increasingly trusting computers to read the "user manuals" of living cells. Six recent AI models (Nicheformer, CellPLM, scGPT-spatial, GenePT, scELMo, and Novae) are like six translators working from different dialects of the same cellular language.
q-bio.PE2026-07-161 minarXiv:2607.14653express
Why Genes Mess Up the Family Tree
Imagine a cookbook that gets rewritten from generation to generation. One day, the borscht recipe receives a secret ingredient from a pastry chef, and from that moment, the taste remembers not only ancestors but also a neighbor. In the Standard Model of evolution, genes behave the same way.
Signal Processing2026-07-161 minarXiv:2607.14938express
Where Are You? Radio Waves Know the Answer
Radio signals in a room act just like sound in a concert hall: bouncing off walls and objects, they create a unique pattern — a silent melody for every location. Your phone, like a keen ear, catches this melody amid noise and distortion. It remembers which chord matches the corner of the couch, and which one fits the doorway.
Machine Learning2026-07-161 minarXiv:2607.14995express
How to Teach AI Not to Confuse Similar MRI Scans
Ordinary AI looks at medical images like puzzles made of unfamiliar pieces: it considers any two unmatched pictures completely different. It's like trying to tell twins apart without knowing they're related. In medicine, however, patients with the same condition look nearly identical on MRI, and the algorithm mistakes 'relatives'.
eess.AS2026-07-161 minarXiv:2607.15198express
How to fish out a friend's voice from a noisy conversation
A human effortlessly picks out the desired voice from the clamor of a party. For a computer, the same task is still at the edge of its abilities. Previous programs trained on sterile recordings, free of echo or the clinking of appliances.
eess.AS2026-07-151 minarXiv:2607.13408express
How AI Stopped Mixing Up Sounds in a Script
Models that turn text into audio are good at recreating single sounds, but get confused when the script has several in a row. For example, the script might call for a ring, then footsteps, then a door slam. The generator might shuffle events, so the listener hears footsteps before the ring.
cs.SD2026-07-151 minarXiv:2607.13477express
AI Judges Listen Not with Ears, but with Labels
Automatic speech judges are algorithms that evaluate pronunciation and emotions. However, they often work like a food critic who delivers a verdict after reading the menu but not tasting the dish.
eess.SY2026-07-151 minarXiv:2607.14273express
How to Track Heat Across an Entire Room with Just One Thermometer
In a soup pot, you only need two thermometers to gauge the temperature everywhere — if you know heat travels in circles from the bottom. Science works the same way: from a handful of measurements, you want to reconstruct the full heat picture. The system captures the gist of heat flow, compressing it into a handful of governing numbers.
Machine Learning2026-07-151 minarXiv:2607.14086express
Brain Signals: Learning to Understand Without a Teacher
The brain communicates in short electrical clicks. Previously, to learn this language, thousands of examples with translations from the experimenter were needed. MOJO takes a different path: like learning a language through immersion, it first listens to the 'stream of consciousness' without any hints and finds recurring patterns on its own.
Machine Learning2026-07-151 minarXiv:2607.14410express
LATTICE: How to Unify Cellular Data
For a long time, biologists studied cells by listening to just one voice in a polyphonic choir: measuring gene activity while ignoring how DNA is packed. It's like trying to grasp a conversation knowing only one language. LATTICE fixes this: the program learns five molecular 'languages' at once — from gene activity to spatial markers.
Computer Vision2026-07-141 minarXiv:2607.12375express
AI with a magnifying glass and a spectrometer: image quality assessment
Assessing image quality is a job for a meticulous inspector with a toolbox. The IQA-T1 neural network acts exactly like that: it doesn't just stare at the picture, but applies either a spectroscope or a light meter.
Machine Learning2026-07-142 minarXiv:2607.12417express
A Neural Network That Perceives Phase: From Hearing to Quantum Computing
A sound signal is conveniently represented as a complex number: its modulus (amplitude) reflects loudness, while its argument (phase) indicates the wave's position in time: its instantaneous shift.
Machine Learning2026-07-141 minarXiv:2607.12771express
AI Masters the Cookbook of Chemical Reactions
Cooking a complex dish requires a clear recipe: first chop, then sauté, add spices at just the right moment. Chemical transformations also follow step-by-step recipes — mechanisms where atoms of carbon and hydrogen rearrange in a strict order. Previously, neural networks often invented nonexistent steps, like a whimsical cook swapping salt for sugar.
cs.ET2026-07-141 minarXiv:2607.12862express
Choosing a Recipe for a Neurochip: How to Best Fix Interference
Neurochips work differently from conventional computers, which are built according to the blueprints of von Neumann with separate memory and processor. At the core of these chips is a network of simple cells, similar to neurons, and, like water that evenly fills containers of any shape, they easily find a solution without unnecessary steps.
How AI Quit Overreading
Code-writing assistants often behave like a chef who, to boil a single egg, rereads the whole cookbook. When asked to make a tiny fix, they reopen dozens of files and retrace long-familiar connections. This is unnecessary work — energy going to waste.
eess.AS2026-07-141 minarXiv:2607.13330express
How a Sound-Generating Neural Network Slimmed Down
A neural network that generates sounds from text is like a kitchen where dozens of chefs prepare each 'dish.' Many just get in the way — sending false signals and wasting resources.
Signal Processing2026-07-141 minarXiv:2607.13114express
AI completes the signal from noise: ultra-light communication for 6G
The transmitter sends only a rough sketch of the message — like an artist drawing contours without details. At the receiving end, the AI, like an experienced restorer, reconstructs the full picture, cleaning it of interference. This is how LiTCom technology works, changing the game in future 6G networks.
Language Amnesia in Neural Networks: A Solution Exists!
A large neural network like Whisper masters hundreds of languages and resembles the standard model of the universe: everything is balanced. But training on a rare language disrupts this equilibrium — it turns into a black hole, pulling in all training resources. The network starts forgetting the other languages.
eess.AS2026-07-131 minarXiv:2607.11260express
Sound Concentrate: Transmitting Speech Without Redundant Bits
The standard way of digitizing speech: taking volume readings thousands of times per second. It's like tasting broth with a teaspoon: each sip is watery, with almost no detail. The data stream swells, though the substance is minuscule. Scientists proposed first "simmering down" the audio signal.
math.CO2026-07-132 minarXiv:2607.11325express
The Evolutionary Tangle: How to Measure Chaos in the Web of Life?
A family tree becomes a tangle if you account for adopted children and remarriages. Evolution often resembles such a tangle: although the Standard Model, the traditional view of divergent evolution where species branch like a tree, paints a neat tree of species, in reality genes travel not only by inheritance but also directly between unrelated organisms — a process…
eess.SY2026-07-131 minarXiv:2607.11459express
The Power Industry Opens Data for AI
A living power system is like an organism: power lines are vessels, current is blood. Data, however, are nerve impulses that used to be lost. mAIEnergy connects these 'nerves' to a shared 'brain'. This data repository combines satellite images, electricity meter readings, articles, and weather reports.
eess.SY2026-07-131 minarXiv:2607.11551express
The Future File: Predicting Motion Without a Single Formula
An experienced detective doesn't calculate a criminal's path using the laws of physics: he checks a file of similar cases. The new prediction method works the same way: instead of complex equations, it has a vast archive of trajectories that have already happened — planetary motions, currency exchange rates, signals from space.
Computer Vision2026-07-131 minarXiv:2607.11120express
Doubt is Given Away by Silence: AI Reads the Pauses
A new system analyzes a short interview to recognize human doubt by combining text, voice, and facial expressions. But the main key to uncertainty turned out to be the dark matter of speech — pauses and filler words that standard speech systems erase like a black hole.
cs.SE2026-07-112 minarXiv:2607.10347express
Quantum System Confuses Blueprints, but Engineers Don't
Quantum devices are no longer lab curiosities. They are assembled, tuned, and repaired like ordinary hardware — except inside them, it's not just current but intricate tangles of particles that can exist in multiple states at once. Engineers draw schematics, programmers write code, physicists calculate effects.
Cosmology2026-07-083 minarXiv:2607.06959
Cosmic Tuning Fork: KLT-Net Neural Network Rewrites the Distance Ladder
Modern cosmology has hit a dissonant chord. Local measurements of the Hubble constant from supernovae and Cepheids give around 73 km/s/Mpc, while global constraints from the cosmic microwave background within the standard ΛCDM model point to 67 km/s/Mpc.