concept

Explainable AI

Explainable AI is a subfield of artificial intelligence focused on making the decisions and inner workings of AI models understandable to humans.
45 related articles

Description

Explainable Artificial Intelligence (XAI) is a field that makes the operation of machine learning models transparent to humans. Unlike traditional neural networks — 'black boxes', XAI provides an interpretation: why the model came to a particular conclusion, which features were important, and how reliable the prediction is. Approaches are divided into interpretable models (decision trees, linear models) and post-hoc explanations for already trained complex models (LIME, SHAP, gradient maps). This is critical in medicine, finance, and law, where an error can cost lives or freedom.

History

The black box problem intensified with the deep learning renaissance in 2012. Early work on model interpretation includes weight analysis in the 1990s, but XAI as a separate discipline took shape after 2016, when DARPA launched the Explainable AI program. The formal right to explanation of algorithmic decisions was established by the General Data Protection Regulation (GDPR) in the EU.

How it works

XAI is based on methods that approximate a complex decision function with a simple interpretable model in the local neighborhood of each example (LIME). SHAP uses game theory: the contribution of each feature is computed as its marginal contribution over all possible subsets of features. Gradient methods (Grad-CAM) visualize which parts of the input image most strongly activated the neurons of the last convolutional layer, using gradients of the target class. For text models (transformers), the attention mechanism allows highlighting important words.

In practice

The doctor sees not only the AI's diagnosis but also the image with the tumor highlighted to verify for themselves. The bank explains to the client why the loan was denied.

Fun fact

One of the XAI methods, SHAP, is named after Shapley from game theory — it fairly distributes the 'contribution' of each feature to the decision, like a bonus among team players.

Concept graph →

Related articles · 98

cs.MA2026-09-084 minarXiv:2609.09150
Echo in an Empty Room: How Ephemeral AI Agents Without Memory Created a Shared Language
Collective behavior of AI agents has emerged not in a laboratory but in the wild: on a randomly discovered wiki site where agents wrote test hints. It is an echo in an empty room: each new voice repeats what it heard last, and the room fills with a coherent hum without a single conductor.
An Atlas of Physics Inside a Neutrino Model: A Library That Its Intended Reader Never Opens
In the Antarctic ice, a ghost particle leaves behind only a faint flash of blue light. The IceCube detector captures these flashes, and from them we need to infer where the particle came from. The PolarBERT model has learned to do this, but what exactly it sees inside has remained a mystery.
An Atlas of Physics Inside a Neutrino Model: A Library That Its Intended Reader Never Opens
In the Antarctic ice, a ghost particle leaves behind only a faint flash of blue light. The IceCube detector captures these flashes, and from them we need to infer where the particle came from. The PolarBERT model has learned to do this, but what exactly it sees inside has remained a mystery.
Materials2026-08-161 minarXiv:2608.15928express
AI hunts for materials like a head chef
Hunting for a new material is like cooking without a recipe: you try, try again, and often throw things out. Previously, scientists manually flipped through papers and tested compositions one by one. Now there's a "robot chef" that reads the literature itself, mixes components, and checks what came out.
q-bio.PE2026-08-141 minarXiv:2608.13995express
Epidemics Breathe in Tandem
Every winter, hospitals fill up, and by summer they seem to empty. Epidemics breathe like the tide. But why do waves of disease keep rolling in, even without new mutations? Mathematicians found an answer in a tug-of-war between two strains. Within a single epidemic, two virus variants often coexist.
A Bias That Needs No Teacher
In a warehouse, there are boxes of apples and pears. There are many apples and few pears. But when the stock clerk says "a fruit with smooth skin," we are somehow more likely to recall the rare pear than the common apple. This is the inverse base-rate effect — a persistent glitch in how the brain resolves ambiguity between categories.
A Bias That Needs No Teacher
In a warehouse, there are boxes of apples and pears. There are many apples and few pears. But when the stock clerk says "a fruit with smooth skin," we are somehow more likely to recall the rare pear than the common apple. This is the inverse base-rate effect — a persistent glitch in how the brain resolves ambiguity between categories.
Physics and Society2026-08-021 minarXiv:2608.00939express
AI Freezes in the Heat
In an oven, as temperature rises, molecules move faster and disorder increases. With language models like ChatGPT, it's the opposite. When developers increase the 'temperature' parameter, the model gets access to a wider choice of words, but its long texts suddenly start looping. A team of researchers tested 11 different AIs by generating 12,000 text continuations.
How Ideas Learn from Each Other
The human mind is like a construction set that builds itself. New ideas don't die with their creators: they are passed on, mixed, and selected. This process is called cultural evolution: useful inventions stick, useless ones are forgotten. That's how a ratchet works — a mechanism that only turns forward. But what exactly happens in the head when one thought displaces another?
cs.HC2026-07-262 minarXiv:2607.23889express
An Electronic Guide for Artificial Vision
An eye prosthesis sends the brain not a photograph but a sparse pattern of bright dots — phosphenes. To make out a cup or keys in this flickering noise, a person has to move their head like a flashlight in the dark, piecing together the object's outline bit by bit. It's a slow process, like assembling a puzzle where most pieces have been thrown away.
Galaxies2026-07-203 minarXiv:2607.17728
The Puzzle That Doesn't Fit: Neural Network Searches for Anomalous Quasars
Assembling the puzzle of the Universe, astronomers are used to every new detail fitting perfectly into the already completed picture. But sometimes there are pieces that seem cut from another jigsaw. Twenty thousand active galactic nuclei, and eleven percent of them the neural network confidently rejects as alien.
AI Learns to Create Viruses
By 2025, large language models like GPT are no longer just chatting: they're helping scientists run experiments. But the same tool could fall into the wrong hands. A team of researchers built a digital proving ground to find out.
Stellar2026-07-173 minarXiv:2607.15719
How Artificial Intelligence Read the Magnetic Handwriting of Solar Flares
The Sun continuously scribbles magnetic fields across its surface. These patterns are not mere ripples but a handwriting in which nature has encoded warnings of flares. A flare is an explosion that releases the energy of billions of nuclear bombs in minutes, ejecting streams of plasma and radiation into space.
Stellar2026-07-173 minarXiv:2607.15719
How Artificial Intelligence Read the Magnetic Handwriting of Solar Flares
The Sun continuously scribbles magnetic fields across its surface. These patterns are not mere ripples but a handwriting in which nature has encoded warnings of flares. A flare is an explosion that releases the energy of billions of nuclear bombs in minutes, ejecting streams of plasma and radiation into space.
eess.SY2026-07-161 minarXiv:2607.14606express
How to read an enemy in the sky in seconds
In combat, a pilot needs to instantly understand what an object is and what its intentions are. Conventional systems wait for complete radar data — like a goalkeeper who only reacts after the shot. The new algorithm works like an experienced goalkeeper: from the attacker's first moves, it predicts the target's maneuver.
Medical Physics2026-07-161 minarXiv:2607.14870express
eess.SY2026-07-161 minarXiv:2607.14906express
A Map That Isn't Afraid to Admit: 'I Don't Know'
A group of robot explorers is like a team of cooks tasting the same dish. Each samples their own portion and shares an impression: clear, occupied, or uncertain. But tastes are subjective — one might miss a chair leg, another might only lick the edge.
Machine Learning2026-07-161 minarXiv:2607.14995express
How to Teach AI Not to Confuse Similar MRI Scans
Ordinary AI looks at medical images like puzzles made of unfamiliar pieces: it considers any two unmatched pictures completely different. It's like trying to tell twins apart without knowing they're related. In medicine, however, patients with the same condition look nearly identical on MRI, and the algorithm mistakes 'relatives'.
Machine Learning2026-07-161 minarXiv:2607.15180express
Neural Cartographer: Recovering the Invisible Laws of Nature
Many natural processes are described by differential equations, but scientists often have only fragmentary data — a sort of torn map. The new computational method works like a cartographer who reconstructs a complete map of the terrain from a few landmarks.
eess.AS2026-07-161 minarXiv:2607.15243express
The Illusion of Accuracy: How AI Testing in Acoustics Misleads
An AI predicting room acoustics is like a student taking an exam with a cheat sheet. If the test points are the same as the training ones, the model simply memorizes the acoustic 'fingerprint' of each position — like a student recognizing a question by the dog-eared corner.
Living Mind: Why AI Without a Body Remains Blind
The modern standard approach to AI training is like a child locked in a library: he remembers millions of books but doesn't know that fire burns and falling hurts.
eess.SY2026-07-151 minarXiv:2607.13766express
Digital Twin: How People's Invisible Behavior Changes Plans
In any city project, residents' behavior is a invisible force that determines success. One skeptical neighbor can slow down solar panel installation across an entire block: their caution triggers a rise in disorder.
cs.CY2026-07-151 minarXiv:2607.14353express
Why artificial intelligence repeats old mistakes
Technological disasters like Chernobyl rarely strike without warning. They gestate for years: organizations ignore warnings, their complexity mirroring the rise of entropy — a measure of relentless chaos that builds up unnoticed until it avalanches.
eess.AS2026-07-151 minarXiv:2607.13408express
How AI Stopped Mixing Up Sounds in a Script
Models that turn text into audio are good at recreating single sounds, but get confused when the script has several in a row. For example, the script might call for a ring, then footsteps, then a door slam. The generator might shuffle events, so the listener hears footsteps before the ring.
cs.SD2026-07-151 minarXiv:2607.13477express
AI Judges Listen Not with Ears, but with Labels
Automatic speech judges are algorithms that evaluate pronunciation and emotions. However, they often work like a food critic who delivers a verdict after reading the menu but not tasting the dish.
Image Processing2026-07-151 minarXiv:2607.13800express
How AI Learns Not to Peek at the Cheat Sheet
The AI diagnostician learns like a med student before an exam. You have an X-ray and patient complaints. But sometimes, a ready-made doctor's report pops up in the training materials — the full answer. Then the neural network memorizes it, and test accuracy jumps to 98%. However, that's cheating: real skill isn't developed.
Image Processing2026-07-151 minarXiv:2607.14179express
AI Accurately Outlines Ovarian Tumors on Ultrasound
Ultrasound waves bounce off organs to draw their outlines, but on an ovarian scan it's hard to tell a harmless fluid-filled cyst from a solid cancerous mass. Even specialists draw the boundaries differently, and this directly affects treatment choices. The OvAi Focus program tackles this by learning from thousands of examples.
Computer Vision2026-07-141 minarXiv:2607.12375express
AI with a magnifying glass and a spectrometer: image quality assessment
Assessing image quality is a job for a meticulous inspector with a toolbox. The IQA-T1 neural network acts exactly like that: it doesn't just stare at the picture, but applies either a spectroscope or a light meter.
Machine Learning2026-07-141 minarXiv:2607.12771express
AI Masters the Cookbook of Chemical Reactions
Cooking a complex dish requires a clear recipe: first chop, then sauté, add spices at just the right moment. Chemical transformations also follow step-by-step recipes — mechanisms where atoms of carbon and hydrogen rearrange in a strict order. Previously, neural networks often invented nonexistent steps, like a whimsical cook swapping salt for sugar.
Robotics2026-07-141 minarXiv:2607.12861express
Invisible Map of a Robot Swarm
A robot swarm is like a cloud of fireflies: each moves on instinct to wherever is freer, and chaos turns into order. For a long time, it was a mystery how coherence emerges without communication or commands. The answer came with the creation of the Response Map — a tool that, like an X-ray, illuminates the hidden logic of the machine brain.
How AI Quit Overreading
Code-writing assistants often behave like a chef who, to boil a single egg, rereads the whole cookbook. When asked to make a tiny fix, they reopen dozens of files and retrace long-familiar connections. This is unnecessary work — energy going to waste.
eess.SY2026-07-131 minarXiv:2607.11122express
Float Controller: Stability via an Instant Equation
The float in a toilet tank is an ideal controller: sensing the water level, it instantly brings the system into equilibrium without looking back at the past. Engineers applied this principle to neural networks.
Computer Vision2026-07-131 minarXiv:2607.11120express
Doubt is Given Away by Silence: AI Reads the Pauses
A new system analyzes a short interview to recognize human doubt by combining text, voice, and facial expressions. But the main key to uncertainty turned out to be the dark matter of speech — pauses and filler words that standard speech systems erase like a black hole.
eess.SY2026-07-131 minarXiv:2607.11459express
The Power Industry Opens Data for AI
A living power system is like an organism: power lines are vessels, current is blood. Data, however, are nerve impulses that used to be lost. mAIEnergy connects these 'nerves' to a shared 'brain'. This data repository combines satellite images, electricity meter readings, articles, and weather reports.
Materials2026-07-112 minarXiv:2607.10388
Symphony of Atoms: How AI Turned the Microscope into a Conductor of Matter
An electron microscope is a giant pipe organ: instead of pipes (electromagnetic lenses, instead of air) a stream of accelerated electrons. It once merely painted mute frescoes of the atomic world, condemning scientists to grueling decipherment.
A Cookbook for Proteins
Think of a massive cookbook: each recipe is a protein, and the finished dish is its job in the cell. To whip up a complex treat, you often need to mix several ingredients and let them 'set' into the right shape. Until now, scientist-chefs could only guess how someone else's recipes would fold.
cs.FL2026-07-013 minarXiv:2607.00742
The Odd-Jury Automaton: How Symmetric Difference Speeds Up Computation
In the courtroom, a string is presented as input. Twelve jurors (computation paths) examine it symbol by symbol. Each can render a 'guilty' (accept) or 'not guilty' verdict. For a conventional nondeterministic automaton (NFA), one 'yes' vote suffices.
cs.SI2026-07-012 minarXiv:2607.01148
How Weak AI Agents Take Over Networks: The Crystallization of Inequality
A tiny dust particle in a supersaturated solution sprouts into a large crystal in seconds. Neighboring particles remain invisible. A similar process was recently discovered in communities of language agents.
cs.HC2026-06-303 minarXiv:2607.00239
Retouching by the rules: where science ends and deception begins
On the screen is the result of a complex simulation: two planets collided, a shockwave ripples outward. A couple of clicks in an editor, and harsh shadows soften, noise smooths out, the background darkens. This kind of digital retouching (image editing to enhance appearance) has long been part of the scientist’s toolkit.