method

medical image to genomic mapping

A deep learning technique that predicts gene expression from medical imaging data like histology or MRI.
60 related articles

Description

Medical image-to-genomic mapping is a deep learning approach that bridges the gap between visual medical data—such as histology slides, MRI scans, or CT images—and the underlying molecular state of tissues, particularly gene expression. Instead of requiring expensive and invasive genomic sequencing for every sample, these models learn to infer gene activity patterns directly from images. This matters because it could make genomic analysis more accessible, faster, and cheaper, enabling personalized medicine in settings where sequencing is not routine.

History

The field gained traction in the late 2010s with the rise of deep learning in pathology, notably with studies demonstrating that gene expression could be predicted from hematoxylin and eosin (H&E) stained whole-slide images. The exact origin is diffuse, but key early work appeared around 2019-2020.

How it works

The technique typically uses a convolutional neural network (CNN) or a vision transformer to extract features from medical images. These features are then mapped, often through a regression head or a multi-task learning framework, to predict the expression levels of thousands of genes simultaneously. The model is trained on paired datasets where both the image and the corresponding gene expression profile are known, learning visual patterns that correlate with molecular activity.

In practice

It is used in cancer research to infer molecular subtypes and mutations from routine histology slides, potentially guiding treatment decisions without additional genomic tests.

Fun fact

Some models can predict gene expression from MRI scans, not just histology, opening the door to non-invasive genomic profiling of brain tumors.

Concept graph →

Related articles · 60

q-bio.BM2026-08-182 minarXiv:2608.17759express
A Protein Thread Woven from Noise
Under an electron microscope, a protein looks like a series of blurry shadows — hundreds of thousands of images of the same molecule, but from different angles and with heavy noise. Previously, from these shadows, one first assembled a volumetric 'haze'—a map of how electric charge is distributed inside the protein.
Machine Learning2026-08-172 minarXiv:2608.16134express
Brain-Computer Interface: Teaching Machines to Read Minds Day After Day
Every day, the brain sounds a little different. Yesterday, the thought of moving your left hand produced one pattern of electrical waves; today, it's slightly different. For a brain-computer interface (a device that turns mental commands into actions), this is a big problem: yesterday's settings don't work today.
Quantum Physics2026-08-161 minarXiv:2608.15795express
Complex quantum system revealed through small parts
Quantum systems made of many particles are the basis of future computers and secure networks. But the more particles there are, the harder it is to understand how they are arranged: the number of possible states grows like an avalanche. The usual approach, measuring everything, takes an incredibly long time.
Medical Physics2026-08-091 minarXiv:2608.08919express
How a Low-Dose X-Ray Becomes a Sharp Image
A low-dose X-ray image looks like a view through a fogged-up window: outlines are there, but details drown in a hazy veil. Doctors need such images during radiation therapy — to aim the beam precisely at the tumor and spare healthy tissue. But saving on radiation comes at the cost of noise, streaks, and artifacts that obscure what matters.
Image Processing2026-08-082 minarXiv:2608.08173express
Separating Brain Scans: Where Disease Ends and Distortion Begins
Two scans of the same brain taken at different times are like photographs of the same house before and after renovation. To see what changed, you overlay them and look at the differences. But an MRI scanner is not a perfect camera: it adds its own distortions, as if the photo were taken through slightly warped glass.
Computer Vision2026-08-031 minarXiv:2608.02208express
A Full-Body Scan Told Us About Future Diseases
An X-ray machine takes a scan, and the doctor sees only numbers — bone density, fat percentage. But the image holds far more. A team of researchers created a model called LeDXA that reads these scans like a map of hidden bodily signals. The model was trained on 11,540 scans from the Human Phenotype Project with no labels at all.
How a Gene Map Learns from Nature Itself
A huge orchestra without a conductor. Each musician is a gene, and the melody is the cell's complex work. Previously, looking at a tissue photo, a computer tried to guess each instrument's part individually. It didn't know that violins usually play with violas, and trumpets with trombones. The program just output a list of notes, without sensing the harmony.
Machine Learning2026-08-011 minarXiv:2608.00776express
Chemistry Looks at Molecules Through the Eyes of a Computer
Predicting the yield of a chemical reaction is a bit like trying to guess the harvest knowing only the soil composition. Traditionally, chemists have relied on long columns of numbers, quantum descriptors, that describe electron energies but say nothing about molecular shape.
Machine Learning2026-07-161 minarXiv:2607.14995express
How to Teach AI Not to Confuse Similar MRI Scans
Ordinary AI looks at medical images like puzzles made of unfamiliar pieces: it considers any two unmatched pictures completely different. It's like trying to tell twins apart without knowing they're related. In medicine, however, patients with the same condition look nearly identical on MRI, and the algorithm mistakes 'relatives'.
q-bio.GN2026-07-161 minarXiv:2607.15022express
Gene Map Predicts Breast Cancer Outcomes
Genes are like houses in a city: in some, life bustles; in others, silence reigns. This 'urban' genetic landscape was analyzed in a thousand women with breast cancer.
Image Processing2026-07-151 minarXiv:2607.13812express
A Fresh Perspective on 3D Medicine
The data from a single CT scan is comparable to several feature-length films. TCAM-Diff bypasses this bottleneck: the program compresses the 3D scene into three shadow projections — like a sculptor reconstructing a solid statue from silhouettes on three walls.
q-bio.QM2026-07-151 minarXiv:2607.14163express
Genes as a Dish: A Fresh Look at Cells
Data on gene activity usually looks like a long string of numbers, like a rainbow stripe (spectrum) with no visible connections. In the scVision model, these numbers are turned into a meaningful picture — not a list of ingredients, but a photo of a finished dish.
Machine Learning2026-07-151 minarXiv:2607.14410express
LATTICE: How to Unify Cellular Data
For a long time, biologists studied cells by listening to just one voice in a polyphonic choir: measuring gene activity while ignoring how DNA is packed. It's like trying to grasp a conversation knowing only one language. LATTICE fixes this: the program learns five molecular 'languages' at once — from gene activity to spatial markers.
Machine Learning2026-07-142 minarXiv:2607.12382express
gradCSCG: How an Algorithm Maps a Room by Mimicking a Child and Brain Neurons
A child exploring an unfamiliar apartment crawls from object to object and remembers what is visible from each corner. This is how the brain builds a mental map — an internal model of space. The gradCSCG algorithm works on the same principle: it receives a video stream from a camera and independently draws a map of the room, relying only on the changing images.
q-bio.QM2026-07-042 minarXiv:2607.06583express
How Every Cell Ages: A Journey Through the Methylation Landscape
DNA methylation consists of tiny chemical marks on our genes. Over the years, they change on a fairly strict schedule. Standard epigenetic clocks look at these marks and output a single number: biological age. But a number can't tell you how the pattern of marks rearranged day by day, year by year. The authors of this work decided to turn a static photograph into a continuous film.
Cosmology2026-06-303 minarXiv:2606.31988
JADE: The AI Detective That Reads the Cosmic Web in Light Trails
Light from distant galaxies travels to us for billions of years, but along the way it gets bent and twisted by the invisible web of dark matter. This effect, predicted by Albert Einstein within general relativity, is called gravitational lensing.
q-bio.PE2026-06-191 minarXiv:2606.21351express
How Antibodies Learn: A Map of Beneficial Mutations
The immune system is like a tireless editor, revising antibody drafts to make them stickier to viruses and bacteria. This process is called antibody maturation: random typos (mutations) arise in the genes, and natural selection keeps those that improve target capture.
Neural Network Embedded in a Neuron Model
A neuron resembles an electrical circuit with batteries and variable resistors. The resistors are played by ion channels — protein pores in the membrane that open and close, letting charged particles through. Whether a nerve impulse fires depends on how fast and at what voltage these pores operate.
Machine Learning2026-06-092 minarXiv:2606.11382express
Teaching Neural Networks to Read Molecules at a Glance
Finding the right molecule among billions of options is like finding a single grain in a sack of rice. To avoid sifting through everything by hand, chemists increasingly turn to neural networks. But large models require powerful computers, while small ones often lack the big picture. GLACIER tackles this problem like a bright student learning from several mentors at once.
Machine Learning2026-06-061 minarXiv:2606.08287express
A neural network that senses stress in parts of any shape
Designing machines is like cooking: you need to know exactly where the 'dough' will tear. Old computer simulations check every point on a part for hours, like measuring the entire pot to make sure the handle doesn't fall off.
Medical Physics2026-05-242 minarXiv:2605.24815express
How to Make MRI See a Beating Heart
A photographer shoots a runner with a long exposure and gets a blurry silhouette. That's roughly how a standard MRI scanner "sees" moving organs: the heart contracts, the lungs breathe, and the image smears. To avoid this, patients are asked to freeze and hold their breath. But for a child or someone short of breath, that's nearly impossible. The method called GRASP tackles the problem differently.
Medical Physics2026-05-022 minarXiv:2605.01265express
X-ray imaging without interference, like a photo without glare
An X-ray machine with gratings takes not one but three pictures at once: a regular image, a map of beam deflections invisible to the eye, and an image of tiny structures that scatter light. Together, they help spot, for example, emphysema or pulmonary fibrosis. But there's a catch: to separate these three layers, you need a clever mathematical trick.
Medical Physics2026-04-282 minarXiv:2604.25356express
X-ray detective without cheat sheets
You taste a fruit smoothie through two different straws. One lets through only the big chunks, the other only the liquid. Individually, the sensations are misleading, but together they give you the key: here, it seems, is banana, and here, strawberry.
Computer Vision2026-04-251 minarXiv:2604.23137express
Two Eyes Are Better Than One: How a Neural Network Learns to See Tumors
A doctor studies a brain scan: somewhere in the tangle of tissue, a tumor is hiding. One glance picks out small spots and edges, another takes in the overall shape and location. A person combines both, but across hundreds of scans that gets tedious. A new program does the same thing automatically.
Disordered Systems2026-04-252 minarXiv:2604.23431express
Magnetic Patterns Under the Microscope
A magnetic film just tens of atoms thick stores information in tiny magnetic vortices. But you can't look inside and see these vortices directly: the instrument only senses the weak magnetic field around the film. Reconstructing the magnetization pattern from such a field is like guessing the shape of an invisible relief from its shadow on a wall.
Machine Learning2026-04-251 minarXiv:2605.00876express
AI learns to look like a doctor
A doctor rarely makes a diagnosis from a single quick look at a scan. They peer, zoom in, compare with an atlas, flip through articles. Until recently, neural networks worked differently: see the image, immediately produce the text. The authors of the new study decided to give the model time and tools.
Computer Vision2026-04-231 minarXiv:2604.21573express
Histology Predicts Genes
A microscopic image of tissue is like a city map: streets, houses, parks. An experienced eye can tell a lot about a neighborhood from such a map. But what if you could predict from the map which houses are noisy today and which are quiet?
q-bio.QM2026-04-182 minarXiv:2604.18634express
The Heart as a Map: Topology Reveals Hidden Patterns in Cardiac Rhythm
Each heartbeat can be drawn as a point on a sheet of paper. Thousands of these points merge into a cloud. The Mapper algorithm turns this cloud into a network: it connects points that behave similarly and colors them according to blood flow strength. The result is not just a pile of numbers but a colored map showing which parts of the heart work in sync and which are out of step.
Machine Learning2026-04-161 minarXiv:2604.15273express
Graph Labels: When Quantum Methods Yield to Simplicity
A city map: intersections are nodes, streets are connections. For an algorithm to understand the structure, each node needs a label. The classical approach — standard model — assigns simple tags, like house numbers.
Body Map: How a Routine Tissue Image Reveals Genes
A pathologist stains a tissue section with hematoxylin and eosin — cells turn blue and pink, and under the microscope you can see their shape and arrangement. But which genes are active in each cell cannot be read from such an image. That requires a separate, expensive assay that reveals the activity of thousands of genes at once, but it is rarely performed.
Computer Vision2026-04-042 minarXiv:2604.03710express
How Graphs Find Melanoma on Your Skin
A mole on your skin looks like a mosaic of colored pieces. Each piece is a small patch of the image where color and texture barely change. Such pieces are called superpixels. If you connect them with lines, you get a network that looks like a subway map: stations are the pieces, and lines are the connections between them. The weight of each line shows how strongly two pieces are related.
Medical Physics2026-03-271 minarXiv:2603.27003express
Heart by Contour: How AI Builds 3D Shape
The heart is not a perfect sphere; its shape changes with every beat. To build an accurate 3D copy for a computer model, doctors typically take dozens of MRI slices. But that's like assembling a watermelon from thin slices: the fewer slices, the more guesswork. A new approach teaches a neural network to understand the heart's shape from sparse contours.
q-bio.QM2026-03-271 minarXiv:2603.27017express
Your Smartphone Instead of an X-Ray: How a Photo of Your Body Predicts Health
The scale tells you your weight, but it's silent about where those pounds sit — in muscle or belly fat. And it's the fat tucked deep inside, around your organs, that's most often linked to heart disease and diabetes. Previously, you could only see it with an expensive scanner at a clinic. Now, a smartphone is all you need.
Quantum Gases2026-03-262 minarXiv:2603.25775express
Neural Network Spots Knots in Quantum Fog
In a cold cloud of atoms chilled to near absolute zero, energies can behave like threads that weave into braids. These braids are not just a pretty image: their pattern is tied to topology, the branch of mathematics that studies properties preserved under continuous deformations.
Medical Physics2026-03-232 minarXiv:2603.21980express
Heart MRI in a minute: how Gaussian blobs speed up scanning
An MRI scanner is like a camera that photographs your insides, but very slowly. The patient has to lie still for several minutes while the heart keeps beating. If the person moves or the rhythm goes off, the image blurs. Doctors then have to repeat the scan, wasting time and nerves.
Biological Physics2026-03-211 minarXiv:2603.20764express
Artificial Intelligence Tracks Microbes
To avoid losing a fast soccer player in the frame, a drone doesn't need a bright beacon — recognizing the silhouette is enough. Similarly, a neural network now tracks microbes: the algorithm analyzes an ordinary microscope image and instantly finds a bacterium in a drop of water.
cs.CE2026-03-202 minarXiv:2603.19829express
The Brain as Clay: Why It Shrinks in Alzheimer's Disease
Alzheimer's disease begins quietly: deep in the brain, two proteins, tau and amyloid-beta, start to accumulate. They behave like weed seeds: one clump that appears causes neighboring cells to produce similar clumps. Gradually, the proteins spread along nerve fibers, like water through the capillaries of a paper towel, and invade new areas.
Chemical Physics2026-03-181 minarXiv:2603.18291express
Why Molecules Don't Sing Just One Note
Strike a tuning fork, and it rings with a single pure note. But if the fork is trembling and people are chattering nearby, the sound won't be perfect. Same with molecules: each one absorbs light of a specific color, but due to its own vibrations and the jostling of neighbors, that color gets smeared out. The authors of the paper figured out how to show this to students without complex formulas.