A Medical 3D Scanner That Learns to See It All at Once express
To capture every detail of a statue's shape, you could break it into millions of tiny cubes and describe each one — slow, heavy, and your computer's memory would groan at the seams. That's how 3D medical scans are usually handled: a CT or MRI is a huge array of such cubes (voxels), and to create or improve an image, the machine has to plow through all of them.
The new approach is more like a sculptor who doesn't build up the form bit by bit but defines smooth curves with a single mathematical line. The authors built a system where the entire volume (a liver, a brain, or lungs) is described as a continuous function of coordinates. Just plug in three numbers (x, y, z), and the model tells you the tissue density at that point. At the heart of the method is a compact autoencoder: it first compresses the scan into a small set of key features (like lecture notes), then reconstructs it back, but as a continuous object. This trick avoids the need to slice the image into overlapping patches, which was previously unavoidable.
The practical benefits show up in two scenarios right away. The first is generating synthetic but anatomically plausible 3D images for training doctors or algorithms. The second is restoring a high-quality image from incomplete data. For example, in accelerated MRI, the machine takes fewer measurements so the patient spends less time in the tube, and the model fills in the gaps based on learned "continuous" anatomy. And all of this runs on a single standard GPU, without assembling the image from patches with visible seams.
Of course, direct voxel-by-voxel reconstruction is still slightly more accurate in fine details. But the gains in speed and image integrity are a trade-off that paves the way for processing high-resolution 3D scans in routine clinical practice, without waiting for supercomputers. And then the statue, once assembled from cubes, will finally take on smooth, lifelike contours.
Original abstract
as written by the authors arXiv:2607.16491
High-resolution three-dimensional (3-D) medical diffusion models remain constrained by the cost of processing full volumes, even when denoising is performed in a compact latent space. We introduce a continuous 3-D latent diffusion model (LDM) framework for computed tomography (CT) and magnetic resonance imaging (MRI) generation and measurement-guided reconstruction. Its central component is a compact autoencoder (AE) with a coordinate-conditioned local implicit image function (LIIF) decoder that represents a volume as a continuous function of spatial coordinates. By evaluating the convolutional decoder once on the latent grid and restricting repeated computation to a lightweight implicit head, the proposed design avoids overlapping sub-volume decoding while remaining differentiable for inverse-problem optimization. We evaluate the framework on CT volumes of 512^3 voxels and MRI volumes of 256^3 voxels. On high-resolution CT, the proposed AE is approximately x12-32 faster than the evaluated reference autoencoders, achieves the lowest peak graphics processing unit (GPU) memory use, and retains comparable structural fidelity despite a moderate reduction in voxel-level accuracy. The resulting frozen 3-D latent prior generates coherent full volumes without visible patch seams and can be applied, without task-specific retraining, to sparse-view CT and accelerated MRI reconstruction through hard data consistency. Although direct pixel-domain reconstruction remains more accurate, the results demonstrate that a single volumetric latent prior can support both unconditional generation and measurement-conditioned reconstruction on one GPU. Overall, the framework provides a practical trade-off between continuous volumetric decoding, computational efficiency, and fine-detail preservation. Our code will be made available at https://github.com/mellak/.
A typical high-resolution CT scan (512×512×512 voxels) is comparable to 134 million pixels — about 17 frames of 4K video packed into a single cube.
In the movie Total Recall, Arnold Schwarzenegger's character walks through a scanner that instantly creates a 3D holographic copy of his body — the new model takes a step toward making such reconstruction routine rather than science fiction.