One Galaxy, Two Telescopes: How to Separate Physics from Distortions express
If you photograph the same mountain with a film camera and a modern smartphone, you get different images: colors are brighter in one, details are blurrier in the other. The same goes for telescopes: each instrument sees the sky in its own way, adding its own distortions to the true light of galaxies, like a filter through which we look.
This new work offers a way to remove that 'filter.' The scientists collected pairs of images of the same galaxies taken by two large sky surveys: DESI Legacy Imaging Survey and Hyper Suprime-Cam. The neural network was trained as follows: it receives two images and must reconstruct one from the other, but taking into account that the cameras are different. During training, the network itself finds what is common to the two images (that is the real galaxy) and what differs (that is the distortion of a particular telescope).
As a result, the neural network outputs a 'clean' representation of the galaxy — without the noise and distortions of a specific instrument. This allows, for example, searching for similar galaxies in the archives of different telescopes without fear that their similarity is spoiled by different cameras. This approach is useful not only for astronomy: it provides a general recipe for training neural networks on data from different instruments — from medical scanners to sensors in cars.
Original abstract
as written by the authors arXiv:2604.09787
Data collected from the physical world is always a combination of multiple sources: an underlying signal from the physical process of interest and a signal from measurement-dependent artifacts from the sensor or instrument. This secondary signal acts as a confounding factor, limiting our ability to extract information about the physics underlying the phenomena we observe. Furthermore, it complicates the combination of observations in heterogeneous or multi-instrument settings. We propose a deep learning framework that leverages overlapping observations, a dual-encoder architecture, and a counterfactual generation objective to disentangle these factors of variation. The resulting representations explicitly separate intrinsic signals from sensor-specific distortions and noise, and can be used for counterfactual view generation, parameter inference unconfounded by measurement distortions, and instrument-independent similarity search. We demonstrate the effectiveness of our approach on astrophysical galaxy images from the DESI Legacy Imaging Survey (Legacy) and the Hyper Suprime-Cam (HSC) Survey as a representative multi-instrument setting. This framework provides a general recipe for scientific and multi-modal self-supervised pretraining: construct training pairs from overlapping observations of the same physical system, treat sensor- or modality-specific effects as augmentations, and learn invariant representations through counterfactual generation.
Both telescopes whose data the scientists used are on Earth, but one of them (Hyper Suprime-Cam) is installed in Hawaii, while the other (DESI Legacy) scans the sky from an observatory in Arizona. The distance between them is about 4,800 km, but they see the same galaxies.