Look into the night sky — before you is not a scattering of lights, but a multi-layered manuscript. Under the bright ink of stars and galaxies appears a faded, almost erased text. This is the message of dark matter, written by gravitational lensing: invisible masses distort the light of distant galaxies, leaving barely noticeable watermarks on the cosmic parchment. For decades, astronomers — from Fritz Zwicky to Vera Rubin — have sought evidence of its existence, and Einstein predicted the effect back in the 1930s. But to decipher this hidden layer, one must measure the 'shear' — the systematic change in galaxy shapes. Previously, this required meticulous work: separately identify each galaxy, precisely measure its ellipticity, and then account for atmospheric blurring and detector noise. Like a medieval monk scraping off the top layer of letters with a scalpel.
A new approach — neural posterior estimation (NPE) — works differently. It doesn't scratch the surface, but immediately reads the entire sheet, discerning the hidden truth through noise and distortions. Trained on synthetic universes — realistic simulations of the future LSST survey with multi-band photometry (riz filters), variable point spread function, stars and CCD artifacts — the deep neural network learns to directly transform pixels into a probability distribution of shear. Millions of virtual cosmos born in numerical simulations become a testing ground for machine learning, where the ability to automatically discard confounding factors without explicit formulas is honed. Thus detection, separation, measurement and calibration merge into a single implicit act — a neural paleographer that has mastered the handwriting of dark matter.
Tests showed impressive accuracy. Even with the addition of saturated stars with diffraction 'whiskers' and CCD artifacts, the root-mean-square error (RMSE) remained around ~0.0016–0.0019, and the Pearson correlation coefficient exceeded 0.99. The classical AnaCal algorithm showed greater scatter. Moreover, the posterior intervals were excellently calibrated: the empirical coverage of 90% confidence regions almost exactly matched the nominal, meaning the neural network not only guesses, but honestly reports its uncertainty. In essence, it provides not just a number, but a measure of its ignorance — a critical skill when each pixel may hold a key to the nature of dark energy. The measure of systematics remains the equation \(\hat{\gamma}_j^n = (1+m_j)\gamma_j^n + c_j\), where the multiplicative bias \(m_j\) and additive \(c_j\) in the new method tend to zero. In other words, the estimates are close to the truth without the traditional chain of calibration corrections.
This breakthrough means that the analysis of cosmological surveys can be radically simplified. Stage IV projects — LSST, Euclid, Roman — will count many billions of galaxies, and the slightest systematic can distort conclusions about dark energy and the expansion of the Universe. The neural network approach can process such data volumes, directly turning pixels into maps of dark matter and — with knowledge of redshift — into three-dimensional tomography. We are entering an era when the palimpsest will be read automatically and continuously, and manual calibrations will remain in archives.
Ahead lies the main challenge: the discrepancy between the simulator and the real sky. Distribution shift can undermine accuracy, so the next steps are diagnosing model misspecification and techniques of robust Bayesian inference. Simultaneously, ever more detailed simulators with variable shear and convergence are being created, and then NPE will be able to directly reconstruct the dark matter density field at the pixel level, bypassing catalogs. After all, it is still unknown whether dark matter is particles or a modification of gravity — and an accurate lensing map will give a decisive answer. In the long term, we will learn to read not only the two-dimensional but also the three-dimensional text of dark matter, finally discovering what the hidden palimpsest of our Universe is truly made of.
The LSST survey is expected to observe about 20 billion galaxies. If one galaxy were a grain of sand, you could cover a football field with a layer several centimeters thick.
Gravitational lensing inspires science fiction: in Robert Charles Wilson's novel 'Spin', a mysterious field distorts starlight, hinting at hidden forces.