A weather forecaster might correctly predict rain every day, but when he says "70% chance of precipitation," it actually rains only 30% of the time. His forecasts are useful, but the numbers can't be trusted. The same happens with models that predict cancer survival from tissue images and genetic data.
Researchers tested 14 such models on data from five cancer types. They compared the models against a benchmark: if a model says "the probability of surviving a year is 60%," then among similar patients, about 60% should indeed survive a year. It turned out that most models systematically get the probabilities wrong, even though they do a good job ranking patients by risk. For example, one model correctly sorted patients into risk groups, but its probabilities were off in all five checks.
The good news: a simple mathematical adjustment, applied after the model is trained, noticeably improves calibration. The model starts giving more honest probabilities without losing accuracy. This means such predictions can be made more reliable for clinical use.
The forecaster from the first sentence could calibrate his percentages by looking at his past forecast statistics. Cancer models learn the same trick, and that's a step toward doctors trusting not just the order of patients on a risk list, but the numbers themselves.
as written by the authors arXiv:2604.04239
Multimodal deep learning models that fuse whole-slide histopathology images with genomic data have achieved strong discriminative performance for cancer survival prediction, as measured by the concordance index. Yet whether the survival probabilities derived from these models - either directly from native outputs or via standard post-hoc reconstruction - are calibrated remains largely unexamined. We conduct, to our knowledge, the first systematic fold-level 1-calibration audit of multimodal WSI-genomics survival architectures, evaluating native discrete-time survival outputs (Experiment A: 3 models on TCGA-BRCA) and Breslow-reconstructed survival curves from scalar risk scores (Experiment B: 11 architectures across 5 TCGA cancer types). In Experiment A, all three models fail 1-calibration on a majority of folds (12 of 15 fold-level tests reject after Benjamini-Hochberg correction). Across the full 290 fold-level tests, 166 reject the null of correct calibration at the median event time after Benjamini-Hochberg correction (FDR = 0.05). MCAT achieves C-index 0.817 on GBMLGG yet fails 1-calibration on all five folds. Gating-based fusion is associated with better calibration; bilinear and concatenation fusion are not. Post-hoc Platt scaling reduces miscalibration at the evaluated horizon (e.g., MCAT: 5/5 folds failing to 2/5) without affecting discrimination. The concordance index alone is insufficient for evaluating survival models intended for clinical use.
To assess the quality of probabilistic forecasts, researchers use a calibration plot where a perfect model produces a straight line at a 45-degree angle. If the line deviates, the model is systematically over- or under-estimating probabilities.