A lab sends a robot a list of a thousand candidate molecules for a new drug. The predictor robot, trained on old data, gives each one a 'fit or no fit' score. But when the list comes from an unfamiliar area, a different virus or a rare tumor, the robot starts to get confused. Its confidence stays high, while actual hits drop to zero.
The CaliPPer tool solves this problem without a single test answer. It measures how far a new sample has drifted from familiar training examples — much like a navigation app estimates how far you've strayed from known roads. For each molecule, it computes the 'distance to familiar data' (S2DD), and based on that, the model honestly says: 'In this area I navigate poorly, better not trust my estimates.'
Across ten different models predicting immune cell binding to targets, CaliPPer showed a correlation between distance and real accuracy of 0.80–0.92 (where 1 is a perfect match). The error in quality estimation dropped to 0.008–0.070. In five real studies where standard models found no working molecules (0 out of 5), recalibrating confidence allowed 3 out of 5 to be confirmed — simply by downgrading false favorites and promoting modest but reliable candidates in the ranking.
This is not a replacement for experiments but a smart filter between the computer and the test tube. Instead of spending weeks checking hundreds of duds, the biologist gets a short list with an honest assessment of the odds.
The distance to training data (S2DD) acts like 'foreignness': if a new molecule is very different from anything the model has seen before, its prediction is almost certainly wrong, even if the model is confident in it.