How to Test a Predictor Without Knowing the Answer
A lab sends a robot a list of a thousand candidate molecules for a new drug. The predictor robot, trained on old data, gives each one a 'fit or no fit' score. But when the list comes from an unfamiliar area, a different virus or a rare tumor, the robot starts to get confused. Its confidence stays high, while actual hits drop to zero.