A doctor scrolls through a patient's chart and sees a line: "The drug reduces risk." But how much can you trust that? Is the claim based on a couple of case reports or on tens of thousands of patients in a blinded trial? The answer determines the treatment. Today, such texts are increasingly written not by humans but by large language models — the same ones that help search for information and summarize documents. Researchers from several groups decided to test whether the machine understands the difference between a shaky hint and a rock-solid conclusion.
They gathered 45,134 clinical claims from open sources and mapped 20,611 of them onto a four-level scale of evidence strength, from "almost no data" to "confirmed by multiple studies." Then they asked 22 language models — from small ones, about the size of a desk book (0.6 billion parameters), to quite large ones (70 billion)—to rate each claim. The models were diverse: general, medical, and logic-focused. The result was surprising. When the researchers "peeked" inside the model with a special mathematical probe, the signal about the true strength of evidence read beautifully — on average with 71.8% accuracy (as if seven out of ten claims were correctly weighed). But when the same model was asked directly, "How reliable is this?"—the answers degraded to random guessing. The gap was 25–27 percentage points.
Curiously, this signal turned out to be largely lexical — tied to words and phrases rather than deep understanding. As soon as the model was moved to a different medical topic or a different rating system, the ability to read the signal vanished. Yet it did not coincide with a simple truth-versus-lie distinction: the model could correctly identify that a claim was poorly supported even if it was factually true. So inside the artificial brain lives an unspoken intuition for the fragility of arguments, but it doesn't come out.
This silent compass is both a hope and a warning. Hope, because the signal can be extracted and used even if the model itself stays silent. Warning, because today, when we read generated text, we don't hear the machine's inner voice, and we risk mistaking the shaky for the solid. The doctor at the monitor sees neat lines but doesn't know whether the ground beneath them trembles.
The internal signal for evidence strength turned out to be strongest in general-purpose models, while specialized "medical" and "reasoning" models showed weaker signals, as if narrow specialization dulls universal intuition.