A doctor scrolls through a patient's chart and sees a line: "The drug reduces risk." But how much can you trust that? Is the claim based on a couple of case reports or on tens of thousands of patients in a blinded trial? The answer determines the treatment. Today, such texts are increasingly written not by humans but by large language models — the same ones that help search for information and summarize documents. Researchers from several groups decided to test whether the machine understands the difference between a shaky hint and a rock-solid conclusion.
They gathered 45,134 clinical claims from open sources and mapped 20,611 of them onto a four-level scale of evidence strength, from "almost no data" to "confirmed by multiple studies." Then they asked 22 language models — from small ones, about the size of a desk book (0.6 billion parameters), to quite large ones (70 billion)—to rate each claim. The models were diverse: general, medical, and logic-focused. The result was surprising. When the researchers "peeked" inside the model with a special mathematical probe, the signal about the true strength of evidence read beautifully — on average with 71.8% accuracy (as if seven out of ten claims were correctly weighed). But when the same model was asked directly, "How reliable is this?"—the answers degraded to random guessing. The gap was 25–27 percentage points.
Curiously, this signal turned out to be largely lexical — tied to words and phrases rather than deep understanding. As soon as the model was moved to a different medical topic or a different rating system, the ability to read the signal vanished. Yet it did not coincide with a simple truth-versus-lie distinction: the model could correctly identify that a claim was poorly supported even if it was factually true. So inside the artificial brain lives an unspoken intuition for the fragility of arguments, but it doesn't come out.
This silent compass is both a hope and a warning. Hope, because the signal can be extracted and used even if the model itself stays silent. Warning, because today, when we read generated text, we don't hear the machine's inner voice, and we risk mistaking the shaky for the solid. The doctor at the monitor sees neat lines but doesn't know whether the ground beneath them trembles.
as written by the authors arXiv:2606.29034
Large language models (LLMs) increasingly summarize clinical evidence, where a claim's weight depends on how strongly it is supported. Yet these models convey confidence poorly, and properties they never state, such as truth, are often readable from their activations. Whether a clinical model registers evidence strength, distinct from truth, and states it when asked is untested, and any such signal could be lexical. We compiled 45,134 clinical claims from six public sources, harmonized 20,611 into a four-level evidence grade under three independent frameworks, and tested 22 local, open-weight LLMs from several developers (0.6-70 billion parameters; general, medical, and reasoning), with lexical, truth, and cross-framework controls. A linear estimator recovered the grade in every model (median AUROC 71.8), yet decodability did not rise with scale and was weakest in reasoning models. The grade the models stated fell to chance, 25-27 percentage points below the estimator. The recoverable signal was largely lexical and did not transfer across topics or frameworks, yet it was distinct from factual truth and still flagged weakly supported claims (AUROC 69.2). Clinical LLMs thus carry an ordered evidence-strength signal they do not express, so their stated grades fail to convey a claim's support even when it is recoverable from their representations and text.
The internal signal for evidence strength turned out to be strongest in general-purpose models, while specialized "medical" and "reasoning" models showed weaker signals, as if narrow specialization dulls universal intuition.