📄 Showing the "Simple" version — "Advanced" is not ready yet. Add it to favorites to help prioritize it. Ask for the full read

An ECG is not just a wavy line on paper. Like a diary, it records the state of the entire heart: rhythm, wall thickness, and hints of future problems. But currently, programs that learn to read this diary are trained on very narrow tasks — mostly detecting arrhythmias. That's like teaching someone to understand a book by showing them only a list of typos.

The authors decided to check how honestly such programs are evaluated. They took five popular training approaches and ran them through six different exams: three standard ones (the arrhythmia-focused ones) and three new ones — detecting structural heart diseases, estimating intracardiac pressure, and predicting patient outcomes. It turned out that on the old exams, all performed well, but on the new ones, the picture changed. Moreover, a random set of numbers that never learned anything performed no worse than trained algorithms on many tasks. That's like a person who doesn't know the language guessing the book's meaning from the cover — and hitting the mark as often as an expert.

What do the authors propose? First, expand the list of tasks: add detection of structural heart changes (e.g., wall thickening) and prognosis — what will happen to the patient over time. Second, always compare results against a simple random "student." If a smart program doesn't beat the dummy, its success means nothing. It's like in sports: a record counts only if it beats a novice's result.

This approach could transform the field: instead of chasing impressive numbers on narrow tests, researchers would tackle problems that truly matter to doctors and patients. An ECG is too valuable a source of information to waste on guessing typos.

Original abstract

as written by the authors arXiv:2602.17531

This position paper argues that current benchmarking practice in 12-lead ECG representation learning must be fixed to ensure progress is reliable and aligned with clinically meaningful objectives. The field has largely converged on three public multi-label benchmarks (PTB-XL, CPSC2018, CSN) dominated by arrhythmia and waveform-morphology labels, even though the ECG is known to encode substantially broader clinical information. We argue that downstream evaluation should expand to include an assessment of structural heart disease and patient-level forecasting, in addition to other evolving ECG-related endpoints, as relevant clinical targets. Next, we outline evaluation best practices for multi-label, imbalanced settings, and show that when they are applied, the literature's current conclusion about which representations perform best is altered. Furthermore, we demonstrate the surprising result that a randomly initialized encoder with linear evaluation matches state-of-the-art pre-training on many tasks. This motivates the use of a random encoder as a reasonable baseline model. We substantiate our observations with an empirical evaluation of five representative ECG pre-training approaches across six evaluation settings: the three standard benchmarks, a structural disease dataset, hemodynamic inference, and patient forecasting.