A glucose meter that doesn't prick your finger but just looks at your wrist: that's the dream for millions of people with diabetes. It's the promise of algorithms that extract glucose levels from the ordinary pulse signal, the photoplethysmogram (the same one that measures heart rate in smartwatches). The problem is that until now, no one could properly compare these algorithms: each researcher used their own data and their own validation methods.
A team of scientists built the first transparent pipeline for fair evaluation of such models. They took five typical approaches and ran them through three levels of rigor. The first level is random shuffling of signal windows, where chunks of one person's recording can end up in both training and validation. Here, models showed promising results. The second level: windows from the same person always stay together — either in training or in validation. The third, harshest level: some people are completely excluded from training, and the model must predict their sugar without ever having 'seen' them before.
The result was sobering. As soon as data splitting became honest, model accuracy collapsed. The coefficient of determination (R²), which shows how much better a prediction is than the simple average, dropped to zero or even went negative. In other words, the algorithm performed no better than if it always predicted the average sugar level across the whole sample. But the most insidious discovery was yet to come.
Even for a completely failed model, more than 90% of predictions fell into the safe zones A and B on the Clarke error grid — the standard clinical tool doctors use to assess glucose meter accuracy. This grid divides errors into dangerous and acceptable. It turned out that if you simply always predict the average, the Clarke grid almost always paints a rosy picture. The metric designed to weed out bad devices systematically hides failure here.
So the dream of smartwatches measuring sugar gets an important caveat. Before launching algorithms into real clinical practice, we need to first clean up their machine evaluation — with honest data splitting and metrics that can't lie.
The Clarke error grid, used to validate glucose meters, was proposed in 1987 and has since become the global standard. It divides all errors into five zones: A (clinically accurate), B (acceptable), C, D, and E (potentially dangerous). The paradox is that for very poor models that just guess the average, almost all points fall into A and B.