Why a Neural Network That Reads Your Pulse from Your Face Doesn't Always Explain Its Decisions ✛
A treasure map drawn by a neural network promises to show where the gold is buried. But if you take that map to a different island, the marks may point to nothing. Similarly, the explanations of a model that estimates heart rate from facial video don't always match actual reliability.
The RhythmFormer neural network picks up on subtle color changes in the skin caused by heartbeats. Researchers applied four explanation methods and measured skin coverage (the fraction of the model's attention on the face) and fidelity coefficient (how well the importance ranking of regions matches their true influence). Within a single participant, these metrics did not predict pulse error. But across recording conditions, skin coverage from the Beyond Intuition method moved in step with estimation quality.
The treasure map doesn't show where the gold is buried for a specific person. Explanations carry extra information, but they don't replace accuracy metrics. Developers of remote pulse monitoring systems should interpret explanations cautiously as an indicator of reliability.
In the dimmest lighting (40 lux), the Beyond Intuition model stopped focusing on skin, and its explanations became worse than random. Meanwhile, motion, which degraded pulse accuracy more severely, did not cause such a breakdown in explanations.
The idea of reading a pulse from the face without contact echoes the medical scanners of science fiction, such as the tricorder from 'Star Trek,' which remotely assesses vital signs. The question of whether we can trust AI explanations also arises in 'Westworld,' where the behavior of artificial intelligence remains opaque to its creators.