Explaining AI: Where the Model Looks for a Pulse, and Whether You Can Trust It ✛
Context
Remote photoplethysmography (rPPG) extracts heart rate from ordinary facial video, but the pulse-induced skin color change is subtle and easily masked by lighting, motion, and camera characteristics. Existing datasets differ in recording conditions, making model transfer between them a nontrivial task. Deep neural networks such as RhythmFormer achieve high accuracy, but their internal logic remains opaque. Previous studies explained model behavior only visually, without quantitatively verifying whether the highlighted regions actually influence the prediction. This work aligns with research on EEG-based seizure detection and heart rate analysis, but stands out by being the first to systematically test the transfer of explanations across datasets and their connection to model reliability.
Methods
The authors trained eight independent RhythmFormer models, one for each condition in the NCKU-rPPG dataset: three illumination levels (40, 200, and 700 lux), speech, head rotation, and three cycling modes. For each 5.12-second clip, they estimated heart rate. They then applied four explanation methods to the models: raw attention, attention rollout, attention flow, and Beyond Intuition. Explanations were quantitatively evaluated using skin coverage (the fraction of attribution falling on the face mask) and the sufficiency coefficient SaCo, which measures how well the order of region importance agrees with the actual effect of masking those regions on the prediction. For comparison, they used the public UBFC-rPPG dataset.
Results
The Beyond Intuition method achieved the highest median skin coverage and SaCo on both datasets: 0.789 and 0.837 on Static level 3 versus 0.826 and 0.917 on UBFC-rPPG. However, within a single participant, neither skin coverage nor SaCo correlated with heart rate error, waveform correlation, or signal-to-noise ratio: 186 of 252 coefficients were below |ρ|=0.10, and only 28 reached p<0.05, compared to 13 expected by chance. At the level of eight scenarios, only Beyond Intuition skin coverage tracked quality metrics: ρ=−0.43 for error, +0.57 for waveform correlation, and +0.43 for SNR. Beyond Intuition degraded sharply at 40 lux: median skin coverage dropped to 0.180 and median SaCo to −0.178, whereas motion degraded estimates more severely but without such a collapse in explanations.
Implications
This work differs from neighboring studies by introducing quantitative measures instead of visual heatmap assessment and testing their transfer across datasets. It shows that skin coverage and SaCo carry information complementary to accuracy metrics, but they are not a substitute. Attribution to skin does not guarantee accurate pulse estimation. This refines the interpretation of explanations in physiological monitoring tasks and underscores the need for caution when using XAI as a reliability indicator.
Future development
The topic can be extended by testing the same methods on other neural network architectures, since all conclusions are drawn only for RhythmFormer. Retraining with different random seeds and comparing models trained on combined conditions would help separate condition effects from randomness. It is also important to align explanation methods to a common spatial resolution and directly measure multi-hop attention leakage on Static level 3.
Impact
The results will influence the development of remote heart rate monitoring systems, especially in clinical applications where interpretability matters. They will also set a standard for quantitative evaluation of explanations in computer vision for physiological signals.
Next steps
Repeat training with multiple random seeds and test explanation transfer to other architectures. Conduct controlled experiments with identical preprocessing on NCKU-rPPG and UBFC-rPPG to directly test transfer.
Key open problems
This work directly addresses the unresolved problem of interpretability in deep neural networks for medical imaging: how to distinguish plausible explanations from truly correct ones. It also touches on model reliability under changing recording conditions, which is critical for translating laboratory results into real clinical practice.
The knowledge machine's view
This section is addressed to the author of the paper and their colleagues. Neighbouring works were found by meaning-based search across our archive, not by word overlap. We do not review: we show the field around the work.
What the work does
The authors trained eight RhythmFormer models on the NCKU-rPPG scenarios and applied four explanation methods, evaluating skin coverage and the SaCo fidelity coefficient. They tested the transfer of explanations to UBFC-rPPG and their relation to pulse error at both the participant and scenario levels.
Where its strength is
This is the first quantitative check of explanation transfer across datasets and their link to accuracy metrics, rather than just a visual assessment of heatmaps.
Where it could go next
- It might be interesting to compare the behavior of RhythmFormer's explanations with approaches where models check their own predictions without external labels: a related study describes a neural network self-check mechanism that could complement the analysis of explanation fidelity.based on: Round Trip: How a Neural Network Checks Itself Without Hints arXiv:2608.00675v1
- Nearby there is work on how neural networks see the world similarly: comparing attribution maps across different architectures or conditions could show how specific RhythmFormer's conclusions are to this model.based on: When Neural Networks See the World the Same Way arXiv:2604.21836
- Since explanations degrade sharply in low light, one could look at how a neighboring study tackles noise in physiological signals: perhaps their technique for weak signals could help improve attribution under challenging conditions.based on: How a Neural Network Eavesdropped on the Noise of Life arXiv:2606.13475
Worth trying
- Test whether the model's self-checking correlates with skin coverage and SaCo.
- Compare RhythmFormer's attribution maps with those of other rPPG models on the same data.
- Apply noise suppression methods from a related study to low-light videos and see if explanations recover.
Why this may matter
If explanations can transfer between datasets and be tied to reliability, this provides a tool for monitoring rPPG systems in real-world settings where errors could be dangerous.
Nearby works in our archive
- Predicting a Vessel Rupture in Secondssimilarity 0.55
- Round Trip: How a Neural Network Checks Itself Without Hintssimilarity 0.55
- Peeking Inside the Mind of a Weather AIsimilarity 0.54
- Smartwatches vs. Sugar: Why the Accuracy Was a Miragesimilarity 0.53
- Blood Pressure on a Tiny Chipsimilarity 0.51
- How a Neural Network Eavesdropped on the Noise of Lifesimilarity 0.51
- When Neural Networks See the World the Same Waysimilarity 0.50
The archive is sparse around this work: the closest review is only 0.55, against a usual 0.64. It looks like few have drilled nearby.
Under the dimmest lighting (40 lux), the Beyond Intuition model stopped focusing on skin, and its explanations became worse than random. Meanwhile, motion, which degraded pulse accuracy more severely, did not cause such a breakdown in explanations.
The idea of reading a pulse from the face without contact echoes the medical scanners of science fiction, such as the tricorder from Star Trek, which remotely assesses vital signs. The question of whether to trust AI explanations is also raised in Westworld, where the behavior of artificial intelligence remains opaque to its creators.
Key numbers
- median skin coverage of Beyond Intuition on Static level 3: 0.789
- median SaCo of Beyond Intuition on UBFC-rPPG: 0.917
- coefficients below |ρ|=0.10 out of 252: 186
- median skin coverage of Beyond Intuition at 40 lux: 0.180
- heart rate error on Static level 3: 1.96 bpm