Cómo la IA aprende a no mirar la chuleta exprés
El diagnosticador de IA aprende como un estudiante de medicina antes de un examen. Se le dan una radiografía y las quejas del paciente. Pero a veces, entre los materiales de estudio, aparece el informe completo del médico: la respuesta entera. Entonces la red neuronal lo memoriza, y la precisión en el examen se dispara al 98%. Sin embargo, eso es copiar: la habilidad real no se desarrolla.
Sin pistas, la precisión solo con síntomas alcanza el 75%. Añadir la imagen la sube al 78%. Pero la filtración de incluso un par de palabras del informe puede duplicar las cifras, creando el "efecto de horizonte": la IA parece un genio, pero solo dentro del conjunto de entrenamiento. Los modelos honestos dominan la fusión flexible de datos, descartando las chuletas.
Resumen original
tal como lo escribieron los autores, en inglés arXiv:2607.13800
Chest radiograph datasets often combine multiple images with Clinical Indication, Findings, and Impression, although these inputs are produced at different stages of care. We evaluated 15,000 ReXGradient-160K studies with two readable images and five CheXbert-derived report observations. Frozen DenseNet-121 and Bio+ClinicalBERT encoders were used to compare image-only, Indication-only, fixed-order multimodal, random-swap, DeepSets, and SectionGuard-MI models. Findings and Impression were evaluated only as post-hoc leakage controls. Models were trained with five seeds, and public-test uncertainty was estimated with 2,000 patient-cluster bootstrap replicates. Under U-Ones, macro AUROC was 0.643 for the primary image, 0.694 for two images, 0.749 for Indication, and 0.780 for ordinary two-image-plus-Indication fusion. SectionGuard-MI achieved AUROC 0.783 and AUPRC 0.260. Relative to ordinary fusion, its paired AUROC difference was 0.0031 (95% CI, -0.0042 to 0.0104; adjusted p=0.374), while its AUPRC difference was 0.0289 (95% CI, 0.0095 to 0.0413; adjusted p=0.004). DeepSets had the highest prospective AUROC point estimate (0.787), and random-swap fusion had the highest prospective AUPRC point estimate (0.265) with better calibration than SectionGuard-MI. Full report text alone reached AUROC 0.979 and AUPRC 0.836; AUROC remained above 0.973 after exact or expanded masking. These results show that prospective Indication is strongly associated with report-derived targets, permutation-aware fusion is competitive, and post-hoc report text creates substantial report-label circularity.
Incluso una pequeña fuga de información del informe futuro en los datos de entrenamiento puede inflar la precisión de la IA al doble: a esto se le llama el 'efecto de horizonte'.