Standard Metrics Mask Clinical Error-Detection Failures in LLMs

Researchers evaluated 15 LLMs on 4 standardized clinical error-detection test sets spanning 3 languages, where benchmarks are built by injecting errors into notes so each erroneous note has a matching clean counterpart. Using a pairwise discrimination test that exploits this paired structure, 13 of the 15 models performed worse than random chance, even though the same models achieved F1 scores that standard practice would interpret as moderate performance. A further evidence-scoring analysis found that models consistently identified the error-relevant content in a note but failed to reach the correct verdict on the matched clean counterpart. The paper also reports that bias patterns are language-dependent: a given model may default to reporting "no error" in one language while over-flagging errors in another, and shows that F1 and pairwise accuracy can be pushed in opposite directions by the same underlying bias, so ranking models by F1 can favor the weakest discriminators. This is a benchmarking study; no production clinical deployments were tested.

rss · arXiv cs.AI · Aug 18, 04:00

「Background」 Clinical error-detection benchmarks are usually scored with aggregate metrics such as F1 or balanced accuracy, on the assumption that these metrics reliably reflect a model's ability to distinguish erroneous notes from correct ones. This assumption has underpinned decisions about whether LLMs are ready to assist with clinical documentation review, a safety-critical use case where undetected errors can affect patient care.

「Who Is Affected」 Organizations building or evaluating clinical NLP tools that rely on F1 or balanced accuracy as the primary evidence of an LLM's error-detection competence are in scope, regardless of which specific model is used, since the finding spans 15 diverse models across 4 benchmarks and 3 languages. Teams deploying such tools in non-English clinical settings face compounded risk, since the study found bias direction differs by language for the same model. Anyone relying on published leaderboard rankings for clinical error detection should check whether those rankings were produced using paired or pairwise evaluation rather than aggregate metrics alone.

「Mitigation」 The authors recommend supplementing aggregate metrics with paired, pairwise evaluations in benchmark reporting, and provide code and analysis scripts for this procedure; organizations evaluating or already using clinical error-detection LLMs can apply this pairwise test to existing benchmark data to check whether reported F1 scores conceal below-random discrimination.

Tags: #clinical NLP, #LLM evaluation, #benchmark validity, #healthcare AI, #multilingual bias