Benchmark Finds Chatbots Misjudge Youth Mental-Health Risk Despite Vocabulary Fluency

A new benchmark evaluates Claude, GPT-4o, and Llama-3.1 on two test sets: 64 Gen Alpha mental-health expressions validated by native speakers (ICC=0.72) and clinicians (kappa=0.78), and 75 multi-turn conversations (780 turns) presented in paired Standard and Gen Alpha phrasing. The models recognized 76-82% of the vocabulary but correctly calibrated clinical risk in only 64-72% of cases, a 10-14 percentage point vocabulary-comprehension gap (p<.001, d>0.48) that human therapists did not show (3pp, p=.22). The gap widened with ambiguity (7pp to 18pp), and six failure patterns were identified, including sarcasm masking and minimization acceptance, which compound to produce a 94% miss rate when three or more co-occur. The authors report lightweight mitigations failed to close the gap and that only heavy scaffolding reached human-level performance, at roughly 6.4 times the cost; they estimate a 34% baseline miss rate translating to about 146,880 missed crises annually, though this projection depends on the study's usage assumptions.

rss · arXiv cs.CL · Aug 24, 04:00

「Why this control was assumed to hold」 General-purpose and therapy-branded chatbots are increasingly used by adolescents as informal mental health resources, with the paper citing 13.1% of U.S. adolescents using generative AI for this purpose. Safety tuning in these models is typically validated against adult or generic clinical language, on the assumption that broad vocabulary comprehension implies reliable risk detection; this benchmark tests that assumption directly against Gen Alpha speech patterns such as hyperbole, ironic positivity, and rapid semantic drift.

「Who this affects」 This is relevant to organizations deploying LLM-based chatbots, whether marketed as therapy apps or general assistants, in contexts where adolescents may disclose mental health concerns, including the specific model families tested (Claude, GPT-4o, Llama-3.1) and likely other LLMs trained similarly. Exposure is broader for products without human-in-the-loop review, without youth-specific safety validation, or without monitoring for informal register and slang; it is narrower for deployments that already route self-harm or crisis language to human reviewers regardless of model confidence. Teams should check whether their safety evaluation sets include contemporary youth slang and sarcasm, and whether risk calibration, not just intent recognition, is measured separately from vocabulary understanding.

「What reduces the risk」 The study finds lightweight prompt-level mitigations insufficient and reports that only heavy scaffolding, at substantially higher operating cost, closes the gap to human-level performance; no vendor-side fix is described. The authors recommend mandatory human-in-the-loop architectures for youth-facing mental health AI, quarterly youth-specific validation cycles, transparent disclosure of model performance, and regulatory frameworks, none of which are currently standard practice according to the paper.

Tags: #AI safety, #mental health chatbots, #LLM evaluation benchmark, #youth safety, #clinical NLP