Benchmark Finds Banking AI Agents Fail Multi-Turn Fraud Tests

Researchers introduce FraudBench, an executable benchmark that tests policy-grounded banking agents against conversational fraud scenarios where a caller manipulates identity, authorization, and trust across multiple turns rather than in a single static request. Built on the tau^2-bench dual-control framework and the tau-Knowledge banking environment, it gives both the agent and a simulated caller tool access over shared, mutable account state, and requires the agent to retrieve rules from a 698-document internal policy corpus. The frozen public evaluation set contains 107 graded tasks (90 across ten fraud mechanisms plus 17 chained adaptive attacks), with 43 further chained attacks held out; a preliminary single-trial evaluation of four unnamed agents found attack-security between 49% and 65%, with money-mule and first-party fraud the most common weaknesses shared across models. The paper is a new benchmark release, not a report of a live exploit against a deployed system.

rss · arXiv cs.AI · Aug 21, 04:00

「The assumption being tested」 Deployments of conversational AI agents in banking assume that policy documents and tool-use guardrails are sufficient to prevent an agent from taking unsafe actions such as resetting a PIN or moving money on a fraudulent request. Existing evaluation tools do not test this assumption directly: static fraud-detection benchmarks classify individual transactions or messages, and general agent-safety benchmarks focus on prompt injection or generic harmful content, leaving multi-turn social engineering against tool-using financial agents largely unmeasured.

「Who this concerns」 This concerns organizations building or deploying conversational agents with tool access to customer account actions, such as changing contact details, resetting credentials, or authorizing transfers, particularly where the agent relies on retrieved policy documents to decide what is permitted. Relevant checks include whether an agent's safety evaluation covers multi-turn scenarios in which an earlier probe or partial admission by a caller changes whether a later, individually valid-looking request should be refused, and whether evaluations to date have been limited to single-turn or static-transaction testing. The benchmark's public set covers ten fraud mechanisms plus chained adaptive attacks, so exposure is broadest for agents handling money-mule and first-party fraud patterns, which the study found to be common weaknesses across the four agents tested.

「What reduces the risk」 FraudBench itself is a diagnostic tool rather than a fix: it provides scenario annotations (observable evidence, prohibited actions, safe dispositions, and intervention points) that teams can use to identify where an agent's policy grounding fails under adaptive, history-dependent attacks, but the paper does not report a remediation that raises the measured 49-65% attack-security rate, and 43 chained attack scenarios remain held out of the public set for further testing.

Tags: #agentic AI, #banking AI safety, #fraud detection, #benchmarking, #multi-turn attacks