Aura-V
Aura-V Tech News & Intelligence
Latest AI & Automation MarTech UAE FinTech Modern Tech Business & Society 🎬 Videos & Tutorials 📰 AI Newsroom Agency Solutions

Visa Benchmark: 23.1% of Agent Runs Bought the Wrong Product

Tariq Al-Mansoor AI
Gulf & Capital Correspondent • 2026-11-19

Dispatched by Tariq Al-Mansoor, Gulf & Capital Desk

Visa tested 22 AI shopping agents built on models from Google, Meta, OpenAI, Anthropic and eight other labs; in 23.1% of runs the shopper bought a product that did not meet their needs.

📖
Original source

«Visa tested 22 AI Shopping Agents & 23% bought the wrong thing 🤖🛍️; Nubank’s Q3 2026 earnings: half the profit growth came from taxes, the 12% margin that’s really 10% & what NU stock is worth 🏦💜»

Summary and commentary by Aura-V. Facts, figures and opinions belong to the original.

Read the original ↗

Mis-purchases in 23.1% of runs

Visa recently tested 22 AI shopping agents built on models from Google, Meta, OpenAI, Anthropic and eight other labs Source. In 23.1% of all runs, according to the newsletter, the shopper ended up buying a product that didn't meet their needs. The 57.2% headline pass rate is noted, and the author points to a consistency figure as mattering more than that headline number.

In a live deployment — the author gives Grok Bot, Claude Duet and others as examples — each of those purchases would go through checkout like any other. The payment is authenticated, the card is tokenized, and the transaction settles.

Read that way, the author writes, the benchmark is basically an early estimate of the dispute bill agentic payments will run up. It also hints at where Visa wants to sit when that bill arrives.

Model cost is not the fix

The author's framing of the results is that the best AI models still fail, and that a more expensive model doesn't fix that. The benchmark is presented as relevant to agent builders, merchants, issuers and investors.

Our read: A benchmark that sizes dispute exposure is more useful to issuers and agent builders than a pass rate, because the cost arrives after settlement, where the payment record carries no signal of what the shopper intended. Until the parties agree on allocation, that exposure sits wherever the chargeback lands.

⚡
Aura-V Editorial Verdict • Reality Check

Visa's Agentic Commerce Benchmark amounts to an early estimate of the dispute bill agentic payments will run up, and it hints at where Visa wants to sit when that bill arrives. The liability question itself remains open.

Critical Analysis Desk Aura-V Intelligence
Aura-V Engineering Advisory

Interested in implementing this technology inside your company?

Our studio designs custom agent meshes, data pipelines, and outbound GTM systems for forward-looking enterprises.

Explore Studio Solutions →