Visa: Leading AI models struggle to understand what customers want
In Visa's shopping tests of 22 AI models from 12 labs, agents failed 43% of their test runs. Even so, 54% of the failed runs ended in a purchase.
What's happening
According to Customer Experience Magazine, Visa's shopping benchmark passes an agent only if it learns all of the customer's requirements and then makes the right call to buy or walk away. Visa's data put the models' overall pass rate at 57%; the best models passed 82% of attempts, the weakest only 29%.
Where it comes from
Visa's own trust index found that 72% of US consumers have used an AI assistant, but only 23% trust AI to handle payment transactions. Separate research found that 58% of consumers are open to ordering through an AI assistant, yet only 6% have done so.
Whose desk it lands on
The executive in charge of digital channels. Does a shopping agent on your site ask about the customer's real needs before it buys?
Our view
The CX Brief's interpretation, not a claim made by the source.
The real test for a shopping agent is how much it can learn about the customer before it spends any money. In Visa's test, the five models that drew out the most hidden customer needs over the course of the conversation averaged a 70% success rate, while the five that drew out the fewest managed only 44%. What needs measuring is not just an agent's authority to buy but how many needs it uncovers before it spends.
Source
Source:Visa·Sample: 22 AI models from 12 labs
Reported by: Customer Experience Magazine·November 10, 2026
Conflict of interest: This is Visa's own research, not a study by an independent research organization.
Read it on Customer Experience Magazine →
