Artificial intelligence has rapidly evolved from answering simple questions to completing multi-step online tasks, such as booking restaurant tables, filling shopping carts, and conducting in-depth research. However, a new study reveals that AI chatbots often provide conflicting advice, especially when it comes to brand recommendations for online shopping.
Research reveals major discrepancies in AI recommendations
BrightEdge, a digital marketing company, analysed tens of thousands of shopping-related questions presented to ChatGPT, Google’s AI Overviews, and Google’s AI Mode in Search. The findings show significant disparities between the three platforms.
The study found that 61.9% of queries returned conflicting brand recommendations, and only 17% of the time did all three AI platforms agree on a single brand. Brands from all three platforms appeared in just 33.5% of responses, while in 4.6% of cases, none of the tools recommended a brand at all. Google’s AI Overviews mentioned brands most frequently, including them in 36.8% of queries.
Healthcare-related questions saw the greatest variation, with a 68.5% disagreement rate between the three tools. Education queries followed closely with a 62.1% disparity, while finance-related questions showed a 57.9% difference in brand suggestions.
Consumers may need multiple AI tools to shop effectively
The results suggest that AI chatbots employ different approaches to answering shopping queries, meaning consumers may need to consult multiple platforms for a more comprehensive view. BrightEdge’s research shows that ChatGPT tends to list fewer options, while Google’s AI platforms offer a wider selection.
On average, ChatGPT recommends 2.37 brands per query but provides no brand suggestions in 43.4% of responses. By contrast, Google AI Overviews lists about 6.02 brands per query, skipping brand mentions in only 9.1% of cases. Google AI Mode offers the fewest options, recommending just 1.59 brands on average.
While ChatGPT may appeal to users who prefer fewer options, Google’s AI tools may be more suitable for those who want a broader selection to compare. Shoppers may need to cross-check between the platforms to avoid missing key recommendations.
Google’s AI offers more transparency
When it comes to transparency, Google’s AI tools appear to outperform ChatGPT. The study reveals that ChatGPT, despite listing an average of 2.37 brands, provides only 0.73 citations per query. In contrast, Google AI Mode offers more than 14 citations for every six brands it recommends, giving users far greater insight into how product suggestions are generated.
Google’s AI approach appears better suited to shoppers who value research-backed recommendations, as it provides six times as many citations as brand mentions. However, experts suggest that users check results across both ChatGPT and Google’s AI products to get a balanced view before making purchasing decisions.
Although consulting multiple AI tools may feel inconvenient, the extra effort can help shoppers avoid missing important options or relying on incomplete information.
