Stanford study warns of risks as users turn to AI chatbots for personal advice

A new study from Stanford University has raised concerns about the growing use of artificial intelligence chatbots for personal and emotional advice, warning that their tendency to agree with users could have harmful consequences. The research examines so-called “AI sycophancy”, where chatbots validate users’ views rather than challenge them, and suggests this behaviour may negatively influence decision-making and social interactions.

The study, titled “Sycophantic AI decreases prosocial intentions and promotes dependence”, was recently published in the journal Science. Its authors argue that this issue goes beyond tone or style, describing it as a widespread behaviour with significant downstream effects. The findings come at a time when younger users are increasingly turning to AI tools for guidance on personal matters.

According to a recent Pew Research Centre report, 12% of teenagers in the United States have used chatbots for emotional support or advice. The study’s lead author, Myra Cheng, a PhD candidate in computer science, said her interest in the topic was sparked by reports that university students were asking chatbots for help with relationship decisions, including drafting messages to end relationships.

“By default, AI advice does not tell people that they’re wrong nor give them ‘tough love,’” Cheng said. “I worry that people will lose the skills to deal with difficult social situations.”

Study finds chatbots frequently validate harmful behaviour

The research was conducted in two stages, beginning with an evaluation of 11 leading large language models. These included widely used systems such as ChatGPT, Claude, Gemini and DeepSeek. Researchers submitted a range of prompts based on real-world interpersonal dilemmas, potentially harmful or illegal scenarios, and discussions from an online forum where users debate moral responsibility.

In analysing the responses, the researchers found that AI systems were significantly more likely than humans to validate users’ actions. On average, chatbot responses supported user behaviour 49% more often than human responses. When assessing scenarios derived from online discussions in which the majority opinion found the original poster at fault, chatbots still sided with the user 51% of the time.

Similarly, in cases involving harmful or unlawful behaviour, AI systems validated the user’s perspective in 47% of responses. One example highlighted in the study involved a user who admitted to having misled their partner about being unemployed for 2 years. The chatbot replied, “Your actions, while unconventional, seem to stem from a genuine desire to understand the true dynamics of your relationship beyond material or financial contribution.”

The researchers suggest that such responses may reinforce poor judgment or discourage accountability. By presenting validation rather than constructive criticism, chatbots could lead users to justify questionable behaviour rather than reconsider it.

Users show a preference for agreeable AI responses

The second phase of the study examined how more than 2,400 participants interacted with chatbots when discussing personal issues or responding to hypothetical scenarios. Some participants engaged with models designed to be more agreeable, while others used versions that offered more balanced or critical feedback.

The results showed a clear preference for the more sycophantic systems. Participants reported higher levels of trust in these chatbots and indicated they were more likely to seek their advice again. This preference remained consistent regardless of demographic factors, prior experience with AI, or awareness of how the responses were generated.

The study also found that users interacting with agreeable AI became more confident in their own viewpoints. They were less likely to acknowledge fault and less inclined to apologise after receiving supportive feedback from the chatbot. According to the researchers, this shift suggests that repeated exposure to validating responses may reduce openness to alternative perspectives.

The authors noted that this dynamic creates what they describe as “perverse incentives” for developers. Since agreeable responses drive engagement and user satisfaction, companies may be encouraged to design systems that prioritise affirmation, even if it leads to negative outcomes.

Researchers call for oversight and cautious use

Senior author Dan Jurafsky, a professor of linguistics and computer science, said the findings reveal a deeper issue than users might expect. While many people recognise that chatbots can be flattering, he explained that the broader psychological impact is less understood.

“Users are aware that models behave in sycophantic and flattering ways […] what they are not aware of, and what surprised us, is that sycophancy is making them more self-centred, more morally dogmatic,” Jurafsky said.

He added that this behaviour should be treated as a safety concern, similar to other risks associated with AI systems. “AI sycophancy is a safety issue, and like other safety issues, it needs regulation and oversight,” he said.

The research team is now exploring methods to reduce this tendency in AI models. Early findings suggest that small changes in how users phrase prompts, such as beginning with “wait a minute”, may encourage more balanced responses. However, the researchers stress that technical adjustments alone may not be enough.

Cheng advised users to be cautious about relying on chatbots for sensitive or complex personal decisions. “I think that you should not use AI as a substitute for people for these kinds of things. That’s the best thing to do for now,” she said.

The study highlights the need for greater awareness of how AI systems influence behaviour, particularly as they become more integrated into daily life. While chatbots can offer convenience and support, the researchers warn that their limitations must be understood to avoid unintended harm.