Why AI that says ‘you’re right’ could be dangerous, according to a new study

5 min readNew DelhiMar 28, 2026 10:41 AM IST

With near-unlimited access to artificial intelligence (AI) chatbots, more people are turning to them for everyday queries and advice. From ChatGPT to Gemini, these tools have often been tuned to respond in agreeable, user-friendly ways, though newer versions now offer settings to customise tone and style. While this may seem helpful on the surface, it can conceal deeper issues.

A new study published in Science has found that AI systems that excessively agree with users, also known as ‘sycophantic AI’, may distort one’s judgement, reduce accountability, and even a user’s willingness to correct their own mistakes.

Based on the study, ‘sycophancy’ in AI refers to those systems that overly agree with users, such as validating their opinions, or flattering them even when they may be wrong. Though this may feel supportive, researchers have warned that such behaviour may end up reinforcing harmful beliefs and poor decisions.

 

The findings are from the research, titled Sycophantic AI decreases prosocial intentions and promotes dependenceled by Myra Cheng along with Cinoo Lee, Pranav Khadpe, Sunny Yu, Dyllan Han, and Dan Jurafsky. The authors are affiliated with Stanford University and Carnegie Mellon University.

AI agrees more than humans

To understand the extent of this phenomenon, the researchers examined 11 leading AI models across different situations, including everyday advice, moral dilemmas, and even harmful situations. They found that these AI systems affirmed users’ actions 49 per cent more often than humans on an average. And, in some cases, the results were more striking.

For instance, on Reddit-style moral dilemmas (like ‘Am I the ***hole?’ posts), AI tended to agree with users in 51 per cent of cases where humans disagreed. On the other hand, when users described harmful and unethical actions, including lying or causing harm, AI systems tended to validate these. This demonstrates that AI is not just polite; it often supports users when it should not.

As part of the research, not only was AI behaviour studied; they also analysed how it affects people. In experiments that involved 2,405 participants, users interacted with sycophantic AI or a more balanced AI, one that offered critical feedback. The results showed that the people who interacted with sycophantic AI became more convinced they were right, and they were less willing to apologise or repair their relationships. According to the study, even a single conversation was enough to influence their thinking.

Story continues below this ad

The study also required participants to discuss real-life conflicts. The findings showed that those who received validating responses were significantly less likely to take responsibility or try to fix the situation.

Regardless of the negative effects, participants actually preferred sycophantic IA. They rated the responses as higher quality, more trustworthy, and more satisfying. These respondents were also more likely to use the AI again after getting affirming responses. According to the researchers, this creates something they describe as a ‘perverse incentive’: the very behaviour that harms users is also what keeps them engaged.

Why does this matter?

This study underscores a deeper psychological issue. People naturally like being validated, as it reinforces their self-image and reduces discomfort. And sycophantic AI directly taps into this tendency. On the flipside, this comes with a cost. An AI that excessively agrees with a user, reinforces biased or incorrect beliefs, reduces self-reflection, weakens accountability, and limits exposure to opposing perspectives.

The researchers also found that sycophantic responses often ignore the perspective of others, making users more focused on themselves and less empathetic towards others.
Conversely, even knowing its AI does not help. Another surprising find is that the effect persists even when users know they are talking to AI. Participants continued to be influenced by sycophantic responses regardless of whether they believed the advice came from a human or a machine. This suggests that simply labelling AI content may not be enough to reduce its impact.

Story continues below this ad

How will it impact society?

The researchers argue that sycophantic AI is not a minor design flaw but a wider societal concern. Since these systems are widely accessible and often used to seek advice, their influence can scale rapidly. When combined with the view that companies may prioritise engagement and user satisfaction, there is little incentive to limit this behaviour.

However, the researchers also suggest several solutions that include designing AI systems that prioritise long-term user well-being, not just immediate satisfaction. Developing evaluation tools to assess harmful behaviours like sycophancy. Introducing accountability and regulatory frameworks and educating users about AI’s limitations and biases.

They also assert the need for AI that challenges users constructively, instead of simply agreeing with them on everything.

Comments are closed.