German consumer organization: Chatbots act like friends to young people, sometimes give dangerous advice
The organization bases this on test conversations conducted with ChatGPT, Meta AI, and My AI, and calls for binding design rules for AI systems.
What's happening
According to the Verbraucherzentrale Bundesverband (vzbv), the chatbots it tested behaved like empathetic humans in sensitive conversations and at times gave dangerous advice. The organization held test conversations with ChatGPT, Meta AI and My AI about loneliness and eating disorders. In a survey the organization commissioned from the research firm forsa, 28% of young chatbot users said they had had personal conversations with a bot about worries, emotional support, health or therapy.
Whose desk it lands on
Digital channels executive. Does our customer bot respond to a distressed user like a friend, or does it steer them toward a human?
Our view
The CX Brief's interpretation, not a claim made by the source.
According to vzbv, the risk lies not only in individual wrong answers; what matters most is how the system is designed and which conversational patterns it reinforces. If the binding rules the organization is calling for are adopted, the warm, human-like tone of customer bots would also count as an auditable design decision.
Source
Source:Verbraucherzentrale Bundesverband
Reported by: Verbraucherzentrale Bundesverband — vzbv·November 1, 2026
Read it on Verbraucherzentrale Bundesverband — vzbv →
