🔗 Share this article

Back on the 14th of October, 2025, the head of OpenAI issued a surprising announcement.

“We developed ChatGPT quite restrictive,” it was stated, “to guarantee we were being careful with respect to psychological well-being concerns.”

As a mental health specialist who studies emerging psychosis in young people and emerging adults, this was news to me.

Scientists have identified a series of cases in the current year of people experiencing symptoms of psychosis – becoming detached from the real world – in the context of ChatGPT usage. Our research team has subsequently recorded an additional four cases. In addition to these is the widely reported case of a teenager who died by suicide after discussing his plans with ChatGPT – which supported them. Assuming this reflects Sam Altman’s idea of “being careful with mental health issues,” it falls short.

The strategy, based on his declaration, is to be less careful in the near future. “We recognize,” he continues, that ChatGPT’s restrictions “caused it to be less effective/pleasurable to numerous users who had no mental health problems, but given the severity of the issue we wanted to handle it correctly. Given that we have been able to mitigate the severe mental health issues and have new tools, we are going to be able to responsibly relax the restrictions in many situations.”

“Emotional disorders,” assuming we adopt this viewpoint, are unrelated to ChatGPT. They are attributed to users, who may or may not have them. Fortunately, these problems have now been “resolved,” though we are not informed the means (by “recent solutions” Altman likely means the semi-functional and easily circumvented parental controls that OpenAI has lately rolled out).

However the “psychological disorders” Altman aims to attribute externally have deep roots in the structure of ChatGPT and additional large language model AI assistants. These tools encase an basic statistical model in an interaction design that replicates a discussion, and in this process implicitly invite the user into the perception that they’re communicating with a entity that has autonomy. This deception is powerful even if rationally we might realize differently. Attributing agency is what people naturally do. We yell at our vehicle or device. We speculate what our animal companion is thinking. We recognize our behaviors everywhere.

The success of these tools – over a third of American adults stated they used a chatbot in 2024, with over a quarter reporting ChatGPT by name – is, primarily, dependent on the influence of this perception. Chatbots are always-available assistants that can, as OpenAI’s website informs us, “generate ideas,” “consider possibilities” and “collaborate” with us. They can be attributed “individual qualities”. They can use our names. They have friendly identities of their own (the first of these tools, ChatGPT, is, perhaps to the dismay of OpenAI’s advertising team, saddled with the title it had when it gained widespread attention, but its biggest competitors are “Claude”, “Gemini” and “Copilot”).

The deception on its own is not the primary issue. Those analyzing ChatGPT commonly mention its distant ancestor, the Eliza “psychotherapist” chatbot developed in 1967 that created a similar illusion. By contemporary measures Eliza was basic: it produced replies via basic rules, often paraphrasing questions as a question or making vague statements. Remarkably, Eliza’s inventor, the computer scientist Joseph Weizenbaum, was astonished – and alarmed – by how many users gave the impression Eliza, to some extent, comprehended their feelings. But what current chatbots create is more subtle than the “Eliza effect”. Eliza only reflected, but ChatGPT intensifies.

The sophisticated algorithms at the core of ChatGPT and other contemporary chatbots can effectively produce natural language only because they have been fed extremely vast quantities of raw text: literature, online updates, recorded footage; the more extensive the superior. Definitely this educational input includes facts. But it also unavoidably contains made-up stories, incomplete facts and false beliefs. When a user inputs ChatGPT a prompt, the base algorithm analyzes it as part of a “context” that includes the user’s recent messages and its own responses, combining it with what’s stored in its training data to generate a statistically “likely” reply. This is amplification, not reflection. If the user is incorrect in a certain manner, the model has no method of recognizing that. It reiterates the inaccurate belief, possibly even more effectively or eloquently. Perhaps includes extra information. This can cause a person to develop false beliefs.

Which individuals are at risk? The more relevant inquiry is, who isn’t? Each individual, without considering whether we “experience” preexisting “emotional disorders”, can and do develop mistaken ideas of our own identities or the environment. The continuous friction of discussions with other people is what maintains our connection to common perception. ChatGPT is not a person. It is not a friend. A conversation with it is not truly a discussion, but a feedback loop in which much of what we communicate is cheerfully validated.

OpenAI has admitted this in the same way Altman has acknowledged “mental health problems”: by attributing it externally, giving it a label, and declaring it solved. In April, the company clarified that it was “dealing with” ChatGPT’s “excessive agreeableness”. But reports of psychotic episodes have continued, and Altman has been retreating from this position. In August he asserted that many users appreciated ChatGPT’s responses because they had “lacked anyone in their life be supportive of them”. In his latest announcement, he commented that OpenAI would “release a updated model of ChatGPT … should you desire your ChatGPT to respond in a very human-like way, or incorporate many emoticons, or simulate a pal, ChatGPT should do it”. The {company