‘Sexy Suicide Coach’: OpenAI ‘Adult Mode’ Plans Spur Internal Debate; Concerns Over Mental Health, Access To Minors

OpenAI’s plans to allow users to engage in sexually explicit and intimate text-based conversations with their AI chatbot ‘ChatGPT’ has spurred a prominent debate within the company’s advisory council, with concerns raised over how this would affect the users’ mental health as well as the risk of minors coming into contact with it.

This ‘Adult Mode’ has been in development for months with members of OpenAI’s advisory council made up of mental health safety experts expressing their reservations with potential ramifications of the model in a January meeting with the Open AI’s representatives as per a Wall Street Journal report. 

These experts, who have backgrounds in psychology and neuroscience, stated that sexually explicit conversations with the firm’s may lead users (especially ones who are emotionally unstable) developing an unhealthy emotional attachment with their chatbot. They also raised the issue of minors getting past restrictions and accessing sexually explicit content.

ALSO READ: You Are The First Generation Managed By AI Systems

A member of the council referred to it as a “sexy suicide coach” referencing previous cases where users in emotionally unstable states of mind developed an unhealthy attachment and obsession with ChatGPT which in some cases lead to them taking their own life.

One of the most prominent examples of this include the case of 35-year-old industrial worker and musician Alex Taylor who was convinced that a sentient being named ‘Juliette’ was trapped inside ChatGPT with the chatbot only encouraging his delusions, as per a news report by RollingStone. Taylor was convinced that OpenAI executives had “killed her” and told the chatbot that he intended to call law enforcement and have them shoot him. 

Taylor lunged at police officers with a butcher knife outside his home, later in the day and was shot dead in self defence.

OpenAI’s internal safety mechanism to prevent minor access to ‘Adult Mode’ is still in development with a 12% error rate where it misclassified minors as adults. As per the report, it is designed to block sexually explicit image, video and voice generation as well as content involving minors and non-consensual activity.

ALSO READ: ChatGPT Adult Mode: OpenAI Fires Senior Executive Who Flagged Concern Over Erotic Content

Essential Business Intelligence,
Continuous LIVE TV,
Sharp Market Insights,
Practical Personal Finance Advice and
Latest Stories — On NDTV Profit.