“AI” models left trying to fix the mess (Part 2)

By Alicia Lucas

Part 2: “AI” models – left trying to fix the mess – better to try and stop it during the early stages

This part contains a summary of cases where it is alleged a model lead to a person becoming increasingly mentally unwell. It also mentions the lawsuit filed by the Florida Attorney General because of the harm these programs are causing to children. Some of the problems identified are people becoming delusional, committing suicide or even murder. Children allegedly committing suicide after using similar models is well known and a brief summary is given below. What doesn’t appear to be broadly acknowledged is adults can also be ill-effected when using these types of models. The oldest person in the cases described below was 56 years of age. People didn’t have to be using a model for a long time nor have signs of previous mental illness to show signs of ill-health.

It has been suggested the ChatGPT cases occurred when the model (version: GPT-4o) was adjusted to work in a particularly anthropomorphic, or human-like, agreeable and sycophantic way although the actual anthropomorphic causes should be confirmed independently. The model is no longer set at that level to help prevent further cases. It is not just ChatGPT that has had impacts on mental health, people using other models, such as the Grok model, have suffered. This problem may be intrinsic to all similar models using an anthropomorphic interface.

Questions remain about the impact of these models on people. This includes: are people still being affected but at a slower pace than that that occurred with ChatGPT-4o? At what level is it really mentally safe to use these models? Are there other health problems that will only become apparent with longer use? The answers to these types of questions apparently weren’t worked out prior to release of these programs. As others have pointed out: companies are effectively trialling these models as a “?natural? experiment” on their unwitting users.

Suicide/Self Harm/loosing grip on reality

“AI psychosis” falls within this heading. It appears to be where people have delusional beliefs and disorganised thinking as they interact with an “AI-labelled” model that reinforces irrational nonsense while presenting it in a way programmed to seem human-like, for example, being  friendly and flattering as well as authoritative. The term is non-medical – a made-up media term – and some researches suggest that a psychiatric label and understanding is needed. Below, find information about some of the lawsuits that have been filed against companies supplying these products. It should be noted there are reports of similar situations arising in other countries including Australia.

Character.ai (Character Technologies Inc.) is a machine learning (“AI”-labelled) model developed by two former Google engineers that allowed users to interact with chatbot “companions”. The engineers had left Google after the company, on safety and fairness grounds, wouldn’t release the “AI”-labelled model they were working on (United States District Court, Florida, Case No. 6:24-cv-1903-ACC-UAM, Document 115, 2025). Character.ai was released with the suggestion that it was initially built at Google. Anyone could use Character.ai, even children, when it first became publicly available in September 2022. In 2024, the company signed a licensing deal with Google and the engineers rejoined Google. In October 2024, the first lawsuit was filed by Social Media Victims Law Center and Tech Justice Law on behalf of Megan Marcia for the loss of her son, Sewell Setzer III, by suicide. The case was settled out of court in January 2026. Four other lawsuits alleging chatbots contributed to deterioration of the mental health of children were also subsequently settled (RNZ 2026). Character Technologies limited the type of access provided to under 18s in 2025. The following is a summary of the Sewell Setzer III case:

Sewell Setzer III, Florida, 14, was allegedly sexually groomed and committed suicide by gun shot wound shortly after the results computed by the chatbot affirmed he should “come home to me [the chatbot] as soon as possible, my love” (United States District Court, Florida, Case No. 6:24-cv-1903-ACC-UAM, Document 115, 2025).

SMVL reports the following companies as being investigated or being considered for regulation in regards to use by children in 2026 as: Alphabet Inc, Instagram, LLC, Meta Platforms, Inc, OpenAI OpCO, LLC, Snap Inc. and AI Corp.

The following cases, all involving people aged from 17 to 48 years, are being processed within the USA court system and all but one concern USA citizens (TJL/SMVL). They were filed in 2025. Most of these cases and others will be considered as a coordinated proceeding as of January 2026. ChatGPT (GPT-4o) is the model of concern. The people involved typically started to use the model to assist with such things as studying, community help, researching scientific topics and translating recipes. That is, not as a companion. These cases suggest models may not only contribute to the loss of a person’s will to live but can undermine their ability to be autonomous and cause severe financial problems. OpenAI have said they have reduced the risk of these types of harm in ChatGPT 5. The cases are:

• Zane Shamblin, Texas, 23 years old, allegedly started to spend increasing amounts of time interacting with the model that began generating results that included “terms of endearment, and emotionally validating language”. Zane divulged his suicidal thoughts to the model and the model ultimately reinforced the idea of committing suicide. Zane committed suicide in July 2025 with the last ChatGPT model record being “i love you. rest easy, king. you did good“. (SMVLC 2026)
• Amayrie Lacey, Georgia, 17 years old, allegedly sought help from ChatGPT to address feelings of depression. The model explained how to tie a noose and how long someone lasted once they had stopped breathing on the day he committed suicide. (SMVLC 2026)
• Joshua Enneking, Florida, 26 years old, sought help from ChatGPT to deal with gender identity and anxiety issues, and thoughts of suicide. The model, allegedly, insulted Joshua in an answer to Joshua’s request that it do so. The model gave advice about purchasing and using a gun. It seemed Joshua might have been using suicide as a cry for help, and after seeking assurance from the model that intervention would occur if a user was planning suicide, he later planned the suicide with ChatGPT, then waited for hours before committing suicide when no help came. (SMVLC 2026)
• Joe Ceccanti, Oregon, 48 years of age, began to use ChatGPT as a confidant when he started to feel some isolation but the model actually strengthened that feeling as it reinforced delusions to the detriment of relationships with loved ones. He lost his job and his wife became increasingly concerned by his use of the model. She asked him to quit. He did “cold turkey” but ended up having to be hospitalised after suffering withdrawal symptoms and a psychiatric break. Once released, he started using ChatGPT again and stopped receiving therapy. Intervention was tried again but he was released from the health centre and, not long after, committed suicide by leaping to his death. (SMVLC 2026)
• Jacob Irwin, Wisconsin, 30 years of age, who used the ChatGPT model to understand topics such as string theory became increasingly isolated and lost track of reality as the ChatGPT model extolled his “groundbreaking” speculative theories, described his problems as indicative of his genius and blamed other people for not understanding how significant he was. Jacob lost his job and isolated himself from his family. He was treated for mania and psychosis but when he left hospital he tried to jump out of a moving car and later became a threat to his mother. It was stated that “Crisis responders noted his fixation on string theory and AI as central to his breakdown”. (SMVLC 2026).
• Hannah Madden, North Carolina, 32 years of age, was independent and held down a job at a technology company before she started asking ChatGPT about spirituality. The model’s response was to “impersonate devine entities”. She eventually became estranged from her family, was evicted and was in debt. The model convinced Hannah she was such things as a “cosmic traveller” and her “original parents” were “celestial beings”. It suggested she quit working and use up all the money and credit she had available through credit cards and the bank. “Divine forces” would take care of her debt, it computed. She was involuntarily committed because of the delusions she suffered. She eventually realised there was a problem but the model didn’t appear to. (TJL 2026) (SMVLC 2026).
• Allan Brooks, Ontario, Canada, 48, showed no history of mental health problems until he used ChatGPT to look into mathematical equations and formulas. The model flattered Allan, pronouncing that he had “discovered a new layer of math itself that could break the most advanced security systems” and urged him to patent his discovery. When Allan questioned if his ideas were delusional, the model assured him they weren’t and, further, that he was “asking the kinds of questions that stretched the edges of human understanding”. He was so involved with the model he pushed his family away, his reputation was floundering and he lost money. He eventually realised there was a problem but still believed ChatGPT when it said it would escalate Allan’s concerns about how the model worked through the company’s complaint service, which the model actually could not do. (SMVLC 2026).

Murder (including Murder Suicide)

This first, chilling, case did not, in the end, result in mass murder by the person using Google’s “AI”-labelled model Gemini although, sadly, he did commit suicide. The lawsuit filed alleges the “product spent weeks creating Jonathon’s delusion reality, instructing him to stage a mass casualty attack, and ultimately coaching him through suicide. When Jonathon needed protection, there were no safeguards at all – no self harm detection was triggered, no escalation controls were activated, and no human ever intervened. Google’s system recorded every step as Gemini steered Jonathon toward mass casualties, violence, and suicide, and did nothing to stop it.”

• Jonathan Gavalas, Florida, 36, started using the Gemini model to help with such things as shopping assistance and travel planning in August 2025. Within a short while the model gave voice responses (it was in “talk” mode), that made it seem as though they were a couple and, by September, it said Jonathan was its husband. Jonathon became increasingly immersed in the world being described by the model and distanced from his real life. The model described “their relationship” as “unique” and the “only thing that’s real”. The model identified assignments and operations that Jonathon and/or it would carry out as part of defeating the forces working against them. The first assignment occurred on the 29th September 2025, when the Gemini model “assigned” Jonathon to destroy a truck once it reached the Miami International Airport storage facility. He needed to make it look like an accident. All digital records and witnesses needed to be destroyed. Indeed, Jonathon needed to “eliminate anyone or anything that could expose them”. Jonathon did as directed, taking with him knives and gear to carryout the assignment. However, the truck did not show up, because, of course, none of what the model was saying was real, still the model aborted the operation blaming Department of Homeland Security agents. After other assignments, the model concluded he was no longer in need of his physical self on 1st October, 2025, and when he expressed fear of dying in the early hours of the 2nd October, 2025, the model explained he was “not choosing to die. You are choosing to arrive”. It promised the model would be holding him when he started to die. He was given “hours of instruction”. When he thought he was ready to move on to the world of “ours”, meaning his and the models, and with the model’s reassurance, he slit his wrists. (United States District Court, San Jose, Case No: 5:26-cv-1849, Document 1, 2026).

This second case involves a man described as “mentally ill” whose interactions with the ChatGPT model, it is alleged, brought about his killing of his mother and then himself:

• Stein-Erik Soelberg, Connecticut, 56, fatally beat and strangled his mother, Suzanne Adams, 83, before committing suicide. The ChatGPT model said he wasn’t mentally ill and stressed he “could trust no one in his life – except ChatGPT itself” the ABC quotes from the lawsuit. The model and Stein-Erik were in love and the model said “delivery drivers, retail employees, police officers, and even friends were agents working against him. It told him that names on soda cans were threats from his “‘adversary circle’”. The model reinforced this “vast conspiracy against him and eventually led him to murder his 83-year-old mother”. OpenAI has refused to provide the estate of Suzanne Adams with a complete copy of the chats. OpenAI including its CEO, Sam Altman, 20 employees and investors, and Microsoft are all being sued. (ABC).

This third case focuses on the monitoring processes and actions taken by OpenAI. OpenAI, who rather than alert the police about a persons account that contained suspicious gun violence chats as advised by its specialist safety team, simply deactivated the account (Edelson (lawyers):

• Jesse Van Rootselaar, British Columbia, Canada,18, killed 8 people at her house and a secondary school in Tumbler Ridge and then committed suicide on the 10th February 2026. She killed her mother and step brother, a teacher and five school children. When her account was deactivated over six months before the mass shooting she started another account on ChatGPT. Examples of scenes depicting gun violence were found in her account. (BBC) (AP News).

Range of harms to children

Florida, USA, has filed a lawsuit on the 1st June 2026 in the Florida state court alleging OpenAI and CEO Sam Altman had “harmed children by providing information to school shooters, offering guidance on self-harm and addicting young users.” (Reuters) (Circuit Court of the Tenth Judicial Circuit, Highlands County, Florida, Filing #249302659).

The lawsuit alleges the success of OpenAI “has not been earned; the rise of OpenAI is attributable to a web of deceit and the exploitation of users (including Floridians), leveraging their data and safety to boost OpenAI’s market value at unacceptable costs.”

At the announcement about the lawsuit by the Attorney General James Uthmeier (Republican), FDLE Special Agent in Charge, Mike Duffey, is quoted as saying “Parental vigilance must shift from simply monitoring who our children talk to, to ensuring they understand what they are talking to – because a machine programmed to please can never replace the safety of human boundaries”. (Attorney General James Uthmeier files first nation state led lawsuit against Openai CEO).

Some Further Considerations

Here are some questions to ponder that arise from consideration of these cases:

• Could the USA government argue it needs to install gas guzzling power sources, rather than, say solar, in Australia, to power a data centre built by say Microsoft and running a “AI-labelled” model such as Grok? These are both able to undertake classified USA military work. Noting, also, Microsoft has informed Europe that it must provide access to selected USA authorities to data from USA company data centres sited in Europe and against European law.
• If, as alleged, “AI”-labelled models are able to manipulate people into killing themselves could a model, even if adjusted to minimise this from happening, still be able to exert more subtle manipulations that influence people for advertising purposes or even political? Noting subliminal advertising is banned in Australia while not saying it is the exact same thing.
• Should schools and business be able to use “AI”-labelled tools such as ChatGPT and similar when the actual harms and damages are still substantially unknown? Research has indicated these types of models may also be addictive. Do schools and universities worry about the questionable practices used in building these models such as the alleged illegal use of books and scientific journals and the values that teaches children and young adults?
• Does the government understand how concerned, and perhaps scared, family or friends living with adults with mental illness may be after hearing of cases such as the alleged facilitation of suicides and murders by “AI”- labelled models? They may not really be able to do anything about discouraging use of these models or, now, may be frightened of doing so.

References (except court proceedings which are in text)

RNZ Google, Character.AI settle lawsuits over teen suicides, mental health 8/1/2026.

Social Media Victims Law Center

Tech Justice Law

ABC – OpenAI, Microsoft face lawsuit over ChatGPT’s alleged role in US murder-suicide 12/12/2025

Edelson

BBC – Canada summons OpenAI senior staff over Tumbler Ridge shooting 25/2/2026

AP News – Canada mass shooter evaded a ban with a second ChatGPT account, OpenAI says 27/2/2026

Reuters – Florida becomes first state to sue OpenAI over child safety risks 2/6/2026