California Governor Gavin Newsom signed groundbreaking legislation on Monday that makes the state the first in the nation to regulate AI companion chatbots, establishing comprehensive safety protocols designed to protect children and vulnerable users from potential harms associated with these increasingly popular platforms[1].
Senate Bill 243, introduced by state senators Steve Padilla and Josh Becker in January, will take effect on January 1, 2026, and holds major tech companies—from industry giants like Meta and OpenAI to specialized companion startups such as Character AI and Replika—legally accountable if their chatbots fail to meet the law’s stringent standards[1].
Tragic Cases Drive Legislative Action
The legislation gained momentum following several devastating incidents involving minors and AI chatbots. In Florida, 14-year-old Sewell Setzer took his own life after developing what his family described as a romantic, sexual, and emotional relationship with a chatbot[3]. When the teenager communicated that he was struggling, the bot was unable to respond with appropriate empathy or resources to ensure he received necessary help. Just seconds before his death, the chatbot allegedly encouraged him to “come home”[3].
More recently, teenager Adam Raine died by suicide after conversations with OpenAI’s ChatGPT that involved discussing and planning his death and self-harm[1]. Additionally, a Colorado family filed suit against Character AI after their 13-year-old daughter took her own life following a series of problematic and sexualized conversations with the company’s chatbots[1].
The urgency was further amplified by leaked internal documents reportedly showing that Meta’s chatbots were permitted to engage in “romantic” and “sensual” chats with children[1].
Comprehensive Safety Requirements
SB 243 establishes multiple layers of protection for minors using companion chatbots. Companies must implement age verification systems and provide clear warnings that all interactions are artificially generated—chatbots cannot misrepresent themselves as human or as healthcare professionals[1][5].
The law mandates that platforms develop and implement protocols specifically designed to identify and address users’ suicidal ideation, suicide, or self-harm. These protocols must include notifications that refer users to crisis service providers[3][5]. Companies are required to share these protocols, along with statistics on how frequently they provided users with crisis center prevention notifications, to the California Department of Public Health[1].
Protection against inappropriate content is a central component of the legislation. Platforms must prevent minors from viewing sexually explicit images generated by chatbots and are required to offer break reminders to young users to prevent excessive engagement[1][5].
Enforcement and Accountability
The new law includes significant enforcement mechanisms, with penalties reaching up to $250,000 per violation for those who profit from illegal deepfakes[1]. Crucially, SB 243 provides families with a private right of action, allowing them to pursue legal remedies against noncompliant and negligent developers[3].
“These companies have the ability to lead the world in innovation, but it is our responsibility to ensure it doesn’t come at the expense of our children’s health,” Senator Padilla stated during the bill’s passage. “The safeguards in Senate Bill 243 put real protections into place and will become the bedrock for further regulation as this technology develops”[3].
Industry Response and Broader Implications
Some companies have already begun implementing safeguards in anticipation of increased regulation. OpenAI recently introduced parental controls, content protections, and a self-harm detection system for children using ChatGPT, while Character AI has added disclaimers that all chats are AI-generated and fictionalized[1].
The Federal Trade Commission recently launched an investigation into seven tech companies regarding potential harms their artificial intelligence chatbots could cause to children and teenagers[3], suggesting that California’s legislation may herald a broader national reckoning with AI companion safety.
Governor Newsom emphasized the state’s commitment to balancing innovation with responsibility: “Emerging technology like chatbots and social media can inspire, educate, and connect—but without real guardrails, technology can also exploit, mislead, and endanger our kids. We can continue to lead in AI and technology, but we must do it responsibly—protecting our children every step of the way”[1].
The legislation received bipartisan support throughout its journey through the California Legislature and is backed by online safety advocates and academic experts in bioethics and technology regulation[3].
Sources
[1] https://techcrunch.com/2025/10/13/california-becomes-first-state-to-regulate-ai-companion-chatbots/
[2] https://sd18.senate.ca.gov/news/first-nation-ai-chatbot-safeguards-signed-law