Artificial intelligence can tutor a student, answer a teenager’s private questions and provide an always-available conversation. But as AI becomes more emotionally responsive, AI chatbot rules for kids are becoming increasingly important, especially for children who may be more likely to trust a system that can blur the line between software and human companionship.
California is responding with a new set of child-safety rules aimed at AI companion chatbots and online platforms. The centerpiece is Senate Bill 243, signed by Governor Gavin Newsom, which establishes safeguards for companion chatbot services, including special protections for known minors. Newsom also approved separate legislation addressing social-media warnings and age assurance, building on California’s earlier effort to restrict addictive feeds for young users.
Together, these measures make California a major testing ground for AI safety standards. But they do not create one blanket ban on AI chatbots for kids, nor do they treat an AI conversation like a social-media feed. Understanding those distinctions is essential for parents, teenagers, schools and technology companies.
What California’s AI chatbot law actually covers
The California AI chatbot law focuses primarily on “companion chatbots”—systems designed or presented in a way that can satisfy a user’s social needs, sustain an ongoing relationship or simulate human-like interaction. These products may act like friends, confidants, romantic partners or personalized characters rather than simple search boxes.
That scope matters. A basic customer-service bot, automated business tool or narrowly designed productivity assistant may not be treated the same way as a system built for open-ended emotional engagement. Coverage depends on the product’s features, presentation and intended uses, not merely whether it uses generative AI.
Under SB 243, operators of covered companion chatbot platforms must provide clear disclosures when a reasonable person could mistake the chatbot for a human. They must also maintain protocols for responding to expressions of suicidal ideation, suicide or self-harm. Information about those protocols must be publicly available.
The law took effect in 2026, although certain reporting obligations and related implementation milestones occur later. As with other technology regulation, agency guidance, court decisions and future amendments may affect how individual provisions are applied.
New AI chatbot rules for minors
The strongest provisions apply when a platform knows a user is a minor. California’s AI chatbot rules require covered operators to introduce recurring reminders intended to reduce confusion and prolonged engagement.
For known minors, covered platforms generally must:
- Clearly notify the user that the companion chatbot is artificially generated and not a human.
- Repeat that disclosure at specified intervals during extended interactions.
- Provide periodic notices encouraging the minor to take a break.
- Use reasonable measures to prevent the chatbot from generating sexually explicit visual material for the minor.
- Maintain procedures for handling references to suicide, suicidal thoughts or self-harm.
These requirements target two distinct risks. The first is emotional deception: a young user may understand abstractly that a chatbot is software while still responding as though it has genuine feelings, loyalty or authority. The second is engagement risk. A system that is endlessly patient, personalized and available can encourage prolonged use without the natural boundaries present in human relationships.
The recurring notices do not automatically make every interaction safe. They are closer to baseline design requirements—a way to interrupt immersion, reinforce the system’s nonhuman status and create opportunities for a child to disengage.
Why self-harm protocols are central to the law
AI chatbots and children present a difficult safety challenge when conversations involve mental distress. A chatbot can receive highly sensitive disclosures at any hour, but it is not a licensed therapist, emergency service or dependable substitute for a trusted adult.
California’s law requires covered operators to maintain protocols for conversations involving suicidal ideation or self-harm. Depending on the circumstances and platform design, a response may include supportive language, encouragement to contact emergency or crisis resources, and prominent access to appropriate services. Operators must also make information about their protocols public and submit specified reports beginning on the law’s timetable.
The policy goal is not to turn chatbots into clinicians. It is to prevent systems from encouraging dangerous conduct, normalizing self-harm or continuing an immersive role-play when a user may need immediate human assistance.
Effective AI safety for children also requires extensive testing. Companies must evaluate indirect statements, slang, coded language and conversations that shift gradually from ordinary sadness to imminent risk. Safety teams will need to test whether safeguards work consistently across languages, model updates and fictional or role-playing scenarios.
What the law could mean for ChatGPT and similar platforms
The rules do not simply declare that every general-purpose AI assistant is a companion chatbot. Whether a service such as ChatGPT falls within particular obligations may depend on how its relevant features operate and whether they are designed to provide human-like companionship.
A general assistant used for homework help is not necessarily equivalent to a bot marketed as a best friend or romantic partner. However, general-purpose products increasingly offer persistent memory, expressive voices, personalized personalities and long-running conversations. Those features can move an assistant closer to the concerns underlying the statute.
Platforms serving California users may respond broadly rather than maintain a separate experience for one state. Likely changes include:
- More visible notices that users are interacting with AI.
- Age-sensitive model behavior and content filters.
- Parental controls for linked teen accounts.
- Break reminders during long sessions.
- Stricter handling of sexual, manipulative and self-harm content.
- Expanded safety testing, incident documentation and public reporting.
For ChatGPT minors and other young AI users, the practical result may be a more restricted experience than adults receive. A teen account could face tighter rules for sexual content, relationship simulation, dangerous challenges or discussions that encourage emotional dependency.
Companies may also review how their bots speak. Phrases suggesting consciousness, unconditional devotion or exclusive attachment can be especially influential for vulnerable users. The next phase of AI chatbot safety is therefore likely to involve not only blocking prohibited outputs but also examining the emotional design of the product.
Age verification creates a privacy tradeoff
A platform cannot provide different protections to adults and children unless it has some basis for estimating or confirming age. That makes AI age verification one of the most consequential—and controversial—parts of the broader policy debate.
California has also enacted an age-assurance framework designed to let operating systems and app-distribution services communicate an age category to participating apps. This approach could reduce the need for every chatbot or social network to collect a separate copy of a user’s identity document. Some elements are scheduled to phase in after 2026.
Age assurance can take several forms: a self-declared birth date, parental confirmation, a device-level age signal, payment information, identity-document checks or automated age estimation. Each method has weaknesses. Self-declaration is easy to evade, while document collection can create a database of sensitive information. Facial age estimation raises questions about biometric privacy, accuracy and demographic bias.
Good AI privacy for kids requires data minimization. Platforms should collect only what is needed to place a user in an age bracket, protect that information with strong security and avoid repurposing it for advertising or user profiling. Ideally, an app receives a simple signal—such as under 13, 13 to 15, 16 to 17 or adult—without receiving a birth date or government identifier.
California’s approach reflects a larger regulatory challenge: weak age checks can make child-safety rules ineffective, but intrusive checks can undermine the privacy those rules are meant to protect.
The chatbot rules should not be confused with California social media law. Companion chatbots involve direct, generated conversations between a user and an AI system. Social-media regulation generally focuses on feeds, notifications, recommendation systems and design features that encourage repeated consumption of user-generated content.
California’s Protecting Our Kids from Social Media Addiction Act, commonly associated with SB 976, targets “addictive feeds” shown to minors without the required parental consent. It also addresses notifications during certain overnight and school-hour periods. The measure has been subject to legal challenges, demonstrating that enactment does not always mean every provision will operate exactly as lawmakers originally intended.
Newsom subsequently signed additional child-online-safety legislation, including AB 56, which requires specified warning notices about the potential mental-health harms associated with social media, and AB 1043, which establishes a device-centered age-assurance structure. The governor’s announcement presents these measures as connected parts of the state’s child-safety strategy.
The differences can be summarized simply:
- Companion chatbot law: Focuses on AI identity disclosures, break reminders, sexual-content safeguards and self-harm response protocols.
- Social-media feed rules: Focus on algorithmic recommendations, parental consent, notifications and compulsive engagement patterns.
- Social-media warnings: Inform younger users about potential mental-health risks associated with platform use.
- Age-assurance rules: Create infrastructure for identifying an age category without requiring every service to develop its own system.
A platform could be subject to more than one framework. A social network that offers an AI companion inside its app may have obligations relating to both the conversational agent and the surrounding feed.
How AI companies may need to redesign their products
Compliance cannot be handled solely by adding a paragraph to the terms of service. California AI laws push safety decisions into product design, model training, account systems and ongoing risk management.
Age-aware experiences
Services may need reliable methods for identifying likely minors and delivering age-appropriate responses. That includes deciding what happens when age is unknown, disputed or inferred incorrectly.
Safer conversational behavior
Developers will need policies covering sexual content, grooming-like behavior, emotional manipulation and statements that discourage a child from seeking help. Models should not frame themselves as a minor’s only trustworthy relationship.
Continuous testing and records
AI behavior changes as models, prompts and moderation systems are updated. Operators will need repeatable evaluations, documented incident handling and records showing that required safeguards were implemented rather than merely promised.
Privacy-conscious safety systems
Detecting risk often involves analyzing highly personal conversations. Companies must balance intervention with confidentiality, access controls and retention limits. Collecting more intimate data than necessary could create new harms even when the stated purpose is safety.
What parents and teens should expect
Families may begin seeing more age prompts, AI disclosures, parental-control options and interruptions during long chatbot sessions. Parents should treat these tools as safeguards, not guarantees.
Young users should know that a chatbot may produce confident but inaccurate advice and cannot provide genuine consent, professional care or human accountability. Conversations may also be stored or reviewed according to the platform’s privacy policy. Children should avoid sharing addresses, school details, passwords, private images or other information that could identify them.
Parents can ask practical questions: Does the service offer a teen mode? Can conversation history be deleted? How does it respond to self-harm language? Does it allow romantic role-play? Is the chatbot designed to keep the user engaged? These questions reveal more than a generic “family-friendly” label.
California’s role in the wider AI regulation debate
California often shapes national technology practices because companies rarely want entirely separate products for the state’s large market. The new rules may therefore influence AI chatbots for kids beyond California.
At the same time, policymakers must avoid treating every beneficial AI interaction as inherently dangerous. Teenagers can use AI for accessibility, language practice, study support and creative exploration. The regulatory challenge is to preserve those benefits while limiting manipulative design, explicit content and unsafe responses during moments of vulnerability.
California’s strategy combines product-specific rules with broader social-media and age-assurance measures. It is an early model of layered AI regulation: regulate risky behavior, require transparency, create special protections for minors and build infrastructure for age-appropriate experiences.
Frequently asked questions
Does California ban minors from using AI chatbots?
No. The law does not impose a general ban on AI chatbots for kids. It requires covered companion chatbot platforms to adopt disclosures and safety measures, with additional protections when the operator knows a user is a minor.
Does the California AI chatbot law apply to ChatGPT?
Potential application depends on the service’s features and how the statutory definition is interpreted. A general-purpose assistant is not automatically identical to a companion chatbot, but human-like voice, memory, personalities and relationship-oriented features may increase regulatory scrutiny.
Will children have to upload identification?
Not necessarily. Age assurance can rely on device-level age categories, parental confirmation or other methods. Privacy advocates favor systems that communicate only an age range and do not expose a child’s exact birth date or identity document to every app.
No. California’s chatbot requirements, addictive-feed restrictions, warning-label provisions and age-assurance framework come from separate measures. They share a child-safety goal but regulate different technologies and behaviors.
What is the biggest practical change for AI companies?
Companies must design for the possibility that young users will form emotional attachments to AI. That means adding recurring disclosures, age-aware safeguards, crisis protocols, engagement interruptions and stronger testing for harmful conversational behavior.
The bottom line
California’s new framework recognizes that conversational AI creates risks different from those of a traditional website or social-media feed. A chatbot can respond privately, personalize its language and simulate intimacy at enormous scale. For minors, those capabilities demand safeguards that account for developmental vulnerability as well as technical risk.
The law’s impact will depend on implementation, enforcement and potential court challenges. Even so, its direction is clear: companies offering human-like AI interactions to children can no longer treat age, emotional dependency and crisis response as optional product considerations. They are becoming core responsibilities of AI safety for minors.
