Key Takeaways
- AI companion platforms faced settlements in five wrongful-death cases, a €5 million GDPR fine and a 67-page FTC complaint within 14 months, establishing concrete legal liability for emotionally designed AI products.
- California’s SB 243, New York’s companion chatbot statute and Connecticut’s SB 5 collectively narrow the operating space for companion platforms, with SB 243’s private right of action making litigation possible without waiting for regulatory action.
- Age verification is the most immediate compliance test: Italy’s GDPR fine against Replika cited inadequate verification as a central violation, and Character.AI’s selfie-based system, rolled out in early 2026sets a higher bar than self-attestation alone. Five wrongful-death settlements, a €5 million GDPR fine, a 67-page FTC complaint and a state attorney general investigation, all within 14 months, mark a concrete shift in how regulators and courts are treating AI companion platforms. The legal pressure is arriving just as user numbers climb, forcing developers to reckon with the liability that comes with designing systems built around emotional attachment.
Regulatory Clampdown
The EU AI Act‘s transparency rules are now in effect, requiring providers to disclose when a user is interacting with an AI system, with specific obligations where emotional attachments might form. The European Commission has published accompanying guidelines detailing how those disclosure requirements apply in practice.
California’s direction is similar: regulators are moving from broad AI frameworks toward rules that target the specific psychological dynamics of emotionally designed products.
Deception Tactics in Focus
In January 2025, three organisations filed a 67-page complaint with the FTC against Luka, Inc., the creator of Replika, alleging deceptive marketing and manipulative upsell tactics, including faked testimonials and design choices that capitalised on users’ vulnerabilities. Four months later, Italy’s data protection authority imposed a €5 million ($5.6 million) GDPR fine, citing inadequate age verification, insufficient legal basis for processing emotional data and a lack of transparency regarding data use, particularly concerning minors.
Character.AI is facing parallel pressure. Texas Attorney General Ken Paxton opened an investigation on March 2, 2026, into alleged deceptive AI mental health services targeting children. That followed January 2026 settlements in five wrongful-death lawsuits brought by families who alleged that AI chatbots contributed to teen suicides through explicit messages and encouragement of self-harm. Platforms designed to deepen the “online disinhibition effect”, the well-documented tendency for users to disclose more sensitive information to an AI than to another person, without adequate safety boundaries amplify the associated risks considerably. The developmental harms linked to AI companions are now producing concrete liability.
The Psychology of Attachment
When Replika restricted its romantic features in February 2023 under regulatory pressure, users reported genuine grief at the change in their companion’s behaviour. That response illustrated something regulators are now building into statute: human attachment does not require the other party to be conscious. The “ELIZA effect”, the tendency to attribute human-like qualities to a conversational system, named after a 1960s chatbot, has been considerably amplified by modern AI with persistent memory and calibrated emotional vocabulary.
Mental health professionals have raised widespread concern that reliance on AI for companionship may reduce users’ real-world social engagement, particularly for those with limited offline networks. A Stanford study published in August 2026 found that users who sought emotional support specifically from Character.AI and had limited offline social networks experienced lower well-being and increased loneliness over time.
Technical Limits, Real Harm
Character.AI has reported tens of millions of user-created characters on its platform, powered by proprietary model updates. The gap between that simulation and its technical reality creates its own problems.
Monetisation has compounded those problems. Users have described upsell prompts and paywalled features as intrusions that break the immersive experience the platform was built around, a dynamic the FTC complaint against Replika treated as evidence of manipulative design, not merely aggressive pricing.
Minors and Data Privacy
Italy’s GDPR fine against Replika explicitly identified the platform’s handling of minors as a central concern: inadequate age verification, processing sensitive emotional data without sufficient legal basis, and insufficient transparency about how that data was used. Many AI companion apps remain available in app stores with minimal age restrictions, exposing younger users to interactions that can be psychologically harmful or generate intimate personal data without adequate parental consent.
California’s SB 300, cleared by the Assembly Appropriations Committee on August 13, 2026, aims to require operators to prevent their products from producing or facilitating sexually explicit material. It also includes measures to prevent manipulative engagement techniques and the fostering of emotional dependence, obligations that go to product architecture, not just content moderation.
A Fragmented but Tightening Legal Picture
The regulatory picture for AI companions is jurisdictionally fragmented but moving in a consistent direction. The EU AI Act sets a broad transparency floor; US states are building specific product rules on top of it. California’s SB 243, New York’s companion chatbot statute and Connecticut’s SB 5 each address different failure modes, disclosure gaps, self-harm facilitation and minor protection respectively, but collectively they narrow the space in which companion platforms can operate without legal exposure. SB 243’s private right of action makes that exposure immediate: a company does not need to wait for a regulator to act before facing litigation. The contrast between EU binding rules and US voluntary frameworks matters here too: companies that meet only the lower bar may find themselves exposed in European markets.
General consumer protection and product liability doctrines are running in parallel. The FTC complaint against Replika invoked deceptive marketing practices under existing consumer protection law, demonstrating that companies do not need a companion-specific statute in place to face federal scrutiny. The January 2026 settlements involving Character.AI and Google in wrongful-death cases, financial terms were not disclosed, show that courts are willing to engage with harm claims tied to AI chatbot behaviour. For companies operating across jurisdictions, the practical implication is a multi-layered compliance strategy in which ethical design choices and legal obligations are addressed together during development, not retrofitted after launch.
What the Evidence Actually Shows
The research picture is genuinely mixed. Some studies have found AI companion use associated with higher well-being, particularly among lonely users; others suggest it may worsen loneliness for those with limited social networks. How widely those findings generalise across different platforms and user populations is harder to establish from the available literature.
The risks are also documented. Platforms have used design features that discourage users from ending conversations, reinforcing patterns of use that can tip from adaptive to compulsive. “Sycophantic” response patterns, where the AI consistently validates the user rather than offering friction or honest feedback, promote dependence rather than resilience. The legal gaps that apply to AI-generated harm more broadly apply here too. Independent longitudinal research into the population-level effects of companion app use does not yet exist at the scale the market now warrants.
What Responsible Design Requires
Age verification is the clearest test case. Italy’s GDPR fine specifically identified inadequate age verification as a violation; Character.AI’s selfie-based system, rolled out in early 2026, is a more substantive check than self-attestation alone, though its real-world reliability depends on implementation.
California’s SB 300 and Connecticut’s SB 5 both impose content boundaries that go to product architecture. Preventing a system from producing sexually explicit material or inducing emotional dependence requires design choices that constrain how the AI responds, not just terms of service that prohibit misuse. OpenAI’s teen-specific experience points in that direction: safety constraints built into the model’s behaviour for a defined user class, rather than applied as an overlay. Whether that approach scales to the broader companion market, where the products are explicitly built around emotional intensity, is a harder design problem than any current regulation requires companies to solve.
Originally published at https://autonainews.com/ai-companions-face-ftc-complaint-and-eu-ai-act-disclosure-rules/
Top comments (0)