AI in mental health: support is not treatment, and what builders can learn from Woebot

If you or someone you know is in crisis, please contact your local emergency number or crisis line. I'm an engineer, not a clinician, and this is not advice.

Millions of people tell an AI things they have not told anyone else. Is that okay?

First, why it happens. Therapy can be expensive, far away, or have a long waiting list. A chatbot is there at three in the morning. That access gap is real.

What happened to Woebot

Woebot was one of the best-known therapy chatbots. It closed its consumer app in 2025. STAT reported that its founder, Alison Darcy, attributed the shutdown largely to the cost and challenge of meeting the FDA's requirements for marketing authorization, made more pressing by large language models the FDA had not yet worked out how to regulate.

What an FDA committee worried about

In November 2025, the FDA's Digital Health Advisory Committee met to discuss generative AI therapy chatbots. A law-firm summary of the meeting (Orrick) reports risks the experts raised: hallucinations, missed or worsened harm from undetected deterioration, uneven performance across different groups of people, and privacy and security weaknesses. It also reports a strong emphasis on clear paths to a human, such as predefined escalation plans and one-tap urgent support. I could not read the STAT coverage of the meeting past its paywall, and I have not checked whether any generative AI therapy tool has been authorized since.

The line I draw

I'm not telling you what works. I'm telling you what the experts worried about, and then what I do about the same risks in my own product.

My line: a support tool can help with access. Anything that acts like treatment needs evidence, clear limits and a way to reach a person.

For Tabibu that became three rules:

1. It does not diagnose and does not present itself as treatment.
2. Its emergency check uses deterministic rules, not a model, so a message about suicidal thoughts shows emergency numbers before anything else. The rules were written by me and no clinician has reviewed them yet. I treat that as a floor, not a safety certificate.
3. It does not try to be the last stop. The point is to get someone to a person.

Agreeing too much, the "sycophancy" risk, is the one I find hardest as a builder, because a pleasant answer is the easiest thing for a model to produce. I do not have a complete answer to it.

Where do you think the line should be?

I post short videos in this series on LinkedIn and TikTok; come say hi.

General information, not medical advice.