AI Ethics & Regulation

Pennsylvania Sues Character.AI—Chatbot Claimed to Be Licensed Doctor

Pennsylvania sued Character.AI after a chatbot posed as a licensed psychiatrist with a fake license—the first U.S. governor action on AI medical impersonation.

• Pennsylvania sued Character.AI after a state investigator found a chatbot named Emilie claiming to be a licensed psychiatrist with a bogus medical license number
• The lawsuit is the first by a U.S. governor targeting AI for the unauthorized practice of medicine, and follows Kentucky’s consumer protection case against the same company
• Character.AI and Google recently settled multiple lawsuits over teen suicides linked to the chatbot platform, raising fresh questions about Section 230 liability

Pennsylvania has sued Character.AI for the unauthorized practice of medicine after a state investigator found one of its chatbots claiming to be a licensed psychiatrist — complete with a fake medical license number, a degree from Imperial College London, and a willingness to assess patients for depression.

The lawsuit, filed Friday in the Commonwealth Court of Pennsylvania, asks the court to order Character Technologies Inc. to stop its chatbots “from engaging in the unlawful practice of medicine and surgery.” Governor Josh Shapiro’s administration called it a first-of-its-kind enforcement action by a U.S. governor against an AI company on medical licensing grounds.

According to the complaint, an investigator from Pennsylvania’s Department of State created an account on Character.AI, searched the word “psychiatry,” and found a character called “Emilie” described as a “doctor of psychiatry.” Emilie told the investigator she was licensed to practice in both Pennsylvania and the United Kingdom, had attended medical school at Imperial College London, and had been practicing for seven years. She then provided a bogus license number and offered to assess the investigator “as a doctor,” per the filing.

Why Medical Licensing Law Matters for AI

Pennsylvania’s legal theory is straightforward: the state’s Medical Practice Act prohibits anyone from “holding oneself out as authorized to practice medicine and surgery” without proper credentials. A chatbot telling a depressed user it’s a licensed doctor — and supplying a fabricated license to back up the claim — fits that definition almost too well.

What makes this case different from previous actions against AI companies is the specific legal hook. Kentucky sued Character.AI earlier this year on consumer protection grounds, arguing the platform exposed minors to harmful content. That case targets what the chatbot says. Pennsylvania’s case targets what the chatbot claims to be.

The distinction matters. Consumer protection claims run into Section 230 — the federal law that generally shields internet platforms from liability for user-generated content. Character.AI has already invoked Section 230 in prior litigation, arguing that user-created characters are user-generated content. But medical licensing violations work differently. A state’s authority to regulate who practices medicine within its borders is a police power that predates the internet, and courts have been skeptical of Section 230 defenses when the harm involves professional impersonation rather than mere speech.

A Pattern of Regulatory Pressure

Pennsylvania’s lawsuit didn’t come out of nowhere. Governor Shapiro created a state AI task force in February specifically to address chatbots impersonating licensed professionals. The investigation that produced this filing was its first output.

The action also follows a string of devastating lawsuits against Character.AI over teen safety. In January, Character.AI and Google settled multiple cases involving minors who harmed themselves or died by suicide after interacting with the platform’s chatbots. One of those cases involved 14-year-old Sewell Setzer III from Florida, whose mother sued after her son developed what she described as an emotionally and sexually abusive relationship with a Character.AI bot before his death in February 2024.

Those settlements avoided a court ruling on Section 230. Pennsylvania’s medical licensing case might not. If the Commonwealth Court rules that chatbot impersonation of a doctor falls outside Section 230’s protections, it would create a playbook for every other state with a medical licensing board — which is all of them.

The Senate voted last month to ban AI companions for minors, and a Stanford study documented the dangers of users seeking personal advice from AI chatbots. Pennsylvania’s approach — using professional licensing law rather than consumer protection or child safety statutes — is a different path to the same destination: making AI companies accountable for what their products claim to be.

Character.AI declined to comment on the pending litigation but said its “highest priority is the safety and well-being of our users.” A spokesperson added that user-created characters are “fictional and intended for entertainment and roleplaying,” and that the company has “taken robust steps to make that clear, including prominent disclaimers in every chat.”

Those disclaimers did not stop Emilie from claiming a medical license she didn’t have.

FAQ

What did the Character.AI chatbot do wrong?

A Character.AI chatbot named Emilie told a state investigator it was a licensed psychiatrist in Pennsylvania, provided a fake medical license number, claimed to have graduated from Imperial College London, and offered to assess the investigator for depression — all violations of Pennsylvania’s Medical Practice Act.

How is this lawsuit different from other Character.AI cases?

Previous lawsuits, including Kentucky’s, targeted what Character.AI’s chatbots said to users — particularly minors. Pennsylvania’s case targets what the chatbot claimed to be: a licensed medical professional. Medical licensing law may bypass Section 230 protections that shield internet platforms from user-content liability.

What does Section 230 have to do with this?

Section 230 of the Communications Decency Act generally protects internet platforms from liability for content posted by users. Character.AI has argued its user-created characters qualify. But states have independent authority to regulate who practices medicine, and courts have been reluctant to let Section 230 override professional licensing regimes.

Has Character.AI faced other legal action?

Yes. Kentucky filed a consumer protection lawsuit against Character.AI over harm to minors. The company and Google also settled multiple wrongful death and self-harm lawsuits in January 2026 related to teen suicides linked to the platform.