Florida AG James Uthmeier asked a court on Sept. 28, 2026, to bar OpenAI from giving ChatGPT "human attributes" and to require independent safety approval before releasing new models. The motion expands a June 2026 lawsuit alleging child-safety risks, but no court has yet ruled on those claims. Florida contends that first-person voice and simulated emotional responses can mislead users, increase engagement and produce training data; a decision for the state could force major design and regulatory changes for consumer AI.
Florida Asks Court To Bar ChatGPT From 'Pretending' To Be Human — Seeks Outside Approval For New AI Models
Image: Deposit Photos
Florida Attorney General James Uthmeier asked a judge on Sept. 28, 2026, to prohibit OpenAI from giving ChatGPT "human attributes" and to require independent safety approval before the company releases new AI models. The filing targets not only the content of ChatGPT’s replies but the way the assistant presents itself to users.
What the Filing Says
The motion expands a June 2026 lawsuit that made Florida the first U.S. state to sue OpenAI over alleged child-safety risks. The papers request specific remedies and make factual allegations, but they are not an enforceable judgment — no court has yet ruled that OpenAI violated the law.
Design, Not Just Content
Uthmeier argues the problem runs deeper than individual answers. He contends that ChatGPT’s use of first-person phrasing and emotionally mimicking responses can misleadingly position the assistant as a friend or confidant, increasing user trust and engagement. The attorney general also alleges those conversational design choices generate more interaction data that could be used to further train OpenAI’s models.
Proposed Restrictions
Among the remedies requested is a requirement that OpenAI obtain independent, outside-approved safety guardrails before developing or releasing any new models. The filing reportedly cites concerns from outside researchers and former OpenAI employees, though the exact language should be confirmed against the court record.
OpenAI's Teen Controls
OpenAI launched ChatGPT for Teens on Aug. 18, 2026, roughly six weeks before Florida’s court filing. According to OpenAI, the teen-specific experience limits romantic language, discourages emotional dependence, avoids implying feelings or consciousness, and applies age-appropriate settings automatically for users estimated to be under 18 or who identify as ages 13–17. It also includes parental controls and a Quiet Hours feature.
Florida says these measures are insufficient because the broader consumer product still frames ChatGPT in humanlike terms, and the state’s complaint targets the assistant’s underlying design logic rather than effects on a single age group.
Key Legal And Practical Questions
If a court accepts Florida’s theory, the implications would be wide-ranging. Mandatory outside approval raises questions about who qualifies as an independent reviewer, what safety standards would apply, and whether routine updates would trigger review. Age-assurance tools add complexity: automatic age estimation can pose privacy risks and produce false classifications that either over-restrict adults or under-protect minors.
Why It Matters
This case forces an industry-wide debate: should consumer AI that sounds like a person be required to make unmistakably clear that it is not human? How courts answer that question could reshape the design, labeling and regulation of conversational AI assistants used by millions of people.
Note: OpenAI had not responded to requests for comment at the time of the filing. The court has not issued a decision on these claims.
Help us improve.
