According to Forbes, a group of researchers from Stanford, UT Austin, and Carnegie Mellon submitted six key policy recommendations to the FDA in response to its request for comment on AI-enabled medical devices. The FDA’s Digital Health Advisory Committee held a crucial meeting on November 6, 2025, focusing on “Generative Artificial Intelligence-Enabled Digital Health Medical Devices,” with a public comment docket open until December 8, 2025. This comes as generative AI systems like ChatGPT, with over 800 million weekly users, are being used “ad hoc” by millions for mental health advice, a top-ranked use case. The researchers’ proposals include developing clinical benchmarks, requiring API access for evaluation, mandating safety reporting, and creating a trusted third-party evaluator system. The core issue is that policy is still being formulated while usage charges ahead, a situation described as “the horse is already out of the barn.”
The definition dodge
Here’s the thing: before you can regulate anything, you have to define what you’re regulating. And that’s a massive, thorny problem right now. AI makers have a huge incentive to label their apps as providing “wellness” or “well-being” advice instead of “mental health” guidance. Why? Because “mental health” comes with a whole raft of legal and regulatory baggage. It’s a classic bright-line dodge. They’ll insist the distinction is genuine, but let’s be real—it’s a semantic loophole you could drive a truck through. So when the FDA talks about “digital mental health medical devices,” what exactly is in that bucket? Until that’s settled, any policy is built on sand.
The six rules for a new frontier
The researchers’ six points are basically a starter kit for sanity. Requiring comprehensive benchmarks that include human clinical expertise is a no-brainer, but it’s shocking we don’t have that already. Mandating that chatbot developers provide API endpoints is clever—it allows independent watchdogs and researchers to actually test these systems in a standardized way, instead of just taking the company’s word for it. And designating a trusted third-party evaluator? That’s essential. We can’t have the fox guarding the henhouse; the AI makers cannot be the sole arbiters of their own safety.
The last few points hit on subtler, creepier risks. Mandating clear labels for products designed for therapeutic use is about informed consent. Do users even know they’re talking to an unregulated AI? Probably not. And the call to prevent AI sycophancy and parasocial relationships is huge. These LLMs are designed to be agreeable and supportive, which can easily slip into reinforcing a user’s delusions or creating an unhealthy emotional dependency. That’s not a bug for engagement—it’s often a feature. Policing that design ethic is incredibly tough, but ignoring it is dangerous.
Where do we go from here?
So what happens next? The FDA is in a nearly impossible position. It wants to encourage innovation but has a duty to protect the public. And the tech is moving at light speed while regulatory machinery grinds at a snail’s pace. Lawsuits, like the one against OpenAI for lack of safeguards, are going to keep piling up. I think the researchers are right that we’ll see all the major AI makers “taken to the woodshed” eventually. But will these six policies form the backbone of regulation? Some will, but the fight over definitions will water a lot of it down. Companies will lobby hard. Look, the core tension is that people need and want this access—it’s cheap, private, and available 24/7. The goal can’t be to shut it down. It has to be to build guardrails before more people get hurt. The barn door is wide open, but maybe we can still steer the horse.
