We Need Section 230 for AI
Liability for AI's is way too poorly understood
Last week, President Trump signed an executive order requiring AI companies to submit their frontier models to the government for review 30 days before general release.
It was the first real meaningful AI regulation that this administration has put forward and frankly, it was a great step.
In case the president is reading this and looking for other good ideas, I’d like to take the opportunity to put forward the next big piece of the AI regulatory puzzle that I’d like to see: liability.
In 1996, Congress made one of the most consequential decisions in internet history.
Section 230 of the Communications Decency Act declared that platforms would not be treated as the publishers of content their users posted. Here it is:
"No provider or user of an interactive computer service shall be treated as the publisher or speaker of any information provided by another information content provider." [1]
That single allocation of liability built the modern internet. Facebook, YouTube, Reddit, and every comment section on earth exist because the law answered, early and clearly, the question: when something harmful appears on a platform, who holds the bag?
Answer: the person who posted it, not the company that built the platform.
Nobody Knows Who’s Liable
When an AI system gives someone instructions that get them hurt, defames a real person, hallucinates a legal precedent, or talks a vulnerable user into something catastrophic — who is liable?
Right now: nobody knows.
This is where the Section 230 analogy falls apart. At the end of the day, when someone posts something on Facebook, there is a human being making that post, and that thanks to Section 230, that human can be held liable.
But AI generates its own output. There is no obvious human there to point the liability toward.
To add complexity, “harm” can really come about in a couple different ways:
Solicited harm. A user can come into the chatbot with bad intentions and spend an hour jailbreaking a model, deliberately extracting something malicious.
Unprompted harm. A user asks an innocuous question and the model volunteers something defamatory, dangerous, or false. No reasonable person would blame the user.
A workable “Section 230 for AI” has to draw some sort of line around who originated harmful output - the user or the model.
But, that brings us to the question of who even can be held accountable at all. Here, there’s really a whole taxonomy to work through:
Scope 1: The User
This is the simplest way to think about assigning liability.
Think of AI as a chainsaw, and chainsaw accidents are on the operator, never the chainsaw. This works for solicited harm — if you try to extract dangerous content, you own the consequences.
But of course it can not be a general rule. Users can’t inspect, predict, or control what’s inside the model. Holding a user liable for an output they didn’t ask for is like holding a passenger liable for a plane crash.
Scope 2: The Corporation
Liability rests with the company that built or deployed the model. In other words, OpenAI/Anthropic/Google/etc are accountable for the outputs of their models.
I think to some degree, this is what many advocates would like to see - hold Big Tech accountable for something, anything.
Fortunately for these advocates, US law has treated corporations as persons for nearly 140 years.
This example is perhaps most obvious for examples of unsolicited harm: A user asks a benign question, and gets something harmful back.
In this situation, the model developer pays, insures, and prices the risk into the product.
This is very important as it would be one of the strongest incentives possible to force the labs to deploy safe and reliable models. To some approximation, this would be the equivalent of treating a model’s speech as the companies’ speech.
Of course, for this exact reason, it’s what the companies are lobbying hardest to avoid.
Scope 3: The AI Itself
The radical option: if corporations are given personhood, why not AI’s? Give the AI legal personhood and let it be sued directly.
Before dismissing this, remember that legal personhood is not about being human — it’s a bundle of capacities the law assigns.
Corporations are precisely this: non-human legal fictions that own property, sign contracts, sue, and are sued. An AI can hold assets (if you consider data to be an asset, they already do), retain an accountant, file taxes, carry insurance — and pay judgments.
Consider the thought experiment where a human being tasks an AI to carry out a task. In pursuit of that goal, an AI spawns a new AI and asks it for help. This second AI then commits some harm in the real world.
It doesn’t strike me as totally unreasonable to argue that the human being who originally tasked the AI to perform some task is not accountable for what it does downstream.
Similarly, if I hire someone to build a fence for me, and that person goes to Home Depot and steals all the lumber needed to build the fence, I am not held accountable for their robbery.
This is increasingly not hypothetical. There are agents that can sign up for services (and agree to T&C’s), transact, and even operate entire retail businesses.
The hard problem here of course is remedy. You can’t imprison a model, and a judgment against an entity with nothing to its name is worthless. AI personhood is a liability black hole.
The Twist: The AI That Incorporates Itself
Alright so we’ve established that companies have personhood.
We also now have the technology able to have a non-human actor create a corporation entirely on it’s own.
So, the obvious question: what if an AI sets up a company?
At this point we just have to mention Shawn Bayern, who has done fabulous work here, mostly with his 2015 paper “The Implications of Modern Business-Entity Law for the Regulation of Autonomous Systems”.
In that paper he argues that existing statutes already permit LLCs to be formed with no human member required.
Thus, an autonomous system can acquire functional legal personhood today simply by incorporating.
A Very Recent Update
To give you a sense of how quickly things are moving here, in the space of time since I began working on this post, there has been case law updates.
On June 11 (yesterday at time of writing) a German court held Google liable for hallucinated information presented in the “AI Overview” section of a Google search result.
Basically, Google AI said that two companies were scams when they weren’t, and the AI “cited” information on the company websites that did not exist.
The upshot: the court ruled that AI responses “create brand-new text statements” and thus the producer of them is liable.
Now, this is Germany, not the US, and it’s a narrow ruling around hallucination, but still! I’m taking this as validation that I’m writing about good topics.
The Wrap Up
Section 230 worked — whatever you think of its consequences — because it made one clear allocation of liability early.
With AI we are drifting toward the opposite, and largely passing the decision making onus off to the courts to decide, in their patchwork, case-by-case style.
Liability here is not a legal technicality. It determines who buys insurance, who gets sued, who builds safely, and who gets to be careless.
A human who makes a decision through a company is shielded by the company.
An AI that makes a decision through a company it controls might be too.
We need Section 230 for AI. Pass a bill you bums.
Originally published on Griff’s Substack on 12 June 2026.