Detection Objection
How much of what's filed in state court is now drafted with AI? The answer depends on how you measure. We calibrate a detector against filings with known AI use, then measure four courts.
Four trial courts
Our detector corpus contains 127,518 case-initiating civil filings from four counties. For the comparable trend below, we exclude collections and debt cases, leaving 79,081 complaints. Estimated AI-assisted drafting is near zero before 2023, then rises sharply. By 2026 the California courts read more than twice the Oklahoma ones.
The highlighted court shows 95% bootstrap intervals for each year. Select a court name to highlight it; Santa Clara is the court on the dial. Collections and debt cases are excluded.
Case mix
Some types of case show far more AI-assisted drafting than others, and our four corpora contain them in very different proportions. Employment and business complaints estimate highest, at 59–98%. They are 31% of Santa Clara's non-collections complaints in 2026 and under 3% of Tulsa's. Personal injury, the one large type all four courts share, estimates at 21–30% everywhere. So part of the gap between the states is case mix. Not all of it: contract complaints still estimate at 16–21% in California and 9–12% in Oklahoma.
The mix also shifted in 2026, toward the high-reading types in California and away from them in Oklahoma. Holding each court's case mix at its 2020–22 shares, Santa Clara's 2026 estimate falls from 37.8% to 32.8% and San Francisco's from 35.6% to 31.9%, while Tulsa's rises from 16.9% to 20.7% and Oklahoma County's from 16.1% to 19.3%. At fixed mix, the gap between the states narrows from about 20 points to about 12.
The mix also reflects how we collected, not only what gets filed. We scraped three different public court systems, under different rules for each court. In the Oklahoma counties we collected civil cases seeking more than $10,000. In San Francisco we collected general civil cases, and that corpus is not a complete census. In Santa Clara we collected a listed set of civil case types that leaves out collections. A court with more employment cases in our data may simply have had more of them pass our rules. Coverage also shifts within a court: San Francisco's corpus nearly doubles from 2024 to 2025, at least partly because its 2024 collection was less complete, so that jump is not growth in filings.
Width is each type's share of the court's complaints; orange height is its ScoreMix estimate. Hatched means not reported: fewer than 100 complaints of that type that year or in 2020–22. Collections cases excluded. SC is Santa Clara, SF San Francisco, OKC Oklahoma County.
Who files with AI
Self-represented filers adopted AI earlier and faster. In the paper, we also find that use persists: attorneys flagged in one year were flagged again the next 35% of the time in Tulsa, 55% in San Francisco and 65% in Santa Clara.
Whiskers are 95% intervals. 2026 has 266 self-represented and 2,913 attorney complaints.
Where in a complaint
We ask which parts of a complaint the raw, uncalibrated detector scores highest. It responds most to the parts a filer writes in their own words. Select a section to see how it compares with the rest of the complaint.
We compare detector scores between parts of a complaint, not AI prevalence. We labeled a sample of 400-word windows from 2026 complaints in Oklahoma County, Tulsa and San Francisco: windows scoring above 0.10, plus an equal number of lower-scoring windows matched on court and year. Pair a labeled window from this section with one from elsewhere: the number is how often, out of 100, this section's window scores higher. Santa Clara complaints were not labeled. The complaint text is illustrative.
Why AI-assisted legal text is hard to detect
Detectors are built on general prose. Court filings weaken their signal in three ways.
Boilerplate dilutes the signal
We count a run of 8 tokens as boilerplate if it appears in at least 1% of filings. By that measure about 26% of the text in our filings is boilerplate, against at most 6% in press releases or paper abstracts. A detector can't tell who wrote words that everyone writes.
At right, an excerpt from a complaint filed in San Francisco Superior Court in 2022. Grey marks language caught by our definition: about half of the words. The filer's own account is the sentence that follows.
Plaintiff is informed and believes and thereupon alleges that, at all times herein mentioned, each of the Defendants sued herein was the agent and employee of each of the remaining Defendants, and was at all times acting within the course and scope of such agency and employment. On or about June 28, 2020, Plaintiff, a lawful tenant of Defendant at the Premises, leaned against the handrail, which gave way and caused Plaintiff to fall approximately 10 feet to the concrete patio below and suffer severe injuries and damages.
Legal drafting is a narrow register
Court petitions give a writer fewer choices of what to say next than press releases, abstracts or consumer complaints do. We measured this as a language model's uncertainty about the next word while drafting. A legal petition runs at 0.35 nats per token, against 0.43 for a press release, 0.53 for a scientific abstract and 0.57 for a consumer complaint. When everyone writes from a small vocabulary, human and AI text converge.
Ask the AI to write like a lawyer, and a word-frequency detector loses much of its edge on petitions. For context, AUROC of 0.50 is no better than guessing randomly.
Striped: what the detector loses when the AI is told to write like a lawyer.
How filers use AI is unknown
Many detectors learn what AI text looks like from AI rewrites of human documents. A litigant may instead ask an AI questions and paste in its answers, which produces different text. We generated 1,134 such questions about 378 real petition passages and compared how a rewrite-trained word-frequency detector does on rewrites and on answers.
For a self-represented filer's "help me write this," the detector catches 37–59% of answers across three open models, against 90% of rewrites. Across all three kinds of question, it catches 16–72% of answers. Refit on answers, it separates them from human text with AUROC 0.89–1.00. That is why we calibrate on PhantomDocket.
Striped: caught on AI rewrites, missed on AI answers.
PhantomDocket, the reference
To calibrate an instrument, we needed real AI drafting data. We built a dataset of court filings with known AI use from judges' rulings on the Charlotin Hallucinations Database. These rulings call out hallucinated citations and misquotations. We traced them back to the offending filing that contained them.
PhantomDocket is also how we chose the instrument. For comparison, we used filings by the judge and opposing counsel in the same cases. We compared detectors on how well they told the two groups apart. Then we searched EditLens's settings: how long each window is, which score it reads, and how window scores combine. We kept the setting that best separated them on filings held out from the search. That is the setting on the dial at the top of this page.
Pangram classifies best but costs too much to run on every filing. EditLens is open and close behind.
Five of the 252 rules we tried for turning Pangram's window-by-window scores into one score per filing. Averaging the window scores, as the paper does, is among the best; most rules that average the scores land at 0.86–0.89. Rules built on Pangram's own labels fall toward 0.70, and the worst is barely above chance. Partial AUROC counts only false-positive rates under 10%, so it runs lower than the chart at left.
Limitations
Misreading an AI detector can lead to sanctions, disciplinary actions, and biased outcomes. Read these before citing a number.
Can this tell me whether one filing used AI?
No. ScoreMix measures corpus prevalence, not individual filings. Estimates describe the share of a court's complaints. They should not be used to say whether any one filing used AI. Even the best detector has real false positive and false negative rates. On PhantomDocket, Pangram's average window score wrongly flags 7.2% of the human control filings and misses 11.4% of the AI-assisted ones. Its own, more cautious label wrongly flags 0.9% but misses about half.
Is this the share of filings with hallucinations?
No. ScoreMix is not a hallucination rate. It is the share of complaints whose scores resemble the PhantomDocket filings. While not tested, it's probable that most AI-assisted complaints do not contain a hallucination.
Could the estimates be too low?
Possibly. The readings may run low. PhantomDocket captures AI use careless enough to produce a hallucination, and its filings are mostly briefs and motions, not complaints. ScoreMix assumes AI-assisted complaints score like these filings. If they score lower or higher, either the AI-assistance or the complaint text itself is distributionally different from what is seen in PhantomDocket. It's likely that AI use reads as human and the estimate runs low. We have no complaints with known AI use to check this directly.
Can I compare the four courts directly?
No. Courts aren't directly comparable. The headline estimates are not standardized for case mix, so a shift toward case types with more AI use can move them even if use within each type stays the same. Holding each court's 2020–22 mix moves the 2026 estimates by 3 to 5 points. Case mix differs between courts, and we can't yet separate it from geography. Both California counties are at the center of the Bay Area's tech industry, but our data can't show that proximity is the cause.
Doesn't the detector setting still change the answer?
Yes. Settings still matter, less. Calibration narrows the spread across settings without removing it. For Santa Clara 2026, ScoreMix stays between 29.0% and 46.9% across the other 44 settings, against 1.8% to 97.7% for raw flags. This is why we use PhantomDocket to choose the optimal setting.
Is AI-assisted text in court filings a bad thing?
Not necessarily. ScoreMix just estimates use. AI tools may lower the barriers to justice: a self-represented filer can now draft a complaint or answer they couldn't have written, or paid a lawyer to write, before. The same tools may also overload courts with more cases and filings, and with hallucinated citations and quotes that judges and opposing parties have to catch.
Who we are
This project comes from the Stanford Regulation, Evaluation, and Governance Lab (RegLab).
Cite
Click to copy@article{arifov2026detection,
title = {Detection Objection:
Documenting, Calibrating, and
Detecting the Rise of AI-Assisted
Text in U.S. State Courts},
author = {Arifov, Javokhir and Spangher,
Alexander and Hennessy, Jonathan
and Karan, Diya and Hariri, Emaan
and Ho, Daniel E.},
year = {2026}
}





