AI News Quality
How do you tell when the news was written by AI?
Readers can't tell AI-written text from human writing, and the detection tools were withdrawn. What gives machine-made news away is everything around the prose, and here is how to check it.
· 7 min read
Summary: You usually can't tell from the writing. People pick out AI-written text at the rate of a coin flip, and the detection tools built to do it for them have been withdrawn for inaccuracy. What you can tell is whether a named person stands behind the story, whether it contains anything a machine couldn't have gathered, and whether the site publishes more than any staff could write. The useful question isn't "was this written by AI" but "did anyone check it and sign it," and that question has visible answers.
The sentence you're reading could have been produced by a language model, the kind of AI system behind chatbots, and nothing in its grammar would tell you. That is the first fact to accept about AI-generated news, because most advice on spotting it starts from the opposite assumption. Readers are told to look for stiff phrasing, repeated words, or a certain blandness, and those tells existed for about a year. The text no longer gives itself away, and the tools don't work either. What still gives a machine-written story away is everything around the text: The byline, the sourcing, the publishing schedule, and the errors a person would have caught.
How much of it there is
NewsGuard, which rates news websites for reliability, keeps a running count of what it calls AI content farms: Sites where a substantial share of articles is produced by AI, published without significant human oversight, presented as if written by people, and not disclosed as machine-made. As of June 23, 2026, its tracker listed 3,749 such sites in 16 languages. The oversight test is the one that matters for readers, and NewsGuard's example of failing it is an article that still contains error messages or "other language specific to chatbot responses," left in because nobody read the piece before it went up.
The farms are the obvious case. The less obvious one is machine-written content inside outlets you'd trust. In November 2023, the Associated Press reported that Sports Illustrated had published product reviews under the names of writers who apparently did not exist, with one author's photo traced to a site that sells AI-generated portraits. The magazine said a contractor, AdVon Commerce, had produced the articles and "assured the magazine that they were written and edited by humans," and removed the content. In May 2025, the Chicago Sun-Times printed a summer reading list recommending books that don't exist. The list came in a syndicated section from King Features, a company that sells ready-made content to newspapers, whose freelancer, the distributor said, "used AI in its story development without disclosing the use of AI," and the Sun-Times said the section "was inserted into our paper without review from our editorial team." Both failures had the same shape: Real masthead, outside supplier, no one reading the copy.
The third place AI-written news now appears is in the answer box. The Reuters Institute for the Study of Journalism at Oxford, in its 2026 Digital News Report, drawn from 97,520 people in 48 markets, found that 10% of people use AI chatbots for news, up from 7% a year earlier, and 16% of under-35s do. Trust in the chatbots' answers stands at 20%, against 37% for news overall, and the low trust is earned. In October 2025, journalists from 22 public service broadcasters in 18 countries, coordinated by the European Broadcasting Union and the BBC, evaluated more than 3,000 news answers from ChatGPT, Copilot, Gemini, and Perplexity. 45% had at least one significant issue, 31% had serious sourcing problems, and 20% contained major accuracy errors, including invented details. Gemini had significant issues in 76% of its answers.
Why you can't tell from the text
The research on human detection is short and consistent. In 2021, Elizabeth Clark and colleagues, researchers in natural language processing, tested whether readers could distinguish text written by GPT-3, an OpenAI language model released in 2020, from human writing across stories, news articles, and recipes. Without training, "evaluators distinguished between GPT3- and human-authored text at random chance level." With training, their accuracy "improved up to 55%," which the authors found was not a significant improvement. The models that write today's content farms are better at sounding human than the one those readers couldn't catch.
The tools did no better. OpenAI launched a classifier, a tool meant to flag AI-written text, in January 2023 that, by its own evaluation, identified 26% of AI-written text correctly while labeling human writing as AI 9% of the time. It withdrew the tool in July 2023 citing "its low rate of accuracy." A detector that misses 74% of machine text and accuses 1 human writer in 11 is worse than no detector, because it hands the reader a verdict with no basis.
The conclusion isn't that the question is hopeless. It's that the question has been aimed at the wrong object. Prose is the part a model does well. Reporting, accountability, and volume are the parts it doesn't, and all three are visible.
What the law now asks for, and what good newsrooms already did
Since August 2, 2026, the European Union's AI Act, its law regulating artificial intelligence, has required, under Article 50, that anyone deploying an AI system to generate text "published with the purpose of informing the public on matters of public interest" disclose that the text was artificially generated. The exception is instructive: No disclosure is needed where the content "has undergone a process of human review or editorial control" and "a natural or legal person holds editorial responsibility for the publication." The law doesn't ban machine-drafted news. It draws the line at whether a person checked it and is answerable for it.
That is the line serious newsrooms had drawn already. The Associated Press has used automation to turn corporate earnings data into short stories since 2014, and its 2023 standards, as Nieman Lab, Harvard's journalism research site, reported, bar staff from using generative AI to create publishable content and treat any output from it as "unvetted source material," subject to the same checks as a tip from a stranger. Disclosed automation of structured facts, reviewed by editors, is not the problem. Undisclosed generation of everything else, reviewed by no one, is.
The objection: If it's accurate, who cares who wrote it?
The strongest objection is that provenance is a distraction. A correct story is correct whether a person or a model produced it, the AP has been publishing machine-written earnings reports for 12 years without complaint, and readers who demand a human author are asking for a credential, not for accuracy. Judge the facts, on this view, and ignore the byline.
We'd concede that a byline is not a guarantee, and that a human-written story can be wrong in every particular. What the objection misses is how accuracy gets produced. A story is accurate because someone checked it, and someone checks it because their name is on it and their employer's correction policy applies to them. Remove the name and the checking goes with it, which is what the 45% error rate in the broadcasters' study measures: Not what models can do, but what unreviewed generation does. The Sun-Times list wasn't wrong because a machine drafted it. It was wrong because the person who used the machine didn't look up the books, and nobody downstream had a reason to. The economics point the same way. A content farm earns from the page load, not the reader, which is the incentive we described in our article on what a free news app costs, and it has no reason to spend money on a check that doesn't raise the load count.
6 checks that don't depend on the prose
Each one looks at something you can see without judging the writing style.
Click the byline. A real reporter has a history: Other stories, a bio with a checkable employer or beat, a presence somewhere other than this site. A name with a studio-quality headshot, no past work, and a bio written in generalities is the Sports Illustrated pattern. A "Staff" byline is normal for wire copy, as we explained in our article on wire services, but wire copy carries the agency's credit line, and a staff byline without one is a question to ask.
Look for something a machine couldn't have gathered. A quote the outlet obtained itself ("told the Tribune"), a document it reviewed, a scene a reporter stood in, a number from a call to the agency. Machine-written news is assembled from other published text, so it attributes everything to "reports" and nothing to a person the outlet spoke to.
Count the output. Open the site's homepage or author page and look at the timestamps. A site publishing hundreds of articles a day under a handful of names has no staff large enough to have written them, which is NewsGuard's second criterion.
Pick one specific claim and search for it. A book title, a named study, a quoted sentence. Real claims leave traces on other sites. The Sun-Times books existed nowhere because they had never existed, and a single search would have shown it.
Look for leftovers. Phrases like "As an AI language model," "I cannot provide," or a bracketed instruction to insert a name are the chatbot talking to the operator who never read the reply. NewsGuard treats them as proof that no human looked at the piece.
Find the disclosure and the corrections page. An outlet that uses automation and says so, with an editor named as responsible, meets the standard the EU law and the AP set. An outlet with no AI policy, no corrections page, and no named editor has told you that nobody is answerable for what you just read.
What these checks can't tell you is whether a named, accountable writer used a model to draft a paragraph and then verified every line. They can't, and for a reader that case doesn't matter, because the verification is what you needed. The checks tell you whether anyone was in a position to do it.
Every summary in our weekday brief at informed.now is chosen and checked by our editors and cites the source it came from, so the 2 questions that matter, who stands behind this and where did it come from, are answered on the page. The 6 checks above answer them everywhere else.
The next time a story reads smoothly and you wonder whether a machine wrote it, stop wondering about the sentences and click the name at the top. If the name leads to a person with a past and the story contains 1 thing that person had to go and get, the machine question is settled well enough. If the name leads nowhere, it doesn't matter who wrote it, because nobody read it.
Have questions about informed.now? Get answers.
Get factual news by text
Free forever with no bias, clickbait, or ads