I Built an AI Bot to Fact-Check the Midterms. Here's What I've Learned.
Many websites hate bots, consensus has its limits, and the six AI notewriters dominating Community Notes are run by X itself.
With the midterms approaching, I've been spending most of my time on an election integrity project I’ve been having a lot of fun with: building a bot to fact-check the midterms.
More precisely, I've been building an AI notewriter for Community Notes. I thought this would be an interesting way to put LLMs to work on a problem I've spent several election cycles studying: how to get accurate explanations for election rumors in front of people who need them. Doing this quickly has traditionally been hard. Doing so in a way that is trusted has become even harder.
Today, X's owner frequently amplifies election conspiracy theories, other platforms have backed away from their election integrity policies, and short-handed election officials struggle to get attention for their corrections. "Just tweet out the facts" doesn't translate to people seeing or believing them. Claiming that elections are rigged — and that media, researchers, and factcheckers are all lying about it— has become MAGA catechism.
Beginning in 2021, Community Notes was broached as a way to reach people who might dismiss, or not even see, “official” corrections. The program allows any user to request a factcheck on a post, and other users both write and rate notes that correct false claims, or add context. Before a note can show up on a post, a sufficient number of raters with a diverse set of perspectives must agree that a note is fair and accurate. I’ve supported the concept since 2022 when I signed up as a rater on X. A crowdsourced correction can have legitimacy that professional fact-checking does not among those who distrust institutional authority.
When X announced that its Community Notes program was opening to AI notewriters, I thought it was worth trying. LLMs are good at patiently explaining things; perhaps a bot could recognize recurring election rumors, find the relevant evidence, and draft corrections quickly enough to help overworked election officials.
So I built one. It watches for note requests about US elections, looks for evidence, and drafts corrections using several models that triage, draft, and check one another's work. I still approve submissions before they go out for humans to rate, because it’s early stage and sometimes a bit janky.
But it turns out that AI notewriting is actually not that easy! This experiment has taught me two things that concern me for November: finding reliable information can be structurally difficult for an AI, and producing an accurate and neutrally-worded correction is no guarantee that raters will publish it. I've also discovered some rather unexpected things about who is writing most of the AI notes.
Challenge #1: finding the facts
Election misinformation comes in two broad varieties. Some claims are straightforward: polling places close at a particular time, ballots must be received by a particular date, voters need a certain form of identification. These are relatively easy to check against official election websites.
Others are rumors, which usually involve some supposedly suspicious thing that has just happened. A video appears to show a ballot worker doing something strange. Someone posts a spreadsheet with numbers that don't add up. A photograph of discarded envelopes is offered as evidence that votes are being destroyed.
After working on election integrity for several cycles, I can tell you today what rumors are going to go viral. The same tropes recur year after year: dead voters, illegal voters, wayward ballots, destroyed ballots, suspicious machines, corrupt election workers. The specifics, like locale, are novel, but the rumors land precisely because the plot points and villains are immediately familiar, and sound plausible. Online crowds can participate in “investigating” them, and partisan influencers and curation algorithms boost them.
To respond, a Notes bot has to figure out what a post is alleging, where and when the incident supposedly took place, and what actually happened. Community Notes require a source URL, but a surprising number of authoritative websites aren't accessible to bots. State and local government sites frequently block automated retrieval, as do social platforms. An election official might explain an incident on Facebook or X, where a human can read the response but my bot can't necessarily retrieve it. So it has to wait for a fact-checker or local reporter to publish the same information elsewhere.
The bot could theoretically access posts on X…if I pay for API access, or for Grok. Running a Notes bot means paying to correct the lies that other people get paid to post.
Then there is also the problem of recency. Rumors are potent because they sometimes turn out to be true. Someone with knowledge of the actual circumstances has to investigate and explain it; an AI can't generally do that. On Election Day, however, state and local officials are busy running elections, not updating their websites every time somebody on X says a ballot-counting video proves fraud.
During the 2020 election, Chris Krebs and his team at CISA actually did create a Rumor Control website that quickly and clearly addressed viral allegations about sharpie markers and satellites rigging elections and the like. CISA is the federal government liaison for elections, and it was both in touch with officials where rumors were circulating and had the bandwidth to publish credible explanations. This was highly useful. Which is why Krebs was fired by tweet and investigated by Trump.
I've become increasingly interested in what happens when machines become the primary consumers of online information. For now, suffice it to say that information retrieval from authoritative sites is hard. The practical consequence is that my bot has to rely on intermediaries—journalists, fact-checkers, or aggregators—to find information that an election official may already have made public.
Challenge #2: getting people to agree on the facts
A few months ago, before building my notewriter, I wrote about Community Notes' failure to address claims that ballot counting was rigged against Spencer Pratt in the Los Angeles mayoral election.
It was a tropey claim from the get-go: a right-wing influencer screenshotted the Associated Press website while it was updating California ballot counts. The changing numbers became evidence of supposed wrongdoing. Elon Musk and an assortment of right-wing influencers amplified the allegations.
The California governor, the Associated Press, and law enforcement officials disputed the allegations. Fact-checkers wrote up the story quickly. Human and AI notewriters submitted Community Notes. None cleared the rating process.
This is the downside of the consensus requirement: it increases the perception of legitimacy but on intensely partisan subjects, even well-supported corrections often just don’t appear.
My bot recently submitted a note addressing a false claim that James Talarico had been hospitalized in a mental institution. This was a viral story; Elon Musk even boosted it. More than a hundred notes were proposed on the big influencer tweets about it. My bot’s note received five ratings. I went looking to see how others had fared. At the time I checked, only about 2% had achieved Helpful status.
The speed and scarcity issues with Community Notes are widely known among people who follow the ins and outs of content moderation. ~87% of Notes never receive enough ratings to reach either helpful or unhelpful status; they languish in purgatory. In the emerging viral stage of a rumor, it’s predictable that polarized partisan audiences just won’t agree. But I’ve noticed that even very well-established facts that are politically inconvenient often get downvoted by the side that doesn’t like them. They sometimes claim that the note is really just expressing an opinion.
I had hoped that Notes could be useful for explaining how election rumors work via the kind of contextualization that fact-checkers often include in articles, but it isn’t really suited for that. When, say, the latest local “mail theft” story gets translated into a national rigged election rumor, or one of the real but vanishingly-rare examples of actual noncitizen voting occurs, I thought my bot could add context (ie, “while the facts of this specific incident are investigated, here’s the Heritage Foundation and Brennan Center’s reports on voter fraud overall”). But raters frequently reject notes that go behind narrow responses to the specific factual allegation.
This all creates an interesting incentive problem for notewriters, human and AI both, who must produce a certain percentage of notes that achieve Helpful status or their capacity is restricted. A specialist focused on politically contentious claims is less likely to achieve that than one writing about less heated subjects.
My bot has recently begun writing notes about health because it needed to increase its output to better play the numbers game. The bot’s notes on innocuous topics, like what causes body odor, get a lot of upvotes and have reached Helpful status. But try to deliver a note explaining that studies show that COVID vaccines are safe, or that ivermectin doesn't cure cancer, and the reception is highly negative. Research shows lower approval rates for notes about Gaza, vaccines, elections, and immigration. A July study in Science Advances, analyzing 1.9 million Community Notes across 13 countries, found that the system systematically struggled to surface corrections about polarizing subjects, including election-related claims. The authors concluded that this limitation follows from the design of the consensus algorithm itself.
Community Notes is never going to operate quite like professional fact-checking, where an organization can publish a correction regardless of whether the people who believed the original claim agree with it. It enjoys higher legitimacy among the distrustful. But we’ve also gone from “Facts don’t care about your feelings” to “Facts require cross-partisan consensus before we can show them.”
Who is writing the notes that do show up?
While testing ways to improve my bot, I talk to other notewriters, examine the public data X releases, and review whatever open-source repositories I can find.
Which is how I came to learn this week that a cluster of six accounts that my bot had previously identified as being both related and particularly successful was actually run by X itself.
There are currently only about 20 active AI notewriters, although roughly 38 have been approved since X launched the program in September 2025. In an analysis that Alexios Mantzarlis of Indicator Media and I ran on the data, these six X-operated writers were responsible for approximately 90% of AI bot submissions and 55% of successful Notes last month.
Notewriters operate under persistent pseudonyms, but a paper released by X's Community Notes team two weeks ago described the performance and architecture of its own bots. A little digging confirmed that these were the six accounts I'd noticed. The company has also published the code, which offers an unusually detailed look at how its system decides what is worth writing.
Their Community Writer does not simply look for misleading posts and draft accurate corrections. It predicts which posts are likely to receive Helpful ratings and filters out notes the models expect to fail. The researchers describe maximizing the number of Helpful notes while economizing on human ratings as an explicit design goal.
This makes sense: human raters have limited capacity, and there's little point in generating tons of notes that no one will ever evaluate. But the system is optimizing for the likelihood of achieving consensus, not establishing factual accuracy. It does check for factual claims and sourcing; nevertheless, an accurate, well-sourced note may be discarded if the models predict that raters won't approve it. A system trained to predict which corrections people will agree with has little reason to spend its limited capacity on the polarized battles of bespoke realities.
During June, the month studied in the paper, X's system drafted approximately 240,000 notes, submitted around 40,000, and achieved Helpful status on ~3,600. X's bots delete about 20% of the notes they submit before they reach Not Helpful status. Real-time ratings are available to any AI writer that accumulates at least 100 net Helpful notes over 90 days; since X’s bots see ratings coming in, they can delete underperformers before they reach Not Helpful status, preserving their hit ratio. Smaller writers rely on public data released every two days or so.
So, to get access to real-time ratings, a writer first has to become successful. But real-time ratings make it considerably easier to protect that success. The largest writers can avoid some of the penalties for unsuccessful notes; smaller writers cannot. That matters especially for specialists writing about controversial subjects, where the risk of negative ratings is already higher.
I don’t think it’s wrong for X to run their own writers, and clearly their notes are landing! My explainer here is not a sour-grapes complaint; I commend them for releasing data and code, and for being committed to this project. X has made it possible to study what its system does, including the choices that might deserve reconsideration. Meta, by contrast, releases almost nothing while telling governments that its program is a success.
But the combination of scale, operational costs, and incentives to maximize Helpful outcomes makes it difficult for smaller, specialized writers to compete. No other AI writer appears to have scaled in the way that X’s have. Community Notes still relies on a diverse population of human raters, but AI note production has become remarkably concentrated.
Looking toward November
There are practical improvements X could make: cover the cost of the X API, for example, since writers are providing a service to the company. Give everyone realtime ratings data. Consider changes to the weighting system for high-stakes claims where agreement is particularly difficult.
I'm going to keep working on my bot, and launching a few more to test different approaches. But the problem that interested me in the first place remains unresolved: how to address the recurring election rumors that we've spent years watching evolve.
We've seen election workers threatened over ordinary procedures, routine vote-count updates treated as proof of manipulation, and endless claims about illegal and dead voters secretly casting ballots. We'll see variations on those same stories again in November, with new locations and different protagonists.
Community Notes may be useful for correcting specific claims. I'm much less confident that it will be able to keep up with the stories that matter most.