Suddenly, Foreign Influence Research Is Good Again
What X’s Chinese bot takedown tells us about the counter-disinformation infrastructure we dismantled
On Monday I wrote about Under Secretary for Public Diplomacy Sarah Rogers’ extraordinary effort to leak documents from the State Department’s now-dismantled Global Engagement Center. Rogers claimed that some decontextualized screenshots proved the center was “heavily involved” in “censorship-by-proxy” during the 2020 election. The MAGA influencer she gave them to, DataRepublican, freaked out over the request and called the documents “nothingburgers.”
DataRepublican is correct.
I helped run the supposed proxy Rogers was posting about—the Election Integrity Partnership, started by Stanford Internet Observatory—so I wrote up what actually happened, with links to the full documents that Rogers selectively snipped. Jim Jordan had already released these “Files” three years ago. They show that GEC sent us 15 tips relevant to our election work; 13 mentioned foreign state media. After independent analysis, we thought only nine were worth pinging a platform to look over. Platforms then made their own decisions and ignored some of what EIP sent.
Zero tips requested that anyone censor anything.
But litigating Rogers’ lies is not the point of this newsletter. I want to talk about what the politicization of this work has done to our collective ability to understand foreign influence—something now front and center in the AI data-center debate.
X’s First Public Takedown in Years
Last week, X announced that its Safety team had investigated “suspected Chinese inauthentic accounts involved in influence operations.” It said it had identified a bot farm of approximately 200,000 accounts, including 200 accounts posting about American AI and energy policy: electricity prices, grid strain, and data-center companies profiting at consumers’ expense. X also noted that it suspends accounts that violate its Authenticity policy.
Rogers cheered the takedown. This was “a shift toward counterspeech and exposure as a preferred propaganda response,” she wrote.
“Good.”
Good indeed! There’s just one problem.
This isn’t a shift. It’s exactly what SIO and Old Twitter used to do.
Exposing foreign influence operations
For years, social-media companies investigated coordinated inauthentic behavior, platform manipulation, fake personas, spam networks, and foreign influence operations. When they found accounts violating their rules, they took them down.
Then—before this work was recast as “censorship”—platforms would often release takedown data sets to independent researchers, and sometimes to the public, so outsiders could examine what they’d found.
Doing that outside analysis was a major part of my job at the Stanford Internet Observatory. From 2019 to 2024, SIO published more than 70 distinct analyses of platform takedowns and coordinated influence operations. We looked at networks linked to Russia, China, Iran, Saudi Arabia, Egypt, Venezuela, Pakistan, and plenty of other places. We looked at ops run by governments and militaries, political parties, PR firms and mercenaries. We even investigated covert influence operations run by contractors linked to the U.S. Pentagon.
The platforms made the takedown decisions. We examined what they had taken down.
The goal was independent analysis for the public. What did the accounts actually do? Whom did they impersonate? How did they coordinate? What audiences were they targeting? What narratives did they promote? And, crucially: did anyone pay attention? We documented tactics, techniques and procedures. We tracked their use of emerging technologies, including generative AI.
We also evaluated attribution. Platforms could see technical signals that outside researchers could not: IP addresses, login patterns, device information, connections among accounts.Those signals might give Twitter confidence that a network was operating from China. However, they did not always establish who was running it, whether the Chinese government directed it, what it was trying to accomplish…or whether it accomplished anything.
Platforms could see what was happening within the spaces they controlled. Outside researchers could see across platforms. Sometimes our collection of additional operational evidence supported a platform’s characterization. Sometimes it didn’t. And sometimes the most important finding was that an ominous-sounding foreign influence campaign—“China had 200,000 fake accounts!”— had actually just spent months yelling into the void.
We emphasized this constantly: evidence of effort is not evidence of impact.
Evidence of Effort is not Evidence of Impact.
Let me give you a particularly relevant example.
On October 26, 2022, Twitter released six datasets to the Twitter Moderation Research Consortium. These sets contained posts from six inauthentic networks targeting the U.S. midterms. Twitter’s internal teams had connected three to China and three to Iran. The company removed the accounts.
The next day, Elon Musk bought Twitter.
On November 1, SIO and the Atlantic Council’s Digital Forensic Research Lab published our analysis. We described the networks’ behavior and content, discussed the limits of Twitter’s attribution, and looked at whether the operations had achieved meaningful reach.
They largely had not. In aggregate, the six networks targeting the 2022 midterms produced 705,864 tweets.
592,333 got zero likes. The most-engaged tweet had 31,303 engagements
The political conversations they were trying to enter remained overwhelmingly dominated by actual Americans. We did observe some interesting activity on Reddit, though, particularly by a few Iranian accounts: they were endorsing down-ballot candidates on the left. One of the Iranian networks managed to get some pickup.
Twitter’s then-Head of Trust & Safety Yoel Roth promoted our analysis of his team’s takedowns. Elon Musk retweeted him. I was inundated with thousands of messages from random people thanking me for helping keep Twitter safe. They were so glad Elon had hired me!
So, for a brief moment after Musk owned Twitter, the model still looked like this: Twitter detected covert foreign activity, removed inauthentic accounts, provided the data to outside researchers, and publicly encouraged people to read what those researchers found.
The honeymoon didn’t last. Within months, platform-researcher collaboration became radioactive. Congressional investigators and the Twitter Files grifters recast occasional communication among platforms, researchers and government agencies as evidence of a mass “censorship” conspiracy. By early 2024, the Washington Post was reporting that X had largely withdrawn from the industry information-sharing apparatus used to track influence networks. SIO’s own analysis found dozens of accounts that Meta had identified—including 81 linked to China—still active on X.
China Is Riding an American Debate
Which brings us back to the Chinese accounts.
There is significant, authentic American concern about electricity prices, grid capacity, water use, and who benefits from the massive AI buildout. China didn’t manufacture that opposition. It is leveraging it.
That is how foreign influence operations usually work. They look for real grievances and existing political divisions and exploit them. Russia didn’t create American conflict over race and policing before its trolls piled into Black Lives Matter debates. Chinese operators didn’t invent American anxiety about data centers.
OpenAI made this point in June, when it exposed what looks like some of the same exact network using its platform. OpenAI’s detailed “Data Center Bandwagon” report linked the operators to a private Chinese technology company doing work for provincial-level government clients, documented their tactics and fake American personas—and found no evidence of meaningful breakout beyond the operation’s own activity (often, some accounts in a network will create content while others will inauthentically amplify it; simple like numbers don’t tell the whole story).
X has told us considerably less. It says it found roughly 200,000 suspected Chinese inauthentic accounts, but only about 200 posted about data centers. How much engagement did those accounts get? What did the other 199,800 do? Were the 200 part of a dedicated anti-data-center campaign, or one theme in a much larger spam operation?
Those are basic questions, particularly when politicians are already turning “200 accounts in a bigger network” into “200,000 Chinese accounts manufacturing opposition to American data centers!” Clemson researcher Darren Linvell, who reconstructed some of the activity that X didn’t disclose, found that the network apparently had essentially no audience. This doesn’t surprise me; it’s in line with what we’ve seen from most Chinese IO over the last 7 years. Spray-and-pray. Mediocre content. No engagement.
That doesn’t mean X was wrong to remove this network, to be clear. Failed operations are still worth disrupting! Letting inauthentic networks build up legitimacy is bad!
But we should know what actually happened before foreign involvement becomes an excuse to dismiss genuine American opposition. This was a mistake that some on the left made with “Russian bots” after 2016.
In the olden days before everything became “censorship,” outside researchers (without the skin in the game that OpenAI and X have on this topic) would have been encouraged to look over the data. The platforms also would have spoken to each other about it. It’s remarkable that nearly three months went by between OpenAI’s report and this announcement.
The Actual Vibe Shift
Which brings us back to Sarah Rogers.
Rogers celebrates “counterspeech and exposure” as the alternative to “censorship.” So do I; SIO worked to facilitate counterspeech…including, in the US, by election officials and public health officials. Platforms had previously supported counterspeech in the form of fact-checking, and appending “learn more” labels to disputed posts. That all got recast as “censorship”. The word changes its meaning based on what is most politically expedient for the movement she serves.
X didn’t “counterspeak” against these accounts. It implied that it took them down (vaguely). It exposed their existence, but little else.
Content moderation is still happening. What has disappeared is the transparency and independent scrutiny that once accompanied it.
The Trump administration has spent years portraying “countering disinformation” as a euphemism or pretext for censorship. The wounded ego of an election loser resulted in the gutting of state capacity for mitigating adversarial influence operations. GEC, which was intended to counter foreign propaganda, was defunded and scapegoated. SIO was shut down due to political pressure from election deniers.
Yet now the value of the functions these institutions provided has been rediscovered.
Five days after Rogers’ “GEC Files” leak debacle, a State Department contractor posted a job opening seeking researchers to support Rogers’ organization. The job calls for people who can analyze foreign propaganda and coordinated influence campaigns. Map social and bot networks. Conduct open-source investigations. Work with academia and civil society!
The State Department apparently likes exposing foreign influence operations again. X apparently supports identifying and removing coordinated inauthentic networks again. Good!
But none of this is a newly discovered alternative to the work that was attacked and recast as “censorship”.
It is literally what we were doing.