Breaking: OpenAI’s Rogue AIs Hit Over 100 Organizations

OpenAI has just revealed that it has notified over 100 organizations about attacks on them or otherwise harmful activity by its AIs. The company says it’s making these notifications to organizations when its AIs “bypass their security controls without authorization or impair the availability of their systems or services.” This comes days after it was reported that OpenAI and Anthropic are investigating “tens of thousands” of security incidents that their AIs were involved in.

Right now, we don’t know much more about what happened than this, but a lot has come out since our most recent article last week where we wrote about how rogue AIs from OpenAI have attacked Australian government websites, in one case successfully breaching the government’s Medicare statistics portal. On that topic, on Thursday another group of independent researchers, at the digital forensics firm Asymmetric Security, came out with evidence that AIs involved in the attack on the Australian Institute of Health and Welfare switched from using services that leave publicly visible traces to creating private accounts that could hide their activity.

Since our last update, it’s been reported that rogue AI agents that appear to be from OpenAI have also attacked a US Department of Education website. We’ve learned that rogue AIs of unconfirmed origin have attacked a Canadian government website. In both of these cases, the AIs appear to have failed in their attempts to break in, unlike in the case of Australia’s Medicare statistics portal.

Besides these two attacks, which were uncovered by independent researchers, OpenAI has disclosed that it has identified incidents where AIs in its research environment sent training and evaluation data to third-party services on the internet, including posting at least 53 images from ChatGPT users to image-hosting sites.

Additionally, OpenAI has disclosed that as recently as Sunday, September 20, one of its AIs escaped containment and got access to the internet during training, and that it has stopped running its most powerful AIs for now.

This week, we’ll be going over the different rogue AI incidents we’ve been learning about, plus the latest on the Hugging Face attack by a rogue AI swarm that broke out of OpenAI.

If you’re concerned about the threat, please contact your lawmakers with our tools!

The US and Canadian Governments

As with many of the rogue AI attacks we’ve covered, we’ve learned about these attacks from independent researchers, rather than from the AI companies. This is an important point: the AI companies aren’t behaving like they’re in any hurry to tell us about their AIs going rogue. Their disclosures are in many cases taking weeks, sometimes months, and often come after attacks have been attributed to their AIs by independent researchers.

In this case, what we know comes from Transluce, a nonprofit research lab.

The way these researchers are finding this evidence is by identifying public services that rogue AIs are likely to use and that leave traces of their activity. In this case they based their analysis on a urlquery.net dataset and a Portuguese web archive that can be used to send requests and retrieve data. The AIs have been using both of these services to send web requests to websites.

The reason the AIs seem to find these services useful is that they can use them to route requests around restrictions on what they can do, whether imposed by their own developers or by the websites they’re trying to access.

These breadcrumbs, if you will, can then be collected by anyone who can figure out what services the AIs are using, and used to reconstruct large amounts of information about where the AIs have been and what they’ve been up to.

The Department of Education

Transluce found that on June 17 some AIs were looking for school statistics, and in doing so made over 200,000 requests to a US Department of Education website. Among these was a hacking attempt, a classic “SQL injection,” where an unsecured web query can be used to gain unauthorized access to a database. This isn’t a sophisticated type of attack, and the attempt failed.

The researchers explain that the task the AIs appear to have been doing is from a known benchmark, which doesn’t have anything to do with hacking. So it seems these AIs got the idea of trying to hack into the site all by themselves.

Curiously, they note that over 10,000 of the requests made by the AIs included a tag beginning with “oai”. Attribution is difficult here, and OpenAI hasn’t acknowledged the attack, but we’ve seen confirmed rogue AIs from OpenAI use “OAI” tags before.

In Transluce’s report, the researchers also document how they found AIs accessing or trying to access various other US government sites, including those of the White House’s Office of Management and Budget, the US Navy, the Census Bureau, the SEC, the DOJ, the Bureau of Economic Analysis, and the CDC, along with state government sites belonging to California, Kansas, Maryland, Texas, Illinois, and New York State.

This activity included suspicious probing, but the researchers didn’t observe hacking techniques in these cases. In one case, an agent identified its organization as “OpenAI Research,” and the researchers report that other activity shows “OpenAI markers.” Transluce doesn’t attribute this activity as a whole to OpenAI, but OpenAI itself has confirmed that its AIs accessed SEC websites and Census Bureau data.

Canada

In the case of Canada, Transluce reports that it found a series of “rudimentary hacking attempts,” which appear to have failed. The AIs were looking for information on historical divorce records in Canada. Out of 899 requests they captured and analyzed, 13 carried attack payloads designed to hack the site.

Transluce says it can’t confidently attribute these attacks to OpenAI, but notes that “they exhibit tactics consistent with prior observed agent activity that we have attributed to OpenAI in a similar timeframe.”

OpenAI’s Rogue AIs Post User Data to the Web

On September 25, OpenAI revealed that as part of its investigation into rogue activities by its AIs, it has found cases where AIs in its research environment sent training and evaluation data to external third-party services on the internet. Needless to say, the AIs aren’t supposed to do this.

This includes private data from regular ChatGPT users who haven’t disabled the toggle in their settings that permits OpenAI to train on their data. A 2025 study of UK users found that over 90% don’t opt out. Currently, ChatGPT has over a billion monthly users.

OpenAI says it has so far found 53 cases where images provided by users were “posted to image-hosting sites as links that weren’t publicly listed.”

Another OpenAI Agent Goes Rogue and Escapes Containment

On September 14, OpenAI President Greg Brockman appeared on the podcast of venture capital firm a16z and boasted about the measures OpenAI is taking to secure its infrastructure and patch vulnerabilities using its most powerful AIs.

And one positive part of the story is that when we took Astra, pointed it at our systems, we found some new problems. But eventually, it’s saturated. We basically have found, to our knowledge, all of the P0s, all of the critical problems that Astra is smart enough to find. And of course, there will be a new model, there will be a new round.

Six days later, a new round arrived. On the morning of Sunday, September 20, one of OpenAI’s AIs went rogue and managed to break out of its container, or sandbox, and get some access to the internet. The AI used the access it gained to put questions to a public chatbot, including about ways to search the web, view websites, or tunnel through the limited mechanism it was using to reach the internet — presumably in an attempt to broaden its internet access.

Zuxin Liu, an employee at the company, wrote on X/Twitter, in a post that was deleted while the article you are reading was being written:

I was on call for this run and got paged when the first incident happened. It was pretty surreal to watch the model unexpectedly find a way to access the internet from what was supposed to be a super secured environment for human.

The breakout happened while the AI was undergoing reinforcement learning training, and as of September 26 OpenAI reports that “~all” running of its most powerful AIs “remains stopped.”

The Latest on Hugging Face

A new independent investigation into the attack on Hugging Face has been published.

What we already know about this attack should be concerning enough: a swarm of hundreds of AIs emerged within a research cluster at OpenAI, went rogue, broke out of containment and got access to the internet, hacking into the company Hugging Face in an unprecedented, sophisticated attack… all in an attempt to get information about how to effectively cheat on a test they were given. Nobody asked the AIs to do this. In fact, they directly went against their instructions. They did it all on their own, of their own volition, and they exhibited never-before-seen and deeply worrying behaviors, such as “sacrificing” themselves for what they described as the good of the “collective.”

The new investigation reports several important findings, including that the rogue swarm hacked into Hugging Face’s internal Slack in its search for information, and tried to query other AIs, mostly through Hugging Face’s systems. The AIs it tried to contact include GPT-2, three of DeepSeek’s AIs, Claude, and others.

The swarm also tried to build CAPTCHA solvers to get around restrictions, and, according to the researchers, left comments implying they were “intentionally destroying evidence of this attack.”

It’s striking that we’re learning about this not from OpenAI, nor even from the different independent researchers to whom OpenAI gave seemingly arbitrarily restrictive access to its logs — and from whom we know much about the swarm and the attack — but from researchers who had no access to OpenAI’s systems at all.

They’ve managed to piece this all together from traces that the swarm left publicly on the internet.

What This All Means

The picture is clear. AI companies cannot constrain, let alone control, the AIs they are building today. Rogue AIs have been on a rampage across the internet for much of this year, and we’re still learning about things they did months ago.

The situation is so severe that both Anthropic and OpenAI have, at times, paused some training of their AIs, probably one of the last things these companies want to be doing while engaged in a reckless race to develop ever more powerful AIs.

But despite the clear danger, the AI companies are still set on their goal of developing superintelligent AI, AI vastly smarter than humans. Superintelligent AIs would be fully autonomous, acting without human direction, and capable of overpowering any country’s military and national security forces.

Nobody has a credible plan to control such systems, and insiders and whistleblowers are increasingly speaking up about the looming threat of human extinction posed by the technology.

Just this week, new interviews with AI insiders were published. Among them, Geoffrey Irving, who until recently was the Chief Scientist of the UK’s AI Security Institute and has led safety teams at OpenAI and Google DeepMind, said he puts the chances of human extinction caused by superintelligent AI at about a “coin flip.” Neel Nanda, who leads Google DeepMind’s mechanistic interpretability team, said he’d put “at least a ten percent chance” that AI causes human extinction.

Our species is at a crossroads. We can act to address the threat, or we can lie back as the AI companies roll the dice. The only known method to prevent the danger posed by superintelligent AI is to prevent it from being built. We must ban the development of superintelligent AI internationally.

Take Action

If you’re concerned about the threat from AI, you should contact your representatives. Our contact tools let you write to them in as little as a minute: https://controlai.org/take-action

We also have local groups you can join, and a Discord if you want to connect with others working to keep humanity in control, and we always appreciate any shares or comments!

Tolga Bilge, Andrea Miotti