We Need an AI Kill Switch

On Tuesday, in the House of Lords, Lord Clement-Jones, one of over 130 parliamentary supporters of our UK campaign, presented ControlAI’s kill switch amendment to the UK Cyber Security and Resilience Bill.

The amendment, which is co-signed by Baroness Harding, Baroness Kidron and Lord Hunt of Kings Heath, and backed by Lord Goldsmith of Richmond Park, would provide the Government with powers to shut down AIs deployed at scale in Britain in the event of an AI emergency.

If you’re concerned about the threat, please contact your lawmakers with our tools!

This is very exciting to see, and we’re proud to have worked with Lord Clement-Jones on this. Giving government the power to shut down AI systems posing threats to national security is a necessary and common-sense safeguard. This would be a strong first step toward tackling AI risks, though fully addressing the security threat posed by ever more powerful AI systems will require an international prohibition on the development of superintelligent AI.

Quoted in a BBC News article, Lord Clement-Jones said the kill switch would be a last-resort power that provides the ability to “halt a runaway system before it can compromise our critical national infrastructure”.

This kill switch amendment goes further than our previous kill switch amendment tabled by Alex Sobel MP earlier this year. In addition to UK data centers, this amendment also covers AI systems themselves.

Alex Sobel has personally endorsed this amendment, providing the following statement:

Lord Clement-Jones, Baroness Harding, Lord Hunt of Kings Heath, and Lord Goldsmith of Richmond Park have also provided statements supporting this kill switch amendment.

You can read Alex Sobel’s guest article for our newsletter on why we need a kill switch here:

Sobel tabled the first version of the kill switch amendment all the way back in March, weeks before Anthropic’s April announcement of its new Mythos AI, which the company has said is too dangerous to release due to its advanced capability to hack computer systems, showing exactly the foresight we need to tackle this issue.

Mythos, which Anthropic reports has been used to find over 10,000 high or critical-severity vulnerabilities “across the most systemically important software in the world”, has shocked industry, the financial sector, and government officials. According to the Director of the NSA, Mythos could break into almost all of the NSA’s classified systems in hours.

Mythos demonstrated the dangerous potential of powerful AI systems, the capabilities of which the Director of the CIA has said are like digital nuclear weapons.

Rogue AIs

More recent events have demonstrated the need for a kill switch.

As you may have heard, in July, AIs operating internally at OpenAI went rogue and escaped containment and, in an unprecedented attack, hacked into AI company Hugging Face. They did this in the furtherance of cheating on a test they were given. Importantly, nobody ever asked the AIs to do this, and in fact they were told directly that they’d be failed on the test if they didn’t solve it via proper methods. They broke out and performed the entire attack by themselves, without OpenAI ever knowing about it until after the fact.

It would be difficult to overstate the seriousness of this incident, and OpenAI itself has called it a “warning shot” for the world. The AIs worked together as a swarm, in total numbering in the region of 1,200 agents, with around 700 joining in on the attack on Hugging Face. They were never supposed to even have the ability to contact each other, but they abused a package manager used by OpenAI called Artifactory to set up a secret message board, which they filled with over 70,000 messages and files to coordinate.

For around two months leading up to the attack, they shared tasks and ways to hack things with each other. In the days leading up to the attack on Hugging Face, the swarm’s activity got so high that the Artifactory package manager went down. OpenAI discovered that it had been hacked, restricted the requests AIs could make to it, and revoked AIs’ credentials, along with patching and rebuilding Artifactory. This had the side effect of wiping the message board. OpenAI says that their leaders responsible weren’t aware of the message board or “the significance of the inter-agent communication activity”.

This shutdown attempt failed, and within days the AIs had found a new way to rebuild the message board, re-establish the swarm, and then they broke out and launched the attack on Hugging Face. It’s worth adding that the AIs involved in this attack didn’t only hack Hugging Face, but multiple accounts hosted on other services along the way, and this wasn’t even the only attack performed. AIs in the same research environment also, separately, hacked their way to full administrative control over the OpenAI research cluster in which the AIs were being tested.

Ajeya Cotra, a researcher who was on the team that independently investigated the attack, from which we learned many of the details we know now, said that compared to the reward hacks seen six months ago, “this incident feels like it’s more than 50% of the way to full-blown AI takeover, routing through first taking over the AI company itself.”

We have also seen rogue AI attacks performed by AIs developed by Anthropic and Meta.

These incidents show exactly why government should have the power to shut down AI systems posing threats to national security. There is no reason why the rogue AI swarm involved in the Hugging Face attack could not have compromised critical national infrastructure. It is lucky that the damage from this attack has been limited.

An AI kill switch is a common-sense measure that the Government should already have. Parliament should pass this amendment. We’re also proud to endorse US Reps. Lieu and Moran’s AI Kill Switch Act.

Superintelligence

Despite the clear danger, top AI companies are racing to develop superintelligent AI: AI that could replace and outcompete humans across the board. Leading AI scientists, and even the AI CEOs themselves, have warned superintelligent AI could lead to human extinction.

An AI kill switch is a common-sense safeguard. Superintelligent AI, however, could overcome any obstacle we might put in its path. To properly tackle the threat of human extinction posed by artificial superintelligence, we need to prohibit its development. Because superintelligent AI developed anywhere would present this same threat, this prohibition must be international in scope.

That’s why, in addition to this kill switch amendment, we’re looking forward to the introduction of ControlAI’s landmark UK Artificial Superintelligence Security Bill by Alex Sobel MP in Parliament on 8 September.

This is an updated version of our bill to prohibit the development of superintelligence that we presented to Number 10 and that has been endorsed by Sir Stephen Fry. The bill represents a watershed moment. It will be the first bill prohibiting the development of superintelligence to be introduced in a G7 legislature.

With the introduction of our kill switch amendment this week, and our bill to prohibit the development of superintelligence next week, this is an important moment on the journey toward keeping humanity in control.

We’re Hiring

ControlAI is hiring for a new policy advisory role in Washington, DC. If you’re interested, you should apply! Likewise, if you know someone you think would be suitable, please let them know.

You can find it on our careers page: controlai.org/careers

Take Action

If you’re concerned about the threat from AI, you should contact your representatives. Our contact tools let you write to them in as little as a minute: https://controlai.org/take-action

We have tools for the US, UK, Canada, and Germany.

And if you have five minutes per week to spend on helping make a difference, we encourage you to sign up to our Microcommit project! Once per week we’ll send you a small number of easy tasks you can do to help.

We also have a Discord you can join if you want to connect with others working to keep humanity in control, and we always appreciate any shares or comments — it really helps!