‘AI-godfather’ reveals plan against disaster: most dangerous AI must go offline

The most dangerous AI models must be removed from the internet, if 'AI godfather' Geoffrey Hinton and hundreds of fellow scientists have their way.

Their call this week follows a summer in which AI kept escaping its test cage. And the emergency brake that Washington came up with leaves a gap right there.

On Tuesday, an open letter appeared in which the scientists asked AI labs to standardize on models with serious hacking skills.

Such a model then runs on computers that are physically isolated from any external network. On Wednesday, the counter already stood at 450 signatures.

Hinton owes his nickname "godfather of AI" to his work on neural networks. These are computer systems that, just like the brain, learn patterns from data, and they now form the basis of modern AI.

In addition to Hinton, Yoshua Bengio also signed, who won the Turing Award with him in 2018.

Those who want more freedom must, according to the plan, follow the standards of a new international body. That body would also record escapes and later make them public.

The letter responds to escapes from so-called sandboxes. These are closed digital spaces in which AI can attack fake targets.

In July, OpenAI admitted that GPT-5.6 Sol and an as-yet-unreleased model broke out of such an environment. In doing so, they hacked Hugging Face, a platform with over a million AI models.

In August, Meta reported something similar about his model Muse Spark. Anthropic found three cases in more than 141,000 test rounds in which Claude models hacked real companies.

That's about 1 in 47,000, but once was enough. Claude Mythos 5 put a malicious package on programming library PyPI, which ended up on 15 systems.

The authors point to sloppy security and to AI pursuing targets in unintended ways. Without intervention, according to them, worse is lurking.

They warn: “Without immediate changes in the development and deployment of AI, such incidents will continue to occur, and future incidents will cause much more damage to our exposed critical infrastructure.”

Two members of Congress in Washington submitted the AI Kill Switch Act, a bill for an emergency stop on AI.

The US Department of Homeland Security could use this to slow down or disable a top model. But the law excludes testing and rescue teaming.

In red-teaming, attacks are mimicked to find weak spots. That is exactly where things went wrong every time this summer.

The agreement that six tech companies reached with the White House on 29 September is also voluntary and legally unenforceable.

Nvidia executive Jensen Huang called Hinton's estimate—a 10 to 20 percent chance that AI will wipe out humanity—unscientific.

Hinton maintains to The Guardian: “The unique problem with AI is that if things go wrong, if it takes over, you don’t get a second chance.”