DeepMind founder warns: ‘AI must not overtake security’

Artificial intelligence is developing at a rapid pace. According to Shane Legg, co-founder of Google DeepMind, this poses an important danger. According to him, the possibilities of AI should never grow faster than the systems that keep the technology safe and controllable.

The warning comes at a time when the discussion about artificial general intelligence is becoming increasingly serious. Legg thinks that within a few years there is a real chance that a first form of this will become a reality.

“We are now living in a period where the possibilities are advancing very, very quickly,” Legg said in an interview with the Financial Times. “But we must not allow the possibilities to get ahead of safety.”

Concerns are increasing as modern AI systems are able to work more independently. Among other things, they can write software, independently perform tasks and help researchers develop new AI systems.

It is precisely the latter that makes the development sensitive. AI companies warn of a scenario in which powerful systems can ultimately contribute to the improvement of new AI. As a result, there is a fear that technology is developing faster than the means by which people can track and control it.

Dario Amodei, CEO of AI company Anthropic, also recently called for caution. He wants to delay the introduction of the most powerful AI models due to potential security risks, without completely halting development.

Legg calls that thought “interesting” and “worth considering.” At the same time, he emphasises that it remains to be worked out how such an approach would work in practice.

The discussion is becoming more urgent due to the arrival of increasingly powerful models. OpenAI President Greg Brockman stated this month that the latest GPT-6 Astra means that “we are now in the AGI era”. Nvidia CEO Jensen Huang went even further and stated that AGI has arrived according to him.

Legg is not convinced.

AGI stands for artificial general intelligence. Google DeepMind uses that term to refer to a system that has all the cognitive capabilities of the human brain.

According to Legg, it is too early to say that such a system already exists. He even doubts whether Astra meets OpenAI's own definition of AGI, whereby a system must be able to carry out most of the economically valuable thinking.

“They're doing pretty incredible things, of course, but I don't think it's at a point where it can even meet their own definition,” Legg said.

His own expectation does not change in the meantime. Legg still estimates the chance at fifty percent that a "minimum" form of AGI will be achieved in 2028.

Simultaneously with his warning, Legg launches the DeepMind Institute. With this, Google DeepMind wants to broaden the public debate about powerful artificial intelligence and make technical research more understandable for people outside the AI sector.

According to Legg, the time has come for this. “The public debate is really starting now. People take the possibility of powerful general intelligence very, very seriously.”

The institute does not only focus on technology. Topics such as science, education, society, policy, philosophy and human well-being also receive attention. In addition to Google employees, external experts must also contribute.

The first three publications deal with AI security, economic policy and the starting points for a new form of utopian thinking.

In one of those pieces, DeepMind researchers Rohin Shah and Anca Dragan specifically warn of decreasing transparency in powerful AI models. According to them, commercial competition and the rush to release new models can give companies less incentive to provide openness about how their systems reason. According to the researchers, regulation can help to tackle that problem.

Google CEO James Manyika sums up the company's commitment differently. According to him, the only AI race in which Google wants to participate is “the race to do it right”.