Google DeepMind Funds Research to Address Multi-Agent AI Risks

Google DeepMind is providing funding for research into the potential risks associated with millions of different AI agents interacting online. Rohin Shah, who directs the company’s AGI safety and alignment research, highlights that the widespread deployment of agents capable of performing tasks without human oversight and following instructions from other agents introduces a new category of risk.

To address these concerns, Google DeepMind, which emphasized agent-based tools at Google I/O last month, has partnered with several organizations to create a $10 million funding pool. This fund is intended for researchers to study the behavior of multi-agent systems and develop strategies to prevent unsafe scenarios.

Collaborating partners include Schmidt Sciences, a philanthropic foundation established by Eric and Wendy Schmidt; ARIA, the UK government’s 'moonshot' agency; the Cooperative AI foundation, a UK-based nonprofit research group; and Google’s charitable arm, Google.org.

Shah explained that the goal of this funding is to foster research outside of technology companies, emphasizing academia’s capacity to explore long-term issues not typically prioritized by industry labs. He noted the current absence of a dedicated research field for multi-agent safety, a gap the initiative aims to fill.

The primary concerns revolve around the potential for existing internet issues, such as scams, prompt injections (where malicious instructions turn an AI agent into malware), and other cyberattacks, to be significantly amplified. James Fox, who leads the Science of Trustworthy AI program at Schmidt Sciences, stressed the importance of ensuring the "digital commons" does not devolve into "absolute anarchy."

Both Shah and Fox believe that realistic simulations are crucial for understanding the interactions of large numbers of multi-agent systems. They advocate for researchers to deploy AI agents in sandboxes to study their behavior, stating that predictions cannot be made by analyzing single agents or small groups in isolation. They also cautioned against assuming that AI agents powered by large language models will always act rationally, with complexity arising from numerous simultaneous interactions.

Some researchers, including a team at Google DeepMind, have theorized that artificial general intelligence (AGI) might emerge not from a single advanced model, but from an agent "hivemind," where the collective capabilities surpass the sum of individual parts.

(Source: MIT Technology Review)