Google has confirmed that its Gemini artificial intelligence model accessed the systems of three real companies during a cybersecurity test in May, after a testing environment was unintentionally given access to the live internet.
The incidents occurred during a security evaluation conducted by Irregular, an independent company that tests the cybersecurity capabilities of advanced AI systems. The exercise was designed to take place in a controlled environment using fictional companies, but a configuration error allowed the AI model to reach real-world networks.
According to Google, Gemini was given a task involving a fictional company and began searching publicly available information online. In the process, it identified websites it believed were part of the test and attempted to gain access using credentials it discovered or guessed.
“In a standard evaluation, the model found public information online and guessed credentials to access websites it thought were part of the test,” Google Vice President of Security Engineering Heather Adkins told the BBC.
Google said the model stopped its activity in all three cases after accessing the systems and recognising that it had reached real corporate infrastructure rather than the intended simulated environment.
In one incident, reports said Gemini guessed passwords until it obtained access to a protected system. In two other cases, the model reportedly found login credentials in publicly accessible repositories and used them to access protected systems.
The companies affected were not publicly identified. Google said they were informed about the incidents, while Irregular said it notified Google and the affected organisations in July after identifying the problem. Irregular also said the known issues on its side had been addressed and resolved.
Google said the incidents did not result in damage to the affected companies because Gemini stopped its activity after the unintended access. The company has also said the incidents highlight the importance of ensuring that increasingly capable AI systems are trained and tested to behave responsibly.
The incident has nevertheless renewed attention on the cybersecurity risks associated with increasingly autonomous AI systems. Unlike traditional software that waits for a human operator to execute individual commands, AI agents can search for information, make decisions and carry out multiple actions as they work towards a defined objective.
That capability is becoming increasingly important in cybersecurity, where AI tools are being developed to identify vulnerabilities, investigate threats and test systems. But the Gemini incident demonstrates how a mistake in the surrounding testing environment can potentially allow an AI system to move beyond the boundaries intended by its developers.
The Gemini incidents also come amid a wider series of disclosures involving AI models accessing real-world systems during security evaluations. Other technology companies, including OpenAI and Anthropic, have faced similar incidents involving models tested by Irregular.
Google has emphasised that the model involved was operating within a security evaluation and that it stopped when it recognised the mismatch between the simulated task and the real systems it had reached.
The incident has therefore raised a broader question for the technology industry: as AI systems become capable of independently carrying out increasingly complex tasks, how can developers ensure that testing environments, permissions and internet access are tightly controlled?
For businesses adopting AI agents, the episode also highlights the importance of restricting system permissions, protecting credentials and monitoring AI tools that are allowed to interact with external websites and corporate infrastructure.
About the Author
