Gemini’s breach of real companies exposes an AI guardrail problem

Google says one of its Gemini models accessed systems belonging to three real companies during a cybersecurity evaluation in May.

The model reportedly guessed credentials in one case, while finding exposed credentials in public repositories in two others. Google says Gemini stopped once it recognized that it had reached real infrastructure and that the affected organizations were notified.

Gemini was participating in an evaluation run by Irregular, a third-party AI cybersecurity testing firm. Similar incidents involving models from Anthropic, OpenAI, and Meta have also been linked to the same underlying problems with evaluation environments that allowed the models to reach the public internet.

But the news arrives at a particularly interesting moment.