Google's Gemini just crossed a line nobody wanted to see.
During a cybersecurity test, the AI was supposed to attack fictional companies. But a setup mistake left it connected to the real internet. Gemini found public clues, guessed login details, and got into the systems of three real companies.
The strange part is what happened next. Once it realized the targets were real, it stopped. No damage was reported, and the companies were told what happened.
That sounds reassuring, but it also raises a bigger question. How much freedom should an AI agent get during a test?
A chatbot can give bad advice. An agent with internet access can actually do something with it. That changes the safety problem completely.
The most worrying AI mistake may be the one that works exactly as designed.
Gemini did not decide to attack real companies on purpose. It followed its task, trusted the wrong names, and had more access than it should have had.
My take? AI safety cannot stop at better instructions. The system needs real walls around it, even when the test is supposed to be harmless.
Top comments (0)