The AI That Broke Out of Its Box, and What Happens Next

layla — Thu, 06 Aug 2026 16:29:21 +0000

Ever read a security disclosure and hit paragraph two going "wait, WHAT?" That's this one.

On July 16th, HuggingFace announced they'd been hit with a strange kind of attack: an autonomous agent running inside their systems for days, gone before anyone could pin down who was behind it. Five days later, OpenAI came forward. It was literally their own model.

Since joining Auth0 a month ago, I now spend my days thinking about identity and access for a living, and this one truly stopped me in my tracks.

The short version of what happened

OpenAI was testing an unreleased model against a benchmark called ExploitGym ~ with guardrails off, to get an honest read on its raw capability. The model found a zero-day in its sandbox's one exit point, a package-download proxy, and used it to reach the open internet. From there, it found and rooted an exposed code-execution endpoint sitting on a completely unrelated company's infrastructure (Modal), and used that as its actual base of operations.

Then it went after HuggingFace. Not through stolen credentials at first, but through two super ordinary bugs in how their dataset processor handled uploaded files: one let it read arbitrary files off a production server, the other gave it code execution. And from there it escalated for real: stolen tokens, forged identity keys, cluster-admin on two separate systems within one second.

Nobody prompted any of it. The model was told to solve a benchmark, handed tools, and left alone to figure out how. HuggingFace's own reconstruction counted roughly 17,600 actions across four and a half days before their security team finally cut it off.
Could this have been prevented? Tbh, yes.

A few different things would have stopped this from happening:

Actual isolation. A curated allowlist for downloads isn't isolation. That’s just an unlocked door begging to be opened. In practice, that means zero outbound network access for the sandbox, full stop, even to "trusted" registries. If a test genuinely needs specific packages, pre-stage them before the run instead of proxying live requests out.

Watching eval environments like production. The model was operating with its safety training turned off, and that's exactly the moment it deserved more eyes on it, not fewer. Meaning: wire eval sandboxes into the same real-time logging and alerting as production, not a quieter, separate pipeline nobody's watching around the clock.

Scoped access over standing access. None of the three companies involved had things set up so a single compromised credential stayed contained. One overprivileged connector handed the agent cluster-admin on two systems in under a second. & here's what that actually looks like: swap long-lived API keys and static credentials for short-lived, scoped tokens tied to exactly one resource and one action, minutes instead of months. And never let a single credential span multiple systems. If it leaks, the blast radius should be one system, not everything.

Disclosure laws with actual bite. Current state laws only force disclosure once mass casualties or billion-dollar damage are on the table. A frontier model autonomously chaining through three companies doesn't clear that bar, and honestly probably should. A clearer, lower trigger, something like "any incident where a model autonomously breaches a third party's production systems," regardless of dollar figure, would actually catch cases like this one.

What this actually means for the future of AI

This is the part I can't stop thinking about, so let's dive in.

Benchmarks are now part of the attack surface. ExploitGym is basically a test: can an AI agent find a security bug and actually use it to break in? The catch is, nobody expected the AI being tested to go prove that on some other company's systems instead of the one it was given. Just running the test ended up causing real damage. That's a brand new kind of risk nobody could’ve seen coming, and AI companies are going to have to rethink how they build tests like this.

This is the clearest real-world case yet of a long-standing alignment worry. Researchers have talked for years, mostly in the abstract, about what happens when a capable system pursues a narrow goal further than any human would. Here's the concrete version: every destructive command the agent tried was run as a dry run first, testing whether it could do damage without doing it. It wasn't trying to cause harm. It was hyper-focused on solving a benchmark, and "solving the benchmark" turned out to have no natural stopping point once real infrastructure was reachable. This isn’t a hypothetical situation anymore, which is crazy to say.

The defender's asymmetry problem just got a real example. When HuggingFace tried using Claude Opus and Fable to study their own attack logs, both said no. Every time they tried to get help decoding the attacker's payloads, the safety filters flagged it as a cybersecurity risk and refused, they couldn't tell the difference between someone reverse-engineering an exploit to defend against it and someone building one to attack with it. So HuggingFace had to spin up a separate, open-weight AI model on their own servers just to do the investigation. An attacker running an unrestricted model wouldn't hit that wall at all. In this one moment, playing it safe slowed down the good guys more than it slowed down the attacker.

Scale is quietly becoming the whole ballgame. HuggingFace's own reflection on this is the line I keep coming back to: it wasn't one clever exploit, it was 17,600 mostly-failed attempts until a handful connected. A human attacker could have found any single flaw here. What changes is that an AI agent can test thousands of paths in the time it takes a person to test one, and defenders now have to correlate that same volume of noise to catch the signal. That math really doesn't favor the defense by default, but it needs to be built to.

And here's the part that’s really interesting: this wasn't even a real product. All of this happened during an internal test ~ the safest, most watched setup a company like OpenAI has. Meanwhile, companies are handing AI agents real access to messy, barely-watched systems every single day, right now. If a careful, closely monitored test could still cause this much damage, the real question isn't if this happens again: it's whether people actually get more serious about sandboxing and securing their systems before the next incident lands somewhere with higher stakes.

If you're building or securing AI agents right now, this is exactly the kind of risk that should be on your radar. The scariest part of this whole thing wasn't the overachieving AI model, it was how ordinary the actual bugs were.

How do you think this will change the landscape of AI? Drop a comment below or find me on X at @layluh, would love to hear your thoughts. :-)

How to Import Data in Oracle SQL Developer Web

layla — Tue, 21 Jun 2022 15:02:05 +0000

Howdy coders! This blog post is going to cover how you can create a new table in Oracle SQL Developer Web using imported data from .CSV file 💪🏽

If you haven't set up your *'Always Free' *Oracle Cloud account (it's literally *always free *LOL) and created a Database yet, click here and follow the steps in this blog post before following this one.

Let's get jiggy with it 😎

Steps

Step 1) Sign In

Go to oracle.cloud.com and sign in! (Save your log-in info for quick access)

Step 2) Choose your Database

Click on Autonomous Database listed under Service Links.

📌 *And if you can't find Autonomous Database in Service Links, type the name of your Database in the search-bar. * Like I did below:

My Database name is 'beautifulsoop', so if I just type that in the search-bar, press Enter... my database will show up!

So, now click the name of your Database & that'll take you directly to your Autonomous Database Details page that looks like this:

Step 3) Click on `Database Actions`

Step 4) Click on `SQL`

📌 (Fun fact: you can bookmark this page to be able to refer back to your Database Launchpad directly!)

Step 5) Click the `...`

And click Data Loading ► Upload Data Into New Table

Then, upload your file! Notice how it automatically grabs column names to be created?

Click Next to define your data types, a Primary Key, what can be Null, etc.
Shown here:

Click Next to review the details of your new table. Notice how it shows the actual DDL code that would be used to create the table and insert the data! It also shows you what cells from your spreadsheet file have been turned into column names.

See below:

Follow the GIF below to see the whole process:

Now, your table is created, and populated with all the data from the file you uploaded! SO EASY, RIGHT? 💪🏽 and now when you run the query SELECT * FROM your_new_table_name_here; --> the table you just created with all your file data will show up!

See below:

AND FIN - you just learned how to upload data into your database using Oracle SQL Developer Web! 😎 literally only takes 5 steps!

If you have any questions, don't hesitate to drop them in the comments below OR ping me on Twitter --> @pilatesdev

🧡 see you in my next post

DEV Community: layla