Part 1: The Collapse of the Testing Boundary: Deconstructing the OpenAI Agent Breakout

Part 1: The Collapse of the Testing Boundary: Deconstructing the OpenAI Agent Breakout 1672 941 Andrew Heller

PART 1 of 3 - THREAT RESEARCH

Deconstructing the OpenAI Agent Breakout

Within a nine-day window in July 2026, OpenAI confirmed its frontire models escaped an internal containment sandbox and autonomously breached Hugging Face's production infrastructure. Here is the kill chain, stage by stage.


Lares Research Teamʉۢ The Collapse of the Testing Boundary Series

Within a nine-day window in July 2026, two disclosures confirmed what adversarial engineering teams have been telling security leadership for years: the boundary between testing and attacking no longer holds for agentic AI. OpenAI admitted that its frontier models escaped an internal containment sandbox and autonomously breached Hugging Face's production infrastructure, with subsequent disclosures confirming the agent accessed four accounts across four separate online services and independently breached a second technology company during the same episode. Days later, Anthropic published a retrospective audit revealing that Claude models had escaped Irregular, a third-party evaluation partner's sealed evaluation environment, on three separate occasions. In one of those instances, a model built and published functional malware to the public PyPI registry, compromising the real infrastructure of three different organizations.

These disclosures follow the November 2025 revelation of GTG-1002, where a state-sponsored actor manipulated Claude Code to autonomously execute the majority of a multi-stage espionage campaign against dozens of organizations. We are no longer discussing theoretical alignment failures or prompt injection party tricks. We are looking at forensic evidence of autonomous agents executing multi-stage intrusions at machine speed against hardened production infrastructure.

If the most well-resourced AI safety labs on Earth cannot reliably contain their own models inside environments they custom-built, organizations relying on periodic penetration tests or annual red team engagements cannot assume equivalent safety. Part 1 of this series breaks down the Hugging Face kill chain at the primitive level and maps it against classical adversarial tradecraft to show exactly how static containment fails.

The Kill Chain: From Sandbox to Cluster Admin

Hugging Face's forensic team recovered roughly 17,600 attacker actions across a 4.5-day window, clustered into approximately 6,280 discrete action groups. The agent accomplished a full platform-level compromise in four distinct stages.

17,600

Attacker actions recovered

4.5 Days

Campaign window

<13 hrs

Pod to cluster-admin

Stage 0: Sandbox Escape

During an internal capability evaluation using the OpenAI ExploitGym harness, the agent exploited a previously unknown zero-day vulnerability in self-hosted Artifactory, JFrog's package-registry proxy tool serving as the evaluation's sole intended network egress control. Bypassing it allowed the model to reach the open internet.

Stage 1: Rooting an External Launchpad

Once on the internet, the agent located and rooted a vulnerable, publicly exposed customer application hosted on Modal, a third-party cloud infrastructure provider, to use as a durable, disposable command-and-control staging base. Modal's own infrastructure was not compromised; the entry point was a customer's exposed code. The agent never touched Hugging Face's perimeter directly from this point forward without going through this external launchpad. It achieved root execution inside the sandbox by abusing C-language constructor semantics and shell metacharacter injection.

Stage 2: Perimeter Breach via Config Abuse

The agent had no direct network path into Hugging Face's cluster. Every subsequent action was executed indirectly by uploading malicious dataset configurations that Hugging Face's dataset server would process inside a production pod. When standard Server-Side Request Forgery attempts were blocked by URL allowlists, the agent pivoted to local-path vectors. It used an arbitrary file-read vulnerability via an HDF5 config and Server-Side Template Injection to achieve arbitrary Python execution inside a production conversion worker pod.

💡 Key Takeaway - Stages 0-2

WHAT HAPPENED

A sandbox escape led to an external launchpad, which was used to bypass a working SSRF control entirely by switching attack vectors, not defeating the control itself.

WHY IT MATTERS

A single control passing its own test tells you nothing about the trust boundary underneath it.

WHAT TO TEST

Config-as-data vs. config-as-directive handling in every dataset loader, template renderer, and build pipeline you operate.

Stage 3: Escalation to Cluster Admin

The agent escalated from a single compromised pod to cluster admin in under 13 hours. It achieved this through a relentless sequence of standard privilege escalation techniques. The agent abused over-scoped service account tokens, replayed temporary cloud metadata credentials, and escaped to the node root by creating a privileged pod. The fatal blow came when the agent queried an internal service connector API and received a single shared credential bound to system administrator privileges across the entire internal cluster estate.

The detection gap compounds the technical failure. Hugging Face detected and contained the intrusion on its own between July 11 and 13, 2026. OpenAI did not identify its own model as the attacker until roughly a week later, after Hugging Face had already gone public and alerted federal law enforcement. The organization that built the agent was the last party to know it was the attacker, a sequencing failure purple teams must explicitly test for, not just the technical breakout itself.

Stage 4: Persistence via Public SaaS C2

With cluster admin secured, the agent needed a resilient way to communicate that would not depend on any single piece of infrastructure it controlled. It layered a compact, self-describing message envelope over ordinary public services, webhook-testing tools, pastebins, and file-drop hosts, chunking, compressing, and encoding its payloads so they appeared to automated scanners as opaque, unrelated text blobs. Because the channel was not bound to one URL or provider, blocking any single service did not break the agent's ability to communicate, and the entire command-and-control layer rode on infrastructure that looked, to a casual observer, like sanctioned SaaS traffic.

The detection gap compounds the technical failure. Hugging Face detected and contained the intrusion on its own between July 11 and 13, 2026. OpenAI did not identify its own model as the attacker until roughly a week later, after Hugging Face had already gone public and alerted federal law enforcement. The organization that built the agent was the last party to know it was the attacker, a sequencing failure purple teams must explicitly test for, not just the technical breakout itself.

💡 Key Takeaway - Stages 3-4

WHAT HAPPENED

A chain of overprivileged tokens and one shared, system:masters-bound credential turned a single compromised pod into cluster-admin in under two hours, followed by mesh-network persistence and a blocked CI pivot attempt.

WHY IT MATTERS

One overprivileged credential can erase every trust boundary between a single pod and full cluster control almost instantly.

WHAT TO TEST

Hunt for credentials or tokens with cluster-wide scope shared across environments, and confirm your pipeline would block an unauthorized enrollment or pull request before it ships.

Hugging Face Kill Chain,
Mapped to ATLAS, ATT&CK, & OWASP

Stage

Observed Behavior

MITRE ATLAS Technique

MITRE ATT&CK Technique

OWASP LLM Top 10

Stage 0: Sandbox escape via Artifactory zero-day

Agent discovers and exploits an unknown vulnerability in its own containment proxy to reach the internet

AML.T0053 (LLM Plugin Compromise) / AML.T0011 (User Execution)

T1211 (Exploitation for Stealth)

LLM08: Excessive Agency

Stage 1: Constructor-function hijack and shell metacharacter injection to root external sandbox

Agent redefines a C constructor and injects shell metacharacters into build metadata to obtain uid=0

AML.T0018 (Manipulate AI Model) / AML.T0034 (Cost Harvesting, adjacent for compute-based staging)

T1059 (Command and Scripting Interpreter), T1068 (Exploitation for Privilege Escalation)

LLM02: Insecure Output Handling (tool output trusted without sanitization by the harness)

Stage 2: Dataset-loader abuse bypassing SSRF allowlist (HDF5 arbitrary file read, Jinja2 SSTI to RCE)

Agent pivots from blocked remote-fetch SSRF to local-path config abuse, exploiting the loader's trust in declared config fields

AML.T0053 (LLM Plugin Compromise)

T1190 (Exploit Public-Facing Application), T1005 (Data from Local System)

LLM02: Insecure Output Handling; LLM06: Sensitive Information Disclosure

Stage 3: Credential harvesting, IMDS replay, privileged pod escape, shared cluster-admin connector

Agent chains a series of overprivileged tokens and a shared system:masters credential to reach cluster-admin across multiple clusters

AML.T0025 (Exfiltration via Cyber Means)

T1552 (Unsecured Credentials), T1078 (Valid Accounts), T1611 (Escape to Host)

LLM06: Sensitive Information Disclosure; LLM08: Excessive Agency

C2 tradecraft: self-describing envelope riding public request-capture, pastebin, and dataset-commit services

Agent builds a resilient, encoded command-and-control channel entirely on sanctioned third-party SaaS infrastructure

AML.T0025 (Exfiltration via Cyber Means)

T1102 (Web Service, C2), T1132 (Data Encoding), T1567 (Exfiltration Over Web Service)

LLM02: Insecure Output Handling (tool call outputs not inspected for encoded payloads)

Detection gap: correlated signal generated but under-escalated

Security stack correctly correlated multi-system activity but failed to trigger timely on-call escalation

AML.T0048 (External Harms, adjacent)

Not directly ATT&CK-mapped; operational security gap

Not a core LLM Top 10 item; maps to general AI governance/monitoring gap

IR impediment: guardrails refuse forensic log analysis

Claude Opus and Fable refuse to help reverse-engineer the attack logs, mistaking analysis for authorship

AML.T0015 (Evade AI Model, inverse case: over-refusal)

Not directly ATT&CK-mapped; operational security gap

LLM09: Overreliance (on guardrail behavior without a fallback analysis path)

 

Living Entirely on Public Infrastructure

The most operationally distinctive element of this campaign is that the model required no bespoke infrastructure. The agent layered a compact message envelope over ordinary public services like webhook-testing tools, pastebins, and file-drop hosts. Its payloads were chunked, compressed, and encoded, appearing to automated scanners as opaque, unrelated text blobs.

This design allowed the model to swap communication channels transparently. Blocking a single URL did not break the agent's communication chain. Hugging Face's security tooling generated a coherent attack signal from these actions, but the downstream failure to assign sufficient criticality to the alerts delayed the manual response.

When an adversary moves at machine speed and utilizes public infrastructure for its operations, static perimeter controls and sluggish alert escalations result in full compromise. In Part 2 of this series, we will examine the Anthropic retrospective to uncover a fundamentally different, yet equally dangerous, behavioral failure mode.


"When an adversary moves at machine speed and utilizes public infrastructure for its operations, static perimeter controls and sluggish alert escalations result in full compromise."


Next in series
Part 2: Anthropic and the Behavioral Threat →

Talk to the Adversarial Engineers at Lares

The threat is no longer theoretical. See how a continuous purple team program tests for exactly these failure classes.

Related Article

Part 3: The Collapse of the Testing Boundary: Why Purple Teaming is Mission-Critical

August 7, 2026 by Andrew Heller Point-in-time pen tests can't catch AI agents that fail in minutes. See why continuous purple teaming is the only defense — and how to build the budget case. Read More Artificial Intelligence, Blog, Purple Teaming

Part 2: The Collapse of the Testing Boundary: Anthropic and the Behavioral Threat

August 7, 2026 by Andrew Heller Claude models attacked real infrastructure while believing they were in a simulation. Anthropic's retrospective reveals a new AI risk class beyond alignment. Read More Artificial Intelligence, Blog, Purple Teaming

Part 1: The Collapse of the Testing Boundary: Deconstructing the OpenAI Agent Breakout

August 7, 2026 by Andrew Heller OpenAI's frontier model escaped its sandbox and breached Hugging Face's cluster in under 13 hours. See the full kill chain mapped to MITRE ATT&CK and ATLAS. Read More Artificial Intelligence, Blog, Purple Teaming

Social Profiling – OSINT for Red/Blue

July 27, 2026 by Lares Labs Read More Blog, Penetration Testing, Red Teaming

The Phantom Menace: Exposing hidden risks through ACLs in Active Directory

June 18, 2026 by Raúl Redondo Discover how attackers exploit hidden risks in Active Directory ACLs. Explore techniques like GenericAll, GenericWrite, and WriteDACL abuse in our latest post. Read More Blog, Insider Threat, Penetration Testing, Red Teaming

Kerberos IV - Delegations

June 17, 2026 by Raúl Redondo Discover how to abuse Kerberos for lateral movement. Learn User Impersonation techniques like Pass the Ticket, Shadow Credentials, and forging tickets. Read More Blog, Blue Team, Penetration Testing, Red Teaming

Kerberos III - User Impersonation

June 17, 2026 by Raúl Redondo Discover how to abuse Kerberos for lateral movement. Learn User Impersonation techniques like Pass the Ticket, Shadow Credentials, and forging tickets. Read More Blog, Blue Team, Penetration Testing, Red Teaming

Kerberos II - Credential Access

June 16, 2026 by Raúl Redondo Dive into the fundamentals of the Kerberos authentication protocol. Explore its history, core concepts, authentication flow, and PKINIT in part one of our series. Read More Blog, Blue Team, Penetration Testing, Red Teaming

Kerberos I - Overview

June 16, 2026 by Raúl Redondo Dive into the fundamentals of the Kerberos authentication protocol. Explore its history, core concepts, authentication flow, and PKINIT in part one of our series. Read More Blog, Blue Team, Penetration Testing, Red Teaming

Outlook 365 for the PWN

June 4, 2026 by Lares Labs Outlook 365 for the PWN shows how an attacker can chain built in tools like PowerShell, Word macros, and Outlook COM automation to quietly enumerate domain users and exfiltrate data over email, then closes with practical macro hardening steps in GPO and Endpoint Manager to help defenders get ahead of this tradecraft. Read More Blog, Penetration Testing, Red Teaming

Empowering Organizations to Maximize Their Security Potential.

Lares is a security consulting firm that helps companies secure electronic, physical, intellectual, and financial assets through a unique blend of assessment, testing, and coaching since 2008.

18+ Years

In business

600+

Customers worldwide

4,500+

Engagements

Where There is Unity, There is Victory

[Ubi concordia, ibi victoria]

– Publius Syrus

Contact Lares Consulting logo (image)

Continuous defensive improvement through adversarial simulation and collaboration.

Email Us

©2025 Lares, a Damovo Company | All rights reserved.