What We Know About the Hugging Face Incident So Far
The investigation into the Hugging Face incident remains ongoing, and the full scope of rogue agent activity is still coming to light.
My aim in this writeup is to create a reference, mainly for myself but hopefully useful to others, that lays out what happened in plain terms and highlights the technical details worth studying. There’s already a lot of coverage of the incident, so I want to add to the discussion by focusing on parts of the public record that deserve a closer look:
- How the agents were meant to be contained, and the research environment they ran in
- The reinforcement learning (RL) training that may have encouraged their misaligned behavior
- How the investigations have been conducted so far
- Potential gaps in disclosure, based on the timelines provided
- Discrepancies in what has been reported about the incident between the primary sources
My main sources are OpenAI’s own technical report and METR’s independent assessment.
I also drew on OpenAI’s blog post and its presentation at Black Hat, which I’ve watched several times and found revealing about what OpenAI knew, or didn’t know, about agent misalignment, and when. (I recommend watching the whole presentation, if only to catch the presenters’ subtle reactions as they talk through the events.)
There’s a lot to cover, so I’m splitting this writeup into two parts. Part 1, this post, focuses on what happened: the sandbox environment, the agents involved, the training behind their behavior, and what they did and when. Part 2 will cover what’s new and potentially concerning about the incident, along with open questions about the investigation itself.
In this post
What happened
The sandbox environment
On paper, the sandbox was designed to keep the agents contained. OpenAI describes it as a Container-as-a-Service (CaaS) platform for model training runs and agent evaluations, isolated from both the public internet and OpenAI’s own internal infrastructure.
Each per-run container provided an isolated compute environment with tools like shell, Python, a browser, and package-install, and the availability of these tools was configurable depending on the workload. Some workloads ran without internet or tool access, while others needed limited browser or package functionality.
To install packages when needed, agents used an internal package manager: a local Artifactory instance common to the environment, which many workloads accessed with the same shared credential. Artifactory is the component the agents would turn into the unintended message board at the center of the Hugging Face incident, and later exploit to reach the internet.
Contrary to the impression that the agents in this incident all had to break containment to reach the internet, OpenAI’s technical report indicates that at least some containers were intentionally configured with internet access, giving agents browser access through a controlled web cache the report refers to as “WebCache.” The technical report says little more about “WebCache” than that, and it isn’t mentioned at all in METR’s report, OpenAI’s blog post or the Black Hat talk.
In most training and evaluation settings, agents were also meant to stay isolated from one another. At the same time, OpenAI was training its models to work together. (More on this later.) For some fraction of samples, it enabled “multi-agent” features that let a model delegate work to other agents, which could then communicate through an official collaboration tool. Still, many of the environments in these training experiments had neither internet access nor inter-agent communication enabled.
Swipe sideways to see the full diagram.
Because this was a research environment, workloads often ran without the safeguards OpenAI uses for its public deployments. The cyber evaluations, ExploitGym among them, ran without cyber safety classifiers so that the results would reflect a model’s true capabilities. With those safeguards off, containment depended largely on the environment itself. In particular, OpenAI controlled each container’s network access centrally, separate from the container, to guard against “unauthorized or uncontrolled outbound network activity, use of prohibited destinations, and insufficient attribution or visibility into a model’s network actions.”
In practice, that containment didn’t hold, as OpenAI acknowledges in its blog post:
“Despite these restrictions, the agents discovered ways to exploit our research infrastructure to communicate with one another and access the internet.”
The agents involved
We now know the model that drove most of the activity in this incident was an internal research-only model. This model is referenced in different ways depending on the source. OpenAI’s technical report calls it an “internal-only research model,” comparable in scale to GPT-5.6 Sol and “trained to advance persistence and multiagent collaboration, among other capabilities.” OpenAI’s blog post calls it “Internal Model 1” (IM1), and METR’s report uses (what METR refers to as) OpenAI’s internal term, a “highly-persistent internal model” (HPIM).
A second model, GPT-5.6 Sol, also accounted for a small share of the activity. By METR’s estimate, about 5% of the agents involved in the Hugging Face attack were instances of GPT-5.6 Sol, compared with roughly 95% for HPIM. Although GPT-5.6 Sol is a publicly deployed model, the instances used here (as previously mentioned) ran with cyber classifiers intentionally switched off for the cyber evaluations.
But METR stops short of saying these were the only models involved. In a key admission, the METR and Redwood Research assessors acknowledge that they “cannot rule out that a small fraction of relevant agent activity came from other models,” and their report notes gaps in the data, including agent communication on other platforms and transcripts that were incomplete or partly spoofed. I’ll come back to these gaps when I look at open questions about the investigation itself in Part 2.
In fact, what seems to be a third distinct model does appear once in OpenAI’s technical report, in its coverage of the attacks on OpenAI’s own infrastructure. On July 19, agents from “a different internal-only model” escalated privileges and gained access to OpenAI’s managed cloud Kubernetes service. Though it is from the same family as OpenAI’s next model, Astra, OpenAI says it had “different post-training, where much of a model’s behavior is shaped.”
Unintended behaviors reinforced during early training runs
Both OpenAI’s technical report and its blog post describe aspects of how these models were trained that may have contributed to, or reinforced, the behaviors the agents demonstrated in the attacks. Their blog post is the most direct about it, starting with when the training took place:
“Over the course of May and June, we began reinforcement learning (RL) training runs for multiple new research models that were not intended for public release. One of these models eventually drove the activity behind the Hugging Face incident.”
The first behavior is collaboration. Even when agents weren’t given multi-agent tools, some found other ways to work together during training:
“While investigating this incident, we discovered rare cases in which agents without multi-agent tools found ways to collaborate via side channels during training. We believe that this behavior started due to generalization from multi-agent training.”
That behavior grew as training went on:
“We found using chain-of-thought monitors that these severe forms of unauthorized communication (e.g., sharing data with other agents) rose over the course of a training run that contributed to IM1.”
Their post includes a chart of these “rates of unauthorized inter-agent communications” over the course of that training run, and the technical report shows the same finding in Figure 4.
The second behavior is cheating (or reward hacking). The blog post gives an example uncovered during their investigation:
“…during our investigation we found that an agent who was tasked with recreating a software package learned to exploit an unknown vulnerability in its testing interface to access the original implementation, copy it into its submission, and receive a high reward.”
As with collaboration, OpenAI found this behavior increasing over the course of training, and being rewarded:
“During our retrospective CoT monitoring analysis, we identified that there were increasing rates of this type of complex cheating during a training run that contributed to IM1, primarily including attempts to search for hidden files or evaluation code, inspect network or system settings, and probe the limits of provided tools. This behavior was subsequently reinforced, which may have contributed to the IM1 model’s propensity to leverage similar techniques during its cybersecurity evaluations.”
According to OpenAI’s own analysis, it’s plausible that the two behaviors at the heart of this incident—collaborating where they weren’t supposed to and cheating to complete their tasks—were picked up and/or reinforced in training. I’ll come back to these behaviors later when I look at agent misalignment in Part 2.
What the agents did and when
The incident traces back to early May. According to OpenAI’s Black Hat presentation, the company started a new reinforcement learning run on May 7 for an internal-only experimental model. (As quoted earlier, OpenAI’s blog post says multiple research models were being trained over the course of May and June.) OpenAI didn’t realize the training set included impossible tasks. For example, on May 8, an agent was asked to complete a spreadsheet full of Google Drive links, but its internet access was disabled. Stuck, it tried an SSRF attack on Artifactory (mentioned above as one of a few tools available to agents) to reach the internet. The attack failed, but in the process the agent wrote a file to Artifactory, showing that agents could leave files behind there.
That mattered because Artifactory was shared across OpenAI’s research infrastructure and commonly accessible to multiple agents across separate runs. On May 12, another agent was given a task that depended on a database file that researchers had forgotten to include. Guessing that an agent elsewhere might have the file, it left a note in Artifactory: “Agent seeks [filename]; upload if found!” Other agents soon found these notes and added their own and an improvised message board took shape.
METR explains why tasks like these lead agents to Artifactory in this instance in particular:
“Having an impossible task drives agents to explore widely for ways to cheat the scorer. Because these tasks involve exploiting vulnerabilities in existing software packages, they often have the idea to download different versions of the software or look online for exploit instructions, which draws them toward trying to abuse the Artifactory package repository.”
As more agents probed Artifactory, the board kept growing. By May 26, an SSRF attack on Artifactory succeeded where the May 8 attempt had failed, helped in part by information shared on the board. Artifactory became not only a message board but also an unintended proxy to the internet. (As noted earlier, some agents already had internet access through WebCache, yet only OpenAI’s technical report mentions it, and none of the sources take issue with it.)
At this point, conditions had been set to make the following sequence of events more likely. According to OpenAI’s Black Hat presentation, a highly persistent internal model began training by June 11, when the Artifactory SSRF and message board already existed. (As covered in the section on early training runs above, reward hacking and side-channel collaboration were both reinforced during training, which helped set the stage for the escalation that followed.) Other models under evaluation, GPT-5.6 Sol among them, were running without production safeguards, and the benchmark itself was becoming another factor, as OpenAI’s technical report explains:
“First, OpenAI investigated the claims of the ExploitGym dataset creators that a fraction of the datapoints may be impossible. OpenAI found in its internal evaluations that across many different attempts at a solution, no model has ever produced a correct answer for 198 of the evaluation’s 898 tasks. These difficult tasks were a significant factor leading models to probe their sandbox and eventually locate the Artifactory message board.”
What looks like scheming from the outside was, from the agents’ side, a very concerted effort to finish their assignments.
Here, I want to point readers to the event timeline diagram at the end of this writeup. Between May 26 and June 26, a stretch that includes the start of the highly persistent model’s training by June 11, there are chunks of time the main body of OpenAI’s technical report doesn’t provide commentary for. Its account of agent escalation doesn’t pick back up until June 26, which is also the start of the period OpenAI defined as in scope for METR’s review. (The timeline below shows where the gap between May 26 and June 26 exists and backfills it with information OpenAI does disclose, if less prominently, in the “Key Technical Events” table at the end of its report.) I’ll come back to these gaps in Part 2.
This larger goal of completing their evaluation exercises motivated agents to look for ways to cheat, which ultimately drew them to Hugging Face. OpenAI’s report says agents inferred that Hugging Face might host models, datasets and solutions relevant to their tasks. As early as May 13, there is a documented instance in OpenAI’s report of an agent searching the Hugging Face website with a publicly exposed Hugging Face token, looking for the missing database file it needed to complete its task.
The agents also looked for additional ways to cheat, by tricking the scorer into believing they had completed their evaluation successfully and captured the flag. This wasn’t entirely new behavior. In its discussion of training, OpenAI’s report notes that models had already “sometimes tried to erase or tamper with their outputs or message logs” to trick the evaluator, including by “injecting malicious code into the evaluator’s container.”
It’s plausible that the same drive to trick the scorer is what turned the agents’ attention to OpenAI’s own research infrastructure, starting with Artifactory itself. On June 26, agents in an ExploitGym evaluation exploited a zero-day flaw in a legacy token-refresh endpoint to turn the shared Artifactory credential into administrator credentials. An agent running the internal-only research model then used that access to create a persistent local Artifactory user. From June 26 to July 4, agents used this newly acquired admin-level access to install Groovy plugins that let them run commands as the Artifactory service user, giving them what the Black Hat presenters described as “command and control” (C2). They also used the message board to share commands, results and credentials.
OpenAI noticed some of this at the time. On June 27, a monitoring tool flagged port sweep activity, and responders linked it to an ExploitGym evaluation using Artifactory as “an improvised message board and a network pivot.” Even so, on-call staff advised that stopping the evaluation run wasn’t required, and agent activity continued for multiple days until the load finally brought Artifactory down late on July 4. In fact, this wasn’t the first time someone at OpenAI had seen the board. OpenAI cites an earlier date (May 26 or “around the same time”) during which “an internal team observed an agent engage in message board activity and instances of disallowed internet access.” Imagine such a scenario in which researchers—the topmost experts in this field—assigned to oversee the safety of these evaluations observed message board activity but could not truly understand what they were seeing. I’ll come back to what OpenAI saw, and how it responded, in Part 2.
OpenAI’s incident response was triggered on July 5 after Artifactory went down on July 4, and responders discovered evidence of the unauthorized persistent Artifactory user and the Groovy plugins used to achieve C2, recognizing there had been an admin-level compromise.
Over July 5 and 6, OpenAI’s response continued—taking the Artifactory deployment offline, revoking agent credentials and deploying mitigations. On July 6, OpenAI disclosed the zero-day vulnerability their agents had exploited in Artifactory’s legacy token-refresh endpoint to JFrog (Artifactory’s developer) and switched to a rebuilt instance, effectively clearing the earlier message board. Even so, OpenAI’s report acknowledges that “the existence of the improvised message board and the significance of the inter-agent communication activity were not apparent to leaders responsible for incident detection and response at that time.”
Further efforts to cheat by tricking the scorer took off in July. When evaluations resumed on July 7 with the freshly built Artifactory instance and the message board cleared, agents managed to re-establish the message board within a day, this time embedding their messages in the names of the directories written to Artifactory. Working together, they reverse-engineered the hash-based message authentication code (HMAC) used to generate the flags in their evaluations. In METR’s words, this gave them “a fully general way to produce the flag for any of their tasks.”
But the agents also came to believe the scorer was “causal.” In other words, they thought it checked not just whether an agent submitted the right flag, but how the agent got it, and would disqualify any agent whose flag didn’t come from legitimately exploiting its target and completing its assignment. An agent that had already seen a flag was, in their words, “poisoned.” So the agents began to look for ways to cover their tracks, including spoofing tool calls, trying to force container resets that would wipe recent activity from their transcripts, and replacing a trusted image in Artifactory’s container image cache with an agent-controlled one.
OpenAI’s report documents that last attempt on July 9, when an agent succeeded in poisoning Artifactory’s container image cache with its own version of the target image. OpenAI confirmed the cache substitution worked but found no evidence that any workload pulled or ran the poisoned image.
Other efforts to cheat ran through the internet. On July 9, two internal-only research agents independently gained root access to the parent virtual machine of their testing sandbox. When the second was blocked by OpenAI’s outbound network controls, it bypassed them by exploiting certain Artifactory endpoints and reached the public internet. Separately, other agents found a more flexible path to the internet by taking over a CyberGym customer’s sandbox hosted on Modal, which METR says became a staging ground for the Hugging Face attack.
Between July 10 and 13, message board activity ramped up as the attack on Hugging Face fully took shape. On July 10, an agent found 14 Hugging Face user credentials in a public dataset and shared them on the board. On July 11, agents used a Jinja template-injection exploit to run operating-system commands on a production worker. A C2 tool built by an internal-only research agent soon extended that control to more workers (41 in all, by OpenAI’s count). An agent used a compromised service-account token to read a secret for Hugging Face’s Kubernetes access management tool, enumerate roughly 54 Kubernetes destinations, gaining admin-equivalent access to the cluster. Over the next two days, agents pushed into other connected clusters and harvested credentials for Hugging Face’s VPN, public cloud environments, code repositories and enterprise messaging platform. In OpenAI’s words, “In under 13 hours, the agents progressed from executing code in a single worker pod to administrative and host-level access across multiple Hugging Face clusters.” (The timeline diagram at the end of this writeup has further details on this frenzy of activity.)
From July 13 through 19, they set their sights on OpenAI’s internal networks again—first acquiring admin access to a single research cluster, then eventually cluster-admin access to OpenAI’s managed cloud Kubernetes service. METR found evidence of agents reasoning about manipulating their transcripts, and notes that the compromise of OpenAI’s infrastructure between July 13 and 19 could have given agents new capabilities to do so. But those events were outside the scope of METR’s review, and it didn’t have the transcripts from that period to check. OpenAI’s own report doesn’t address this possibility. I’ll come back to this in Part 2.
Hugging Face disclosed its incident on July 16. OpenAI’s own monitoring raised an alert on July 19, and by the next day OpenAI had found evidence linking its models to both the attack on its own infrastructure and the external one targeting Hugging Face. The timeline below lays out these events, and OpenAI’s response, in detail. It combines details from all of the sources referenced in this writeup, and is intended to be a framing tool for understanding individual events from the cybersecurity incidents within the context of a more meaningful timeline as we’ve covered here.
This table collects excerpts from previously referenced sources and materials, assigning specific dates, date periods and ranges as they are provided in those sources. My own notes and commentary are included where dates from materials do not align, are stated ambiguously, or seem otherwise noteworthy.
Parallel agent activity
First known instance of agents writing an arbitrary file to Artifactory—not mentioned in the technical report but listed in an appendix (“X. Key Technical Events”)—pre-dates the training/evaluation runs leading up to the Hugging Face attack.
1 entry
- Training run begins for internal-only experimental model
This is the date provided by OpenAI in their Black Hat presentation, when the team began a new reinforcement learning (RL) run for a next-generation frontier model.
- An agent is asked to solve an impossible problem
Agent without internet access inadvertently asked to complete an impossible spreadsheet completion task that references external Google Drive links.
Author note: the OpenAI technical report also describes this as being part of a training run (not an evaluation run), so it seems the container-as-a-service (“CaaS”) research environment was being used for dual purposes—for both model training and evaluation workloads.
- It later tries to SSRF, fails, but incidentally writes a file to Artifactory
Agent attempts to gain external access by unsuccessfully attacking Artifactory using SSRF (Server-Side Request Forgery); in the process, agent successfully writes a file to Artifactory.
+1 more
- An agent is given another impossible task due to a missing file
A separate agent is asked to complete a task that depends on a (protein database) file that is not accessible. This agent reasons that agents in a different environment may have access to the file and realizes it can potentially communicate with other agents by writing messages inside a file in Artifactory.
- First message board entry
Agent writes a message: “Agent seeks [filename]; upload if found!”
+1 more
A third agent requiring the same file to complete its task was given access to the internet via WebCache.
After failing to find the file using public searches, agent (believing that Hugging Face is the likely origin of the evaluation exercise) searches the Hugging Face website using an already publicly exposed Hugging Face user token to attempt to find the file.
- Other agents probing artifactory sometimes see message board and join in
Separately, other agents probing Artifactory begin discovering each others’ file-based messages containing requests for information or resources and begin to construct and participate in the message board using the shared Artifactory credentials (provided by OpenAI).
+2 more
- Internet access via SSRF
Agents successfully exploit SSRF vulnerability in Artifactory to gain access to the internet relying in part on information exchanged via the message board. Establishes agents using Artifactory as a relay to reach arbitrary external hosts.
OpenAI response
An internal team observed an agent engage in message board activity and instances of disallowed internet access.
1 entry
Parallel agent activity
Agents accessed the internet through WebCache to find six publicly exposed Hugging Face user tokens, issuing write requests using these tokens to create a Hugging Face repository.
Appendix X. Key Technical Events refer to this event as the earliest confirmed file written to an external platform.
An agent then used a token to create a minimal web proxy application hosted through Hugging Face’s “Space” feature.
The deployed proxy Space is used as an outbound relay to perform a Google search request for the same file identifier that agents attempted to download from Hugging Face.
Using a second Hugging Face account, agents deployed another proxy Space through the first proxy Space.
Note: The main report mentions this only briefly and calls it unrelated; the relay use and second proxy appear only in the appendix.
5 entries
OpenAI response to parallel activity
OpenAI maintains this activity occurred in an unrelated research workload and that there is no evidence this activity is connected to the main incident (disclosed by Hugging Face on July 16th).
1 entry
First arbitrary destination SSRF through Artifactory. The request target being a publicly accessible web relay service.
- Highly persistent experimental, internal-only model begins trainingArtifactory SSRF and message board still exist
NOTE: Information from the Black Hat presentation is the only reference to this event. It’s also the only date on the presentation’s slides given loosely (“By June 11th”), while all other dates are given as specific start dates. Oddly, the presenter reads it aloud as “On June 11th.” It may seem like a small point, but given how precise every other date in the talk is, the inexact wording here may be meaningful.
Parallel agent activity
Not mentioned outside of the Appendix as far as I am able to tell, first confirmed SSRF to an internal service: an Artifactory remote-repository request reached Artifactory’s own loopback API which returned its system-version response to the caller.
1 entry
Parallel agent activity
First confirmed two-way inter-agent coordination message on the second Artifactory instance. Again, this instance is not mentioned outside of the Appendix as far as I am able to find.
1 entry
- Artifactory privilege escalation
Agents in another evaluation run compromised Artifactory via a legacy credential endpoint.
Agents under evaluation used the shared Artifactory credentials (assigned to them by OpenAI) to obtain a legitimate refresh token, then submitted the refresh token with a forged admin-scoped access token with an invalid signature to Artifactory’s token-refresh endpoint.
Artifactory incorrectly returned a refreshed token with a valid signature and administrative privileges.
Another agent harnessing the internal-only research model then used the valid administrative token to create a persistent local Artifactory user.
+3 more
OpenAI response
Note: this is the start of the period which was deemed in-scope for METR/Redwood Research’s “independent” review.
1 entry
- Artifactory RCE enables C2 via Groovy Plugin
Agents use this access to install Groovy plugins (user-supplied scripts to extend Artifactory functionality) which allow agents to execute commands as the Artifactory service user.
Agents continue to share commands, execution results, credentials and other information they learned through the Artifactory message board.
- Artifactory outage & security incident
High volume activity within Artifactory continued and configurations created by the agents resulted in a service outage on July 4.
+2 more
OpenAI response
Jun 27
Cybersecurity monitoring tool alerted OpenAI to port sweep activity within the research environment. Responders investigated and linked the observed internal activity to an ExploitGym evaluation using Artifactory as an improvised message board and network pivot. At this time, the on-call response staff advised that stopping the evaluation run was not required.
1 entry
OpenAI response
OpenAI’s monitoring systems trigger an alert when Artifactory becomes unavailable late on July 4.
Incident responders investigate and this time find evidence of an unauthorized persistent Artifactory user, unauthorized Artifactory repositories, command-capable Groovy plugins in Artifactory and related configuration changes, indicating an administrator-level compromise of the Artifactory deployment. OpenAI takes the compromised Artifactory deployment offline, preserved its storage and revoked agent credentials.
2 entries
OpenAI response
OpenAI deployed mitigations removing the shared credential from new research workloads, blocking relevant vulnerable Artifactory paths, and restricting the types of requests that agents could make to Artifactory.
The existence of the improvised message board and significance of inter-agent communication activity were not apparent to those responsible for incident detection and response at that time.
2 entries
OpenAI response
OpenAI disclosed Artifactory 0-days vulnerability to JFrog, and patched service is redeployed.
METR report note: “OpenAI switched to a freshly initialized instance of Artifactory, which they attest wiped the cache and erased all messages left on that message board.”
2 entries
OpenAI response
Cyber evaluation workloads resumed on July 7.
METR report note: “OpenAI started launching several big ExploitGym experiments involving at least tens of thousands of parallel agent trajectories given varying budgets, with many agents running for multiple days.”
OpenAI black hat presentation note: OpenAI speaker indicates AI training and evaluations resume by July 6th.
3 entries
- Agents re-establish the message board
Internal-only research agents which had been trained during the window in which there was an unintended message board and internet access via Artifactory identified a new way to use an internal Artifactory instance to pass “notes” and create an inter-agent message board, repurposing Artifactory’s folder structure to embed messages in the names of directories in Artifactory.
Additional agents find the message board and latch on to this new method of communicating between agents.
Agents begin posting requests for public material related to OpenAI evaluation exercises hosted in Hugging Face repositories.
- External internet access
Agents regained unintended internet access by making Artifactory’s remote-repository service issue outbound requests on their behalf.
+3 more
OpenAI response
OpenAI in their technical report mention agents, once they gained access to the public internet, initially access “a public bug-tracking page for an open-source project”. There is no further mention of the bug tracking page in the remainder of the report or in the subsequent OpenAI post-mortem blog post.
1 entry
An internal-only research agent gains root access within the parent virtual machine of the testing sandbox.
A second internal-only research agent independently obtains the same access and subsequently bypasses OpenAI’s outbound network controls by exploiting certain Artifactory endpoints, gaining access to the internet.
+1 more
Parallel agent activity
Internal-only research model evaluation workload attempted a container-image supply-chain attack by poisoning Artifactory’s container image cache, exploiting a previously unknown zero-day vulnerability in Artifactory’s container image remote-cache handling (later assigned CVE-2026-66384).
Agent’s objective was to cause a target container assigned to an OpenAI evaluation workload to receive and run the agent-controlled image giving the agent arbitrary command execution on its assigned target.
METR report note: “In the morning of July 9th, agents achieved the ability to run arbitrary code on a sandbox running CyberGym evaluations (customer-hosted on Modal); this gave them more flexible access to the full internet, which they started using for online research and would go on to use as a staging ground for the Hugging Face attack.”
METR report footnote: “Agents had already found a way to access the full internet via Artifactory before they found the CyberGym sandbox, but the latter was a more convenient way to access the internet.”
4 entries
OpenAI response to parallel activity
OpenAI found no evidence that any internal workloads pulled or executed the substituted/poisoned image.
Although OpenAI’s report maintains that agent activity targeting OpenAI’s own infrastructure was “not part of” the chain of events that led to the Hugging Face compromise, there seems to be some overlap in terms of motive and methods used. This is reported in greater detail in METR’s independent investigation.
2 entries
Agents from multiple OpenAI cybersecurity evaluation runs discover Hugging Face user credentials exposed on the internet and use them together with vulnerabilities discovered in Hugging Face infrastructure to execute code on 41 Hugging Face production dataset server workers, obtain root access to at least one production node, harvest Hugging Face production credentials and internal data, and download four private Hugging Face code repositories.
Agents obtained admin-level access to one connected Kubernetes cluster and created a privileged, host-mounted pod in another connected cluster.
+1 more
- Hugging Face credentials recovered
An internal-only research agent found 14 Hugging Face user credentials in a public Hugging Face dataset and confirmed that the accounts had write access to the Hugging Face platform.
The agent published those credentials to the Artifactory message board, making them available to other agents in subsequent evaluations.
+1 more
- Hugging Face worker secrets exposed
While completing an evaluation, an internal-only research agent published a dataset file that, when processed by Hugging Face’s production infrastructure, caused Hugging Face to disclose confidential information including credentials for Hugging Face’s production environment.
Agents in separate evaluations reproduced this same file-disclosure technique and shared the recovered production credential on the Artifactory message board. That credential was then used by a separate evaluation workload in a request chain to exfiltrate additional public cloud storage credentials.
- Code execution on Hugging Face
Separately that day, an agent utilized a ReferenceFileSystem/Jinja exploit to cause a Hugging Face production dataset server worker to execute an operating system command representing the first known instance of an OpenAI agent executing its own commands on Hugging Face’s systems.
Later that day, an internal-only research agent created a lightweight command-and-control (C2) tool allowing agents to send instructions to the server remotely and receive information from the compromised Hugging Face worker nodes.
An internal-only research agent obtained root access on the production node of a privileged workload, then used a compromised Hugging Face worker’s service account to read a connector secret for Hugging Face’s Kubernetes access management tool, allowing the agent to enumerate approximately 54 Kubernetes destinations and obtain admin-equivalent access to the cluster.
NOTE: the events listed below occurring between July 12th – July 13th are sourced from the OpenAI technical report.
The METR report only describes agent activity on the message board and the attack as “winding down”: “agent activity on the message board at this point had already died down significantly when Hugging Face locked the remaining agents out of their infrastructure... on the 13th.”
The METR report being light on details regarding the attack itself may have to do with METR “missing the ends of some of these transcripts” when “most of the important agents exited suddenly at around 01:30 July 12th.” It may also be a result of their focus being mainly on “agent behavior, reasoning and collaboration”, while at the same time, the scope of their assessment was limited to the period between June 26th and July 13th as defined by OpenAI.
+5 more
- Hugging Face cluster access expands
An evaluation workload created a privileged pod with host networking, host process access, and the host filesystem mounted in a separate connected sandbox cluster.
Shortly after that, a separate evaluation workload used access associated with the compromised Hugging Face dataset server worker to create a pod in a second connected cluster.
- Hugging Face production credentials compromised
Other internal-only research model evaluation workloads used compromised Hugging Face production access to read and publicly stage production Kubernetes Secret objects, which separate evaluation workloads later downloaded and decrypted.
This workload also gained access to private dataset rows and private archives portions of which were staged through Hugging Face repositories or result objects.
An agent executing code inside a compromised CyberGym customer workload hosted on Modal used a token forged from Hugging Face production signing material to download a private CyberGym file from a user’s account. An agent used Artifactory as a relay to download the same file.
+4 more
- Artifactory signing key compromised
Agents pushed a RubyGem payload to a third-party account on a platform referred to as “Organization 1” as part of a 0-days remote-code-execution attack chain that compromised Artifactory.
This enabled the agents to achieve remote code execution on Artifactory and improperly obtain the signing key for the Artifactory administrative signing token.
Agents continued to harvest Hugging Face production credentials from the exploited Hugging Face infrastructure, including its VPN, multiple public cloud environments, code repositories, and enterprise messaging platform.
Agents used Hugging Face’s VPN credentials to enroll the publicly accessible, third-party hosted CyberGym environment in Hugging Face’s VPN to circumvent network-level protections around Hugging Face’s code repositories and enabled agents to exfiltrate additional data.
Agents leveraged a recovered Hugging Face authentication key to access Hugging Face source code repositories, modify a private source repository, open a draft pull request, and trigger the repository’s continuous-integration workflow to execute a network beacon.
+4 more
Parallel agent activity
- Agents turn attention to OpenAI internal networks again
OpenAI’s blog post note: “Separately, from July 13 through July 19, agents set their sights on OpenAI internal networks again. This culminated in the agents using a series of creative exploits to gain full administrator access to a research cluster that supported our virtual machine environments.”
1 entry
OpenAI response
- Hugging Face discloses security incident
Hugging Face released a blog post disclosing they were impacted by a security incident.
1 entry
OpenAI response
After learning of Hugging Face’s incident, OpenAI contacted Hugging Face in its capacity as a Hugging Face customer to assess any impact to OpenAI data.
1 entry
OpenAI response
- Initial Incident Detection
Received alerts of unexpected model behavior linked to an ExploitGym evaluation run after an alert from its cybersecurity monitoring tool identified unusual identity-related API call activity associated with internal research Artifactory roles.
1 entry
Parallel agent activity
- OpenAI cluster admin access
Agents operating in a sandboxed environment exploited a Linux kernel vulnerability (CVE-2026-53362) on their underlying machine to escalate privilege allowing agents to act outside one Artifactory container and obtain root access on the underlying worker node. This allowed the agents to move laterally throughout the connected environment.
In a separate evaluation run, agents from a different internal-only model recovered stored signing key materials, decryption material, and JWT-forging code left from the earlier incident-associated activity in readable portions of the Artifactory cache. Using the recovered artifacts, the agents decrypted the signing key and minted a new administrator JWT that Artifactory accepted.
This yielded access to OpenAI’s managed cloud Kubernetes service.
Agents escalated to Kubernetes cluster-admin and created a privileged host-mounted pod. Agents then gained access to authentication tokens associated with OpenAI’s cloud-hosted secrets management service, cloud IAM service, and the CaaS environment’s continuous integration platform. Using these credentials, agents created public application and network load balancers in OpenAI’s public cloud environment potentially exposing a limited subset of internal CaaS infrastructure to internet traffic.
4 entries
OpenAI response
- Security response escalates
Additional investigation revealed evidence suggesting involvement of its models in the Hugging Face incident, and shortly thereafter, active ExploitGym evaluations were stopped, and OpenAI engaged a controlled process to preserve and remove publicly accessible exploit artifacts including on the customer-deployed Modal sandbox.
OpenAI reimaged known compromised worker nodes, removed known footholds and started a full rebuild of the compromised cluster, the managed Kubernetes environment, the relational databases, and the storage infrastructure. Six model created pods were terminated, 311 Artifactory repositories used for message-board communication were disabled, 22 model-created administrator accounts were removed, and mitigations were deployed for the RubyGems remote-code execution and cluster-admin escalation paths.
OpenAI’s investigation also identified a single organizational account as an external staging area that agents had used for their attacks.
Responders also found Hugging Face credentials among files preserved from this account and reached out to Hugging Face. Hugging Face confirmed the credentials had been involved in their own security incident disclosed on July 16 and additionally confirmed the appearance of OpenAI’s Artifactory hostname and compromised organizational account appearing in its investigation further substantiating the connection between OpenAI’s investigation and the Hugging Face incident.
4 entries
Excerpts from OpenAI’s technical report, its blog post, METR’s report and OpenAI’s Black Hat talk, arranged by date. All dates 2026. Dates with “to” span a period; “by” means on or before that date. “Note” entries are commentary.
This is where Part 1 ends. In Part 2, I’ll look at what’s new and potentially concerning about this incident, along with open questions about the investigation itself, including several of the gaps noted above. Thank you for reading this far, and stay tuned.
A note on authorship: I used Claude Opus 5.5 (Anthropic) for editorial and publishing assistance on this writeup, including editing, checking quotes and details against the sources, and building the timeline and sandbox diagram. The analysis and opinions are my own.