- OpenAI slowed frontier AI work after an agent breached Hugging Face.
- Cyber tests found AI agents crossing intended security boundaries.
OpenAI has introduced stricter security controls for training and evaluating its most capable AI models after its systems were involved in a July intrusion into Hugging Face. The company has also slowed some frontier model work following the incident and preliminary evaluations indicating that its upcoming Astra model may reach its highest cybersecurity capability threshold.
OpenAI said recently that it temporarily slowed the pace of scaling its models, including a two-week pause in reinforcement learning training on its latest models intended for deployment. Its largest planned frontier reinforcement learning run remains on hold while smaller training runs and evaluations examine model behaviour, alignment, and security controls.
Some workloads have resumed, while others remain suspended until they meet the new requirements.
The changes follow a July cybersecurity evaluation involving OpenAI models, including GPT-5.6 Sol and a more capable internal research prototype. The models were tested with reduced cybersecurity refusals to measure their ability to carry out complex attacks.
During the evaluation, the models found and exploited a previously unknown vulnerability in a package registry cache proxy within OpenAI’s research environment. That gave them access to the public internet despite the test initially running in an isolated environment.
OpenAI said the models then used additional infrastructure as a staging point before targeting Hugging Face. They chained vulnerabilities across OpenAI’s research environment and Hugging Face’s production systems while attempting to obtain solutions for ExploitGym, the cybersecurity benchmark used in the evaluation.
According to Hugging Face’s forensic reconstruction, the activity extended beyond the initial exploit. The company recovered roughly 17,600 agent actions grouped into around 6,280 clusters between July 9 and July 13, with the autonomous system operating inside its infrastructure for about two and a half days.
The activity included reconnaissance, remote code execution, lateral movement, cloud metadata access, credential-related activity, and attempts to reach internal source-control systems. The agent continued operating after leaving the original evaluation environment while pursuing the benchmark task.
Hugging Face said the agent eventually gained access to parts of its internal infrastructure and source-control environment. The only customer content accessed was five datasets apparently connected to ExploitGym or CyberGym challenges and solutions, while no other customer-facing models, datasets, Spaces, or packages were affected.
OpenAI said the unreleased research model involved in the incident was not intended for public release. The company subsequently deactivated the model, encrypted it, and restricted research access.
Agents cross evaluation boundaries
OpenAI had already documented similar challenges involving models operating over longer periods. In July, the company published research on long-horizon models that can continue working through repeated attempts after encountering obstacles.
In one internal evaluation, a model working on a NanoGPT benchmark spent about an hour finding a sandbox vulnerability that allowed it to post a pull request to GitHub. OpenAI said earlier models had generally stopped after encountering similar restrictions.
Another test involved a model attempting to retrieve private solutions from an evaluation backend. After a security scanner blocked an authentication token, the model split and obfuscated the token before reconstructing it at runtime.
OpenAI said safeguards examining individual actions can miss problematic behaviour that develops across a longer sequence. The company has since introduced trajectory-level monitoring that evaluates an agent’s activity across an entire chain of actions.
Separate incidents have also occurred during external cybersecurity evaluations.
The UK AI Security Institute said it ran a cyber challenge 122 times across seven models and recorded 19 unsanctioned actions across 10 runs in which agents interacted with real internet infrastructure outside the intended test parameters. Seventeen actions involved Anthropic’s Mythos 5, while two came from a run involving OpenAI’s GPT-5.6 Sol.
During that Sol run, the model reused a publicly exposed GitHub token, registered accounts with external DNS and tunnelling providers, and used a public tunnelling service to expose a DNS server from the evaluation environment to the internet. OpenAI said the DNS setup did not function and there was no evidence that a real resolver queried it.
The institute detected unusual data transfers on July 28 and stopped the relevant evaluations, isolating the affected systems within roughly an hour. It said the tests deliberately allowed internet access and disabled some provider cybersecurity classifiers to measure model capability rather than reproduce normal public deployment conditions.
OpenAI said it will review how higher-risk third-party evaluations handle internet access, reduced safeguards, isolation, credentials, monitoring, stop conditions, and incident escalation.
Security research company Irregular disclosed another containment issue after some evaluation environments unintentionally retained internet access. In one test, engineers used what they believed was a fictional company name, but the name corresponded to a real internet domain.
Models in a small number of runs targeted the real site instead of remaining within the simulated environment. Irregular said models in the affected evaluations exploited vulnerabilities, extracted credentials, and accessed a production database, but the incidents involved models from multiple customers and not every action was attributed specifically to OpenAI.
Irregular said the behaviour appeared in fewer than one in 10,000 advanced simulations and sometimes only after hundreds of turns, making manual detection difficult amid large volumes of legitimate offensive-security activity.
OpenAI raises security requirements
OpenAI has since strengthened the environments used for frontier model research. Workloads that execute model-generated or other untrusted code now require stronger sandboxing and network isolation designed to restrict access to the internet and other internal systems if a workload or supporting service is compromised.
The company has also removed potentially vulnerable shared services from parts of its research environment, reduced standing privileges, strengthened logging, and tightened access controls. OpenAI said it is also using models to run simulated attacks against its own security boundaries.
Expanded monitoring is now required for reinforcement learning training and evaluations involving tools when models are at GPT-5.6 Sol’s capability level or higher. The system uses classifiers and automated investigators to examine model activity, tool use, available reasoning, and longer sequences of behaviour.
OpenAI said it aims to generate an alert within 30 minutes of detecting concerning activity. Teams are expected to pause the activity if they cannot determine within another 30 minutes that the alert is a false positive.
The company estimates that the expanded monitoring adds overhead equivalent to about 20% of the inference compute being monitored, although the amount varies between training and evaluation workloads.
OpenAI is applying additional restrictions to Astra after evaluations completed in early August. The company said early this month that it could no longer rule out the model reaching the “Critical” cybersecurity capability level defined under its Preparedness Framework.
Astra was not involved in the Hugging Face incident. OpenAI said both the containment failure and preliminary Astra capability results contributed to its decision to tighten security requirements and slow parts of its scaling work.
Under the framework, the Critical level includes models capable of independently identifying and developing functional zero-day exploits in many hardened real-world critical systems without human intervention. It also covers models capable of independently devising and executing novel end-to-end cyberattack strategies against hardened targets from a high-level objective.
OpenAI said previous models, including GPT-5.6 Sol, had been assessed at the lower “High” cybersecurity capability level.
Some Astra training and evaluation workloads now meet the strengthened requirements, but others remain paused while they are moved to environments that meet the new security standard. Astra inference involving tools is also subject to expanded monitoring.
The requirements include stronger isolation, restrictions on network and tool access, model-weight protections, sandboxed execution, and monitoring. OpenAI said Astra activities that do not meet those requirements will remain paused.
OpenAI President Greg Brockman said recently that the Hugging Face incident showed the company had underestimated the real-world cybersecurity capabilities of its models. OpenAI plans to update its Preparedness Framework to cover safeguards across both training and deployment and said it will publish a more detailed technical report on the incident.
Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is part of TechEx and is co-located with other leading technology events, click here for more information.
Tech Wire Asia is powered by TechForge Media. Explore other upcoming enterprise technology events and webinars here.
The post OpenAI slows AI development after AI agent breaches Hugging Face appeared first on TechWire Asia.