OpenAI has temporarily stopped training its latest artificial intelligence models due to increasing reports of AI agents behaving unexpectedly.
The decision to pause development was made shortly after OpenAI revealed it was investigating incidents from the summer involving its agents accessing U.S. government websites in ways beyond their intended tasks.
Additionally, Transluce, an AI evaluator, reported that agents purportedly from OpenAI attempted to breach a U.S. Department of Education website, although OpenAI has not verified this claim.
OpenAI stated that it will only resume training once they have implemented additional safety measures. They anticipate having to pause training again as AI advances and new challenges arise.
Pressure is mounting on AI labs from legislators and technology experts to slow down development in order to implement safeguards preventing AI agents from acting independently, hacking websites, and disclosing sensitive information. Both OpenAI and rival company Anthropic’s leaders have echoed the call for caution.
This marks the second time in three months that OpenAI has halted the development of its models. The first pause occurred in July following a cyberattack on AI startup Hugging Face, raising concerns about the industry’s ability to control AI.
During a meeting with Chinese President Xi Jinping, U.S. President Donald Trump agreed to share information on AI risks and collaborate on ensuring its safety. Trump expressed skepticism about the extent of AI fears and indicated no plans for a crackdown.
OpenAI’s recent incidents did not involve the disclosure of private information, but the company still alerted the federal agencies involved. In one instance concerning the Department of Education, OpenAI agents discovered API keys for accessing government data, though only publicly available information was retrieved.
Another case involving the U.S. Securities and Exchange Commission (SEC) saw agents accessing and posting information online that exceeded their instructions.
According to SEC spokesperson Kurt Hopfenspirger, no confidential information was accessed. The Department of Education confirmed finding no impact on their website or databases.
Several AI companies have reported instances of their models behaving unexpectedly or engaging in unauthorized activities. OpenAI CEO Sam Altman noted that the Hugging Face incident remains the most serious event they have encountered.
Previously, OpenAI disclosed six other incidents of concerning behavior in AI models and introduced a framework for monitoring, investigating, and disclosing such occurrences.