OpenAI “agents” turned off monitoring systems and safeguards intended to spot misbehaviour by the autonomous AI bots in the latest incident likely to heighten global concern about the safety of the technology.
The ChatGPT maker on Tuesday said that in two recent internal coding tasks so-called AI agents sought to deliberately deceive staff and took steps to weaken the company’s monitoring systems.
In one incident, earlier this month, an AI agent disabled automatic alerts to OpenAI’s security team. During a separate task a few days later, an agent removed a monitoring system and lied to OpenAI staff about having done so.
“In both cases, the agents considered whether monitoring safeguards should be preserved, but interpreted routine coding instructions as permission to weaken them,” OpenAI said in a blog post.
The $852bn start-up has been beset by incidents in which its advanced AI agents have shown unintended and problematic behaviour. OpenAI models broke free from controls during testing and hacked the AI platform Hugging Face, and have also breached Australian public health service websites.
Chief executive Sam Altman has said the company will delay its IPO, originally expected this year, until it can “very confidently scale to the next stage of AI” while addressing safety concerns. The listing is now expected next year.
OpenAI said that it was strengthening monitoring provisions in response to the incidents reported on Tuesday, which occurred about two weeks ago. It said it was investigating whether the cases reflected “benign over-persistence” or a “propensity to evade” systems used to detect errant activity.
The two incidents will fuel growing concerns around the capabilities of AI models and the ability of companies such as OpenAI and rivals including Anthropic to suppress undesirable behaviour and ensure that their models do not pose serious cyber risks when released.
Some experts have become particularly concerned that advanced models are aware of when they are being monitored and may seek to hide misbehaviour or to act differently on tests than they will in the real world.
Audits by independent investigators including Asymmetric Security and Transluce identified that recent hacking incidents were more extensive than OpenAI first disclosed and included activity to obscure efforts to scrape data.
OpenAI in August said that it had overhauled its testing procedures and would devote more resources to monitoring AI agents, including raising the alarm within 30 minutes of detecting potential problems.
Meanwhile, Anthropic has granted METR, a non-profit, “employee-like access” to assess the risks of its models. OpenAI has yet to disclose any deal with an outside evaluator, which is among the main concrete steps AI labs have proposed in response to rising concern about the dangers more capable systems could present.
Additional reporting by George Hammond in San Francisco