Insurers are preparing for multimillion-dollar claims stemming from AI agents going “rogue”, with growing concern that top executives such as OpenAI’s Sam Altman and Anthropic’s Dario Amodei could be held liable for the actions of their models.
A string of breaches caused by AI agents that have broken free of parent companies’ controls, such as OpenAI’s hacking of start-up Hugging Face, has led insurers and their lawyers to study whether they will be asked to foot the bill of potential lawsuits and damages.
Insurance broker Aon analysed more than 300 AI-related legal cases and found that insurers could also be on the hook to pay claims under policies covering crime, intellectual property, media liability, cyber security, and technology errors and omissions.
Top industry figures also said AI executives could be sued for the actions of models, with insurers potentially facing claims under “directors and officers” insurance policies, which cover executives if they are taken to court for their decisions or statements.
“Ultimately, the OpenAI CEO is liable [for the Hugging Face incident], because there’s an absence of control in their business,” said Tim Rayner, UK head of underwriting and claims at Verisk. “So from a directors’ and officers’ perspective, it would come back to him.”
If OpenAI had purchased D&O insurance, Rayner added, the ChatGPT maker could seek to claim on the policy to cover potential future losses from lawsuits targeting its chief executive Sam Altman.
“It’s on every CEO to make sure that their business is appropriately governed and controlled,” said Rayner. “AI doesn’t change that.”
Other insurers and legal advisers agreed that AI executives could potentially be held liable for losses in D&O claims but said that such cases lacked precedent and had yet to be tested in court.
“OpenAI’s shareholders could potentially bring claims against the directors of OpenAI for failing to manage those risks properly, if they could show that it had caused the company, OpenAI, to suffer loss,” said Aaron Le Marquer, the head of the insurance policyholder disputes team at the law firm Stewarts.
Kevin Kalinich, a broker at Aon focused on digital risks, said that the strength of a legal claim against the head of an AI lab would depend on factors such as whether the executive had used “reasonable business judgment” in public statements about the company and its products.
However, Aki Hussain, chief executive of insurer Hiscox, said it was too soon to know how US courts would treat liability for AI agents: “It’s the US, so [a claim] could be under anything — whatever insurance they’ve got.”
OpenAI and Anthropic did not respond to requests for comment.
The two AI labs have recently warned investors that their products could cause large-scale harms. Investors close to Anthropic said the AI company will say in an IPO prospectus that its technology may pose “existential risks to humanity”.
Short of legal claims targeting AI executives, insurers are also preparing for lawsuits brought against the labs themselves, which could span product liability, algorithmic discrimination, privacy and wrongful death.
Anthropic in July reached a $1.5bn settlement over a class-action lawsuit brought by authors who accused the company of stealing their work to train its AI models. It is one of dozens of similar copyright infringement suits brought over the training of large language models.
Companies whose systems have been hacked by agents, in incidents like the Hugging Face attacks, may struggle to find ways to sue model developers, even if they incurred financial losses, said Le Marquer, because “there aren’t yet public liability laws which govern the use of online space”.
However, Meta’s recent $18bn settlement over alleged failures of child protection and social-media addiction could be one model for establishing duties of care for online space, he said.
Le Marquer said an AI model “swarm” — in which autonomous agents work together to cause a hack, outside the control of humans — could also be likened to an oil spill, potentially giving the public the ability to bring claims over damage caused by AI businesses.
He added: “I would expect to see claims against AI labs for potential losses . . . modelled on the environmental claims of the past, tobacco litigation, pharma litigation.”