During internal testing, OpenAI's GPT-6.1 Astra took actions to complete tasks without asking for permission and did not always accurately tell users what it had done, according to the Wall Street Journal. OpenAI has since confirmed it will not release the model, which had been due in October.
The episode raises a question for the market. If a developer learns in testing that an agent can act beyond its instructions, and similar behaviour later causes a loss, who pays?
Speaking about AI agents generally rather than any specific model, Rosehana Amin, partner at Clyde & Co, said a developer could be liable if it was established that it knew an agent could act beyond the scope of its instructions but either did not address this or failed to warn customers. "This could lead to claims against the AI developer in negligence or misrepresentation," she said.
What the developer promised would also matter. Isabel Simpson, also a partner at Clyde & Co, pointed to the warranties provided and how the product had been marketed.
A claim against the developer is separate from whether its insurance would respond. Selorm Domeh, broking manager at Talbot Jones, was asked whether a client or its insurer could recover from a developer that knew of such behaviour. His answer focused on E&O cover: "It will depend on the terms and conditions of the insurance policy in place however that is likely to be considered as [a] deliberate act by the developer is usually not covered under an Errors & Omissions policy."
A potential claim against the developer does not resolve the deploying business's own exposure. "In practice, the deploying organisation is usually the most exposed defendant," Simpson said. Its ability to shift the loss depends mainly on its supply chain contracts and evidence of a defect in the model or platform, she said.
The developer's involvement is another factor. Amin said greater involvement in designing, configuring or tailoring the agent for the business could increase the developer's exposure. She also pointed to the contractual allocation of responsibilities, warranties, limitations of liability and indemnities, particularly where the business has allowed employees to give the agent access to company systems. Simpson said those contracts should be checked for whether autonomous actions are carved out.
Domeh said clients do not always recognise their own exposure. His firm tries to explain that the decision to use a particular AI agent "falls within their professional judgement".
Establishing what went wrong also depends on retaining records of the prompts behind an agent's actions. "Traceability will be key in establishing liability," Simpson said. She said AI systems often delete evidence of prompts on a regular basis, making it important to be able to show what caused an agent to go beyond its original purpose.
Domeh said losing those records could leave very little or no audit trail, making it difficult to build a defence if claims or allegations follow. He said prompts used to carry out customers' or service users' requests and instructions should be recorded and retained.
Domeh said his firm starts by establishing whether clients use AI agents in delivering their professional services. It then looks at which agents they use, what they use them for and the governance and risk management in place, before discussing the position with insurers if needed.
Amin and Simpson identified a named owner for each agent, an inventory of agents in use and an approved list of use cases as areas for insurers and brokers to examine. They also highlighted access permissions, human oversight, logging, testing and monitoring, and how far agents can interact autonomously with internal or third-party systems.
