Artificial intelligence can help adjusters process claims more efficiently, but it cannot assume responsibility for the decisions made with its assistance. That accountability must remain with a human, according to Melissa Hill (pictured), North American head of claims at Allianz.
“The human is always accountable. The AI can’t make the decision,” Hill told Insurance Business.
Claims handling in Canada and the US is heavily regulated, and adjusters remain the individuals accountable for decisions made on behalf of insurers, she said. AI can support their work and inform their assessments, but it does not change who is ultimately responsible if something goes wrong.
Read more: Soft market is squeezing claims firms on fees, ClaimsPro president says
“You can’t blame the AI. You have to blame the human, because the human still has to make the ultimate call and the ultimate decision,” Hill said.
Some in the industry are exploring whether AI could handle claims from beginning to end without human involvement. Hill considers that approach too risky, given the complexity of claims and the technology's limitations.
“You can’t do that. It’s too complex, and the AI hallucinates in the process,” she said.
Even an apparently successful automated settlement could leave a policyholder with the wrong payment, she added.
“Even if you pay them, we could still pay them the wrong amount,” Hill said.
The risks extend beyond inaccurate assessments. AI can also introduce unintended bias when systems make decisions based on information that appears relevant but fails to account for the circumstances behind it.
Hill pointed to a publicly reported case involving an employer that used AI to help make layoff decisions. The system considered factors such as employees' keystrokes and calendar density, but failed to account for how medical disability could affect those measures. As a result, employees on medical leave were disproportionately identified for potential layoffs, she said.
“That was positive intent,” Hill said. “But basically, they created a bias without trying to create the bias, and nobody looked at that. It is alleged they let it make the decision, and then had the names and off they went.”
The case is now the subject of a pending class action, she said. For insurers, the broader lesson is that human reviewers need to understand and challenge the information AI uses to reach its conclusions, “because you don’t know what it’s deriving,” Hill said.
Allianz already uses AI in several parts of its claims operation, including receiving electronic first notice of loss submissions, setting up claims, summarizing documents and medical records, and automating payments after claims have been approved, Hill said.
The technology takes on administrative work, while adjusters remain responsible for validating its output and reviewing the underlying documentation.
“A lot of administrative tasks get pushed into AI, so the adjuster can spend their time making decisions and moving the claim along,” she said.
However, AI can also be overly accommodating, producing answers that reflect what it thinks the user wants to hear rather than challenging the information presented to it.
“AI wants to please you,” Hill said. “It’s going to give you the answer that it thinks you want to hear, which is why we have to test it and test it and be very careful. We have to challenge it.”
That means adjusters must be able to explain why they accept or reject an AI-generated recommendation. Introducing that additional scrutiny can initially slow the process, but Hill believes it is necessary to avoid errors as adoption expands.
“I think we need to go slow to go fast,” she said.
Read more: Insurers want to pay claims faster, but their own paperwork keeps getting in the way, broker says
Another challenge is the technology's ability to distinguish important information from details that have little bearing on a claim. Hill said AI can place too much weight on figures or statements without understanding their significance in the broader context.
She gave the example of a settlement demand that substantially exceeds the actual value of a claim.
“The demand might be an outrageous number. They might ask for $10 million, but the claim is worth $10,000,” she said. “The AI puts relevance to the $10 million demand. There is no relevance.”
An experienced adjuster would recognize that a plaintiff's initial demand is often inflated and should not necessarily determine the severity of a claim, Hill said. AI, however, may flag the figure as a sign of significant exposure and recommend escalation without understanding the context.
The limitations are not purely technical. Claims can involve people dealing with some of the most difficult moments of their lives, making human interaction an important part of the process.
“Sometimes this is the worst day of people’s lives when a tragic event happens, and they want insurance. They don’t want a robot,” Hill said.
Beyond the challenges of using AI in claims operations, Hill said risk managers should also examine how their own insurance policies respond to AI-related exposures.
Many policies do not explicitly address AI, she said, although some contain broad endorsements excluding certain AI-related risks.
“The policies are silent today, but there are endorsements that exclude the use of AI, but it’s pretty generic,” Hill said.
Her advice to risk managers is to review their coverage with their brokers and check whether AI-related exclusions or endorsements have been attached.
“If your entire operation shuts down, and there’s any component of AI, you’re not going to be covered,” she said.
