The fatal flaw of artificial intelligence (AI), according to a gathering of international healthcare professionals including underwriters and claims managers, is the technology's ability to get the same thing wrong potentially millions of times. The hybrid conference, attended virtually across the world, has clearly articulated some of AI’s serious and growing problems. The concerns have major implications for the insurance industry, businesses and human society at large.
"The key insight was that AI does not fail the way humans fail," said Eric Lowenstein (pictured), CEO of Tego, a Sydney-based specialist healthcare and medical indemnity underwriting agency offering policies in Australia and New Zealand. "A human gets it wrong once - AI could get the same thing wrong every time, across thousands of patients, often invisibly for months."
Lowenstein, who spoke at the event, said this "changes everything about how we underwrite the risk."
He said some of the findings from the hybrid conference and webinar hosted by global law firm Kennedys, AI in healthcare: Innovation, risk and responsibility, could shape how the industry underwrites healthcare in Australia. The implications likely extend to commercial insurance and businesses generally. The event was attended by healthcare professionals, insurers, indemnity providers, underwriters and claims managers.
That failure mode, Lowenstein argued, reframes the entire risk-control conversation. Safe AI, he said, is far less a technology problem than a people problem – the real difficulty sits not in the model but in the clinicians, workflows, training and change management around it. The underwriting question can no longer stop at what tool a client is running and how secure it is; it has to reach whether the people using it have been trained, and whether they know what to do when it gets something wrong.
His sharpest warning was about visibility. Even sophisticated health systems with strong governance, he said, struggle to keep a current inventory of every AI tool running across the organisation.
"The first underwriting question of 2026 is not 'are you using AI?', it is 'do you know what AI you are using, where is it deployed and what data it has access to?'" Lowenstein said. Most organisations, he added, cannot yet fully answer it.
Australian regulators have landed on the same fault line. In a letter to all regulated entities dated 30 April 2026, the Australian Prudential Regulation Authority (APRA) warned that governance, risk management, assurance and operational resilience practices are not keeping pace with the scale, speed and complexity of AI adoption, and cautioned boards against leaning on vendor presentations without sufficiently interrogating the risks. The Australian Securities and Investments Commission (ASIC) followed on 8 May 2026 with its own open letter urging licensees to strengthen their AI and cyber controls.
Lowenstein told the panel the gap between AI policy and AI practice is not a healthcare problem but a universal one, spanning banks, law firms, insurers and government. The real exposure, he said, sits in how AI is deployed, recorded and audited day to day.
That caution is showing up in policy wordings. From January 2026, the Insurance Services Office (ISO) in the United States issued new generative-AI exclusion endorsements for commercial general liability cover and carriers adopted them within weeks. Last year, Berkley Insurance filed an absolute AI exclusion across directors and officers (D&O), errors and omissions and fiduciary lines. Cyber wordings from a number of insurers have also tightened. The open question, Lowenstein said, is whether Australia is about to import the same general retreat.
His final warning could be the most systemic. The AI economy, he noted, rests on a handful of foundation models from a small number of providers, a monoculture in which a single weak point is shared by everyone.
"A single defect in one of those models could trigger correlated claims across every industry, across countries, across lines, at the same time," Lowenstein said.
It is a concern reinsurers are now naming directly. Gallagher Re, in its report Smart Systems, Blind Spots, warned that flaws in widely adopted AI systems, including foundation models, could generate correlated losses across sectors and geographies. Its 2026 research also found that one in five insurance professionals surveyed reported their insureds had already experienced losses linked to AI risk.
For Lowenstein, that is precisely the exposure brokers and underwriters cannot afford to wave through. "That is a shape of catastrophic loss our market has never priced for and it is not a scenario we can ignore," he said.
