Every machine learning algorithm deployed in insurance underwriting faces a structural problem that most boardrooms have not fully confronted. These tools are built to find all correlations between input data and the outcome being predicted. They do not stop at the correlations an insurer would consciously choose. They also find, and exploit, correlations with race, gender, health status, disability, and every other characteristic that state law explicitly prohibits using. The problem is not intent. It is mathematics.
“Regulators think they can police the data and say, this looks like it might be relevant, so you can use it,” said Daniel Schwarcz, Fredrikson & Byron Professor of Law at the University of Minnesota Law School, whose peer-reviewed research on proxy discrimination in insurance AI has been published in the Iowa Law Review. “They fail to recognize that what the machine learning algorithm will do is find ways to use that data to proxy for protected characteristics that are predictive of claims. Simply denying AI access to the most intuitive proxies does little to thwart this process; it causes the AI to locate less intuitive ones.”
The implication is direct: an insurer can exclude race from its inputs entirely and still produce outcomes that effectively discriminate on that basis. Schwarcz describes the resulting competitive dynamic as self-reinforcing. “There’s almost a competitive imperative,” he said. “These tools work in helping you predict claims better. As long as regulators allow you to do it, in some ways you’re almost compelled by market forces, because it allows you to be more predictive in ways that are implicitly leveraging protected classes.”
Proxy discrimination is the highest-order liability in AI governance today. But it sits within a broader regulatory environment that also demands attention: a patchwork of state standards with wide variation in what is tested and enforced, a 2023 national model bulletin that sets process expectations without mandating outcomes, and testing regulations that remain unimplemented years after their legislative mandate. On the horizon sits a meaningful possibility that regulators eventually move from guidance to outright restrictions on where machine learning can be deployed.
The National Association of Insurance Commissioners (NAIC) adopted a Model Bulletin on the use of artificial intelligence systems by insurers in December 2023, and roughly half of states have since adopted it or issued comparable guidance. It is the most visible reference point for AI governance in the insurance sector and, by design, a process document: it requires carriers to maintain a written AI governance program covering board accountability, risk management, and third-party vendor oversight, but does not specify testing methodologies, mandate outcomes data collection, or direct carriers what to do when a deployed model produces evidence of disparate impact.
Birny Birnbaum, executive director of the Center for Economic Justice and a former associate commissioner at the Texas Department of Insurance who has served as a designated consumer representative at the NAIC for many years, draws the line between governance and protection plainly: “Governance practices are necessary, but they’re not sufficient. Until we start collecting data and doing testing, both by the companies and by regulators, there’s no way to tell whether the governance is having any kind of impact at all.”
The state-level picture reflects that gap. Colorado passed legislation in 2021 requiring insurers to test for unfair discrimination in their use of AI models; as of mid-2026, nearly five years on, no testing regulation is in place for any line of insurance. “Despite having a great legal foundation, Colorado has really failed consumers,” Birnbaum said. New York remains the exception: it has issued bulletins specifically requiring carriers to test for unfair discrimination using AI. Schwarcz views the national environment as leaving far too much underspecified, both for insurers trying to determine what compliance means and for regulators determining how to enforce it.
The NAIC is currently piloting an AI Systems Evaluation Tool across 12 states, designed to give examiners a standardized approach to reviewing insurer AI governance programs during market conduct examinations. The tool is expected to inform whether a formal model law follows. The regulatory direction is toward testing, not just documentation, and executives who wait for a formal rule before building testing infrastructure will be building under examination pressure.
What the emerging regulatory trajectory demands in practice is documented testing at both development and post-deployment stages: comparing actual consumer outcomes to intended ones and examining whether those outcomes diverge by protected class. The technical approach referenced in formal advocacy submissions to the NAIC and the International Association of Insurance Supervisors involves adding a statistical control for a protected class characteristic to a model in development, then measuring how much predictive power the other variables lose. A variable that loses most of its power when a race control is added was predicting race, not the outcome.
Technology is available to support this work. A growing category of AI governance tooling, including model explainability and interpretability platforms, automated bias detection systems, and fairness audit infrastructure, has emerged specifically to make this testing tractable at scale. Carriers using AI embedded in third-party platforms they did not build should treat vendor governance as a near-term gap: the obligation extends to the vendor’s algorithm, and most existing vendor contracts were not drafted with that obligation in mind. Governance structures should also position legal and compliance at the center of AI deployment decisions, not at the end of the review chain.
Insurance regulation moves slowly. Colorado’s testing mandate has sat unimplemented for five years. The NAIC has been discussing AI governance since 2020. That pace does not mean no change. Schwarcz identifies a scenario that should factor into long-horizon AI strategy: regulators moving from governance guidance to affirmative restrictions on machine learning use in specific functions. “Requiring more robust auditing, requiring efforts to examine the effects of machine learning algorithms on particular protected groups, and maybe affirmatively limiting the use of machine learning tools in some settings, particularly in pricing, may be the optimal approach,” he said. “We’re not there. But I think that’s the calculus for a regulator.”
A carrier that has built its pricing architecture around opaque machine learning models with no testing record to show for it would face a difficult position if that restriction arrived. One that has already built testing into its model lifecycle, maintained auditable documentation, and given legal and compliance genuine authority over AI decisions is preparing for a regulatory environment that, on the current trajectory, will eventually move from asking how carriers govern their AI to asking what that AI is doing to consumers.
