The insurance industry has had access to AI tools for years. The carriers and managing general agents (MGAs) pulling ahead are not necessarily those with the most sophisticated technology. They are the ones that got their operations in order first.
In this interview, ReSource Pro’s Michael Costonis explains what it actually looks like when inconsistent processes meet AI at scale, what managed AI services are and how they differ from simply bolting new tools onto an existing operation, and how leaders can honestly tell which camp their own organization is in. If you are trying to figure out where to start, he has a clear answer on that too.
Michael Costonis [00:00:00] The true value comes from discipline, guardrails, transparency, and thoughtful execution.
Michael Costonis [00:00:14] If we think about operational discipline, it’s kind of simple. What do we really need to power the AI? You need a process that you can structure. You need data that’s available. You need clear decision rights as to when and how decisions are going to be, to get made and an outcome that you’re pointing to. Without those sorts of things, you don’t have any of the ability to truly harness the value of that technology. Typically, when you walk into an organization that hasn’t gone through that process, you’ll see AI in pockets, individual people with a productivity hack here and there, or maybe a point function where they’re actually using the solution. But work is being pushed upstream where it doesn’t need to go. And quite frankly, the results are very difficult to measure if I’m actually achieving anything.
Michael Costonis [00:00:59] Typically what you see is a great amount of excitement around an AI solution. It gets pushed down to the individual user and they create their own way to be able to handle the process using the new tool. The challenge is when you go from one person to the next person or one office to the next office or even one function to the next function, you’ll see many different iterations trying to accomplish the same objective. And not necessarily executing it the best way. You’ll see redundancy, you’ll see data that doesn’t necessarily match up, and you’ll see a lot of inefficiency with the way that people are trying to scale the solution, but no real true discipline around it, which results in people not being able to trust the result, not transparent about how the result was actually created, and actually pushing more work back onto the end user to get the true work done.
Michael Costonis [00:01:46] If we take a step back, when AI is really effective, it’s a combination of human and machine. The model needs to be trained. You need to have the right evaluations around it. You need to be able to adjust the workflow. So managed agentic services are humans working with the machines to be able to refine the workflows, provide the right feedback back into the model, work and modify the actual procedures to achieve the outcome, all the while you’re getting better performance out of the actual model itself and changing the business structure that goes along with it.
Michael Costonis [00:02:23] I think there are a couple different categories here to look for. One are the experimenters. The experimenters have rolled out AI to all of their people and they’re using it in individual places. Someone found a productivity hack there. Somebody actually put it into part of their department there. The real tell is — is it scaling and is it adopted? The people that are capturing the real value take a more deliberate approach. Look at an entire function, look at it from end to end, understand are the processes standardized? Is the data available? Do we have a process which when we’re going to use the AI versus we use other techniques, deterministic code, you know, those sorts of things. The true value comes from discipline, guardrails, transparency, and thoughtful execution.
Michael Costonis [00:03:09] I think you need to look at a full end-to-end workflow and really understand who’s doing what part of the process and where’s the value being created and where’s it being destroyed. So for example, there’s a lot of, and then the magic happens here with AI when I put a document into something and I get an output. But as soon as that work goes back to the underwriter or goes back to a supervisor for evaluation, you’ve destroyed the value of actually using the AI to create some efficiency. You’ve lost a ton of effectiveness. So again, I would say look at an end-to-end workflow. How is it being executed? Where’s the work being done? Can I trust the decision that, that comes from it? Can I get transparency into the results that are being achieved? That’s the real litmus test if I’m employing it correctly.
