Trustworthy AI – A Market-Driven approach
Market-driven approach to achieving more dependable AI performance and guardrails through shared responsibility and liability
What if insurance companies - not regulators, governments, or tech giants - hold the key to fixing AI’s biggest reliability, accountability, and safety problems? What if a market-based solution that drives innovation, fits well with existing business practices, and saves money for all parties, were presented to you? Would you consider it?
We already know that AI can cause real harm. It can present false information that appears to be reliable. It can produce results tuned and filtered to please you rather than give a complete response. It can push an agenda embedded in the algorithm that you are not aware of. Its output can damage a company’s reputation, hurt customers, or cause financial losses in unexpected ways.
Some insurance companies are starting to offer varying degrees of AI liability coverage (ex. Armilla affirmative AI coverage, Munich Re's aiSure, Testudo). These policies offer protection after a damage has occurred.
But the insurance industry is well positioned to motivate both the companies that offer the AI and the companies that license the AI to drive processes that proactively make AI more dependable, reliable, and safe through a practical, market-based approach—one that reduces risk, lowers costs, and creates new business opportunities for all of the players involved.
Here is how such a system would work.
AI developers and the companies that use their systems, and who already work under two-party contracts via the click license or customized equivalent, would negotiate and agree to an additional clause in their two-party contract. The modification would require regular quality control checks focused on unusual or extreme situations—often called “outlier cases.” These are scenarios that may be rare, but could cause serious harm if the AI responds badly to them. The outlier cases could also be harbingers of future business opportunities.
If an outlier test shows that the AI produces a harmful result, both parties would be required to act. The user company would first review how it is using the AI. If fixing the local implementation is not enough to minimize the probability of harm, the AI platform developer would then evaluate whether changes to the underlying platform are needed. The platform developer would need to consider how any fixes might affect other customers. Both the user company and the platform developer would be contractually obligated to act within an agreed upon timeframe to reduce the probability of that harm happening again.
The best people to design these outlier tests are subject-matter experts inside the user company. They understand what is normal for their business and what outlier situations could realistically occur in the future. Outside consultants and insurance company representatives could help guide and audit the process, but they should not define the scenarios directly. The AI developer should be excluded from designing the tests, since it could be argued that it would not be in their interest to uncover weaknesses in their own product.
Why would either side agree to this additional work?
The answer is insurance and liability. If harm occurs and a lawsuit follows, then both companies are potential plaintiffs. Through this ‘outlier testing’ process both companies could show that they took reasonable, documented steps to minimize the probability of problems. Courts tend to view these “good faith efforts” favorably, especially if an independent auditing firm corroborates the effort. Insurance companies, seeing that the risk of harm is lower, could use the reduced risk profile to calculate reduced AI liability premiums to both the developer and the user companies. Once one insurance company does this to differentiate their product and strive for competitive advantage, others will follow to remain competitor or they will lose customers who value this protection.
In this way, the insurance industry could become an unexpected driver of safer and more trustworthy AI. It would move to ever-widening AI coverage as the benefits of the audited two-party obligations and the risk profile from potential claims become clear. Over time, consumers would benefit from AI systems that are more reliable and less likely to cause them harm.
This approach could be used anywhere in the world. Because it relies on simply adding a new clause for negotiation in the already-existent contract between two parties, it does not depend on government regulations or industry standards. It could be implemented immediately in the United States, Europe, China, or elsewhere, and it could later be incorporated into broader local, regional, and global standards if desired.
AI platform developers may argue that this system would slow or “stifle” innovation. In reality, it would encourage a different kind of innovation. It would encourage building AI systems that meet clear performance and safety obligations. Once one AI platform developer adopts this approach, it becomes a competitive advantage. Similarly, once an insurer offers lower premiums for companies that follow these practices, adoption would accelerate across the market. So the response to AI platform developers is that by claiming innovation would be stifled through this approach, they are working to stifle innovation of more responsible, safe, and dependable AI.
This model also addresses ethical concerns without debating ethics directly. Laws reflect a society’s shared values, and insurance pricing reflects the cost of breaking those laws. By designing contracts that reduce legal risk, ethical behavior is built into the system automatically. This means that copyright, trademark, and patent law, as well as the rights of the creative community, will be protected to the degree that compliance with the contract terms reduces the probability of legal violations.
By moving toward shared responsibility via the two party contract obligations, we can spread and reduce the risks as AI plays a larger role in our lives. This shared-liability approach offers a practical path toward safer, more dependable, and more reliable AI for everyone. It also rebalances the power to shape our AI future; from the few AI platform developers who are motivated to shape the platforms to serve their own interests, to shared risk, responsibility, and agency with the global community of AI users.