AI Pays Off, But Few Leaders Are Very Confident Explaining Its Decisions

Explaining how an AI-driven decision was reached remains a challenge for many businesses, even when the technology is meeting their financial expectations. A new FICO survey of 1,004 senior data, analytics and AI leaders found that 85.1 percent of respondents said AI had met or exceeded their initial ROI expectations.

However, just 28.5 percent of decisions that directly affect customers, such as personalization or low-risk customer service, were made by AI, and only 5.2 percent said they were very confident explaining AI-driven decisions to regulators and customers.

Rachael Hadaway, vice president of AI product at FICO, said companies can get returns before using AI in decisions that carry direct financial or regulatory consequences. “I think a lot of companies are still getting ROI on just the manual workload stuff that they can easily do without getting in any sort of trouble,” Hadaway told Newsweek. She said some decisions may require a company to explain to the customer involved or a regulator how the decision was reached.

FICO, a global analytics and decision management software company, examined AI returns, governance and explainability in its new report, “State of Responsible AI 2026: The Confidence Gap in AI Decisions.” The ROI numbers come with an important qualifier: Respondents were comparing actual returns with their own initial expectations, not a common financial benchmark. What Companies Have to Explain Scott Zoldi, chief analytics officer at FICO, said describing how an AI model typically handles similar cases is different from explaining why it reached a particular decision.

“You need to understand exactly what drove [the decision] from a model architecture perspective,” Zoldi told Newsweek. He said companies should also be able to trace what data went into decisions such as credit eligibility or fraud-related transaction blocking and where it came from. His concern is that knowing how a model generally behaves does not necessarily tell a customer or regulator why one person received a particular outcome.

The survey found another gap between respondents’ confidence about regulatory readiness and the extent to which they said responsible AI standards were incorporated into operations. Among respondents, 82.8 percent described their organizations as in line with or ahead of anticipated regulatory requirements.

Responsible AI adherence ranked lowest among five operational standards measured, behind model performance monitoring, security, regulatory and compliance adherence and continuous data quality monitoring. Only 60.5 percent rated it a four or five on a five-point scale.

Zoldi said the finding troubled him. He argued that confidence may partly depend on how much an organization is currently required to demonstrate. “What do I have to explain today from a regulatory perspective?

Depending where I am, very little, unless there’s some sort of customer impact,” he said. The finding also comes with a geographic caveat. Sixty-seven percent of respondents were based in North America, a concentration the report says likely shapes how they assess anticipated regulation.

Governance Beyond Policy Hadaway said FICO now brings privacy, legal and trust teams into development earlier. Waiting until a system has been built can leave teams with something that cannot be deployed. “You do need a singular standard, but you need many people to follow it,” she said.

Zoldi also pointed to technology fragmentation. “I spoke to one financial organization that had 26 platforms. And those 26 platforms don’t talk to each other,” he said.

Different parts of a business can solve their own technical problems while making it harder to bring information together and apply common standards across decisions that affect customers. Survey respondents saw potential financial value in a shared platform, with 96.4 percent saying closer collaboration between AI and technology leaders on a single shared deployment platform could generate additional ROI.

Only 8.2 percent said a shared platform was fully in place, while 71.2 percent described theirs as partially built.

Why Agents Raise the Stakes Almost every organization surveyed had begun exploring agentic AI—systems that can take actions with some autonomy—but most remained in the early stages. Some 38.2 percent were in early exploration and 33.

3 percent were running proofs of concept or pilots. Just 2.4 percent reported broad deployment across multiple customer-impacting use cases.

Security was the most commonly cited barrier to generating ROI from agents, at 40.1 percent, followed by data integration and quality challenges at 38.3 percent.

Lack of trust or predictability was cited by 5.6 percent. Zoldi pointed to fraud operations as one place where companies could divide the work between different types of systems.

An established predictive model can determine whether activity appears fraudulent, while an agent takes on surrounding tasks. “To actually work and triage the fraud you could have a set of agentic AI that will go gather information, ask questions, and interact with the consumer,” he said. Hadaway said that setup keeps the fraud decision with the predictive model while using the agent to replace manual steps in working the case.

The challenge grows as agents take on more of the work that shapes a customer’s outcome. “It’s really easy to build an agent, and it’s really hard to manage and govern an agent,” Hadaway said.