Advanced Computing in the Age of AI | Saturday, December 9, 2023

AI Technology Leaders Often Lack Needed AI Ethics Understanding, Study Finds 

If you are an AI technology leader and you often don't have answers on how AI makes decisions within your company's operations, you are not alone.

Nearly two-thirds of C-level AI leaders can’t explain how specific AI model decisions or predictions are made, according to a new survey on AI ethics by credit report and analytics software vendor, FICO, which says there is room for improvement. Knowing exactly how AI model decisions and predictions are made is important to determine and chart a company's AI use and ethics policies and procedures.

FICO hired market intelligence firm, Corinium, to query 100 AI leaders for its new study, called “The State of Responsible AI: 2021,” which FICO released May 25. While there are some bright spots in terms of how companies are approaching ethics in AI, the potential for abuse remains high.

For example, only 22 percent of respondents have an AI ethics board, according to the survey, suggesting the bulk of companies are ill-prepared to deal with questions about bias and fairness. Similarly, 78 percent of survey-takers say it’s hard to secure support from executives to prioritize ethical and responsible use of AI.

More than two thirds of  the respondents say the processes they have to ensure AI models comply with regulations are ineffective, while nine out of 10 leaders say inefficient monitoring of models presents a barrier to AI adoption.

There is a general lack of urgency to address the problem, according to FICO’s survey, which found that while staff members working in risk and compliance, IT and data analytics have a high rate of awareness of ethics concerns, executives generally are lacking needed awareness.

Government regulations of AI have generally trailed adoption, especially in the United States, where a hands-off approach has largely been the rule (apart from existing regulations in financial services, healthcare, and other fields).

Source: FICO’s “The State of Responsible AI: 2021”

With the regulatory environment still developing, it’s concerning that 43 percent of respondents in FICO’s study found that “they have no responsibilities beyond meeting regulatory compliance to ethically manage AI systems whose decisions may indirectly affect people’s livelihoods,” such as audience segmentation models, facial recognition models and recommendation systems, the company said.

At a time when AI is making life-altering decisions for their customers and stakeholders, the lack of awareness of the ethical and fairness concerns around AI poses a serious risk to companies, says Scott Zoldi, FICO’s chief analytics officer.

“Senior leadership and boards must understand and enforce auditable, immutable AI model governance and product model monitoring to ensure that the decisions are accountable, fair, transparent and responsible,” Zoldi said in a press release.

As AI adoption increases among companies, it will only have a bigger impact on people’s lives, says Cortnie Abercrombie, the founder and CEO of the non-profit AI information group, AITruth, who contributed to the report.

“Key stakeholders, such as senior decision makers, board members, customers, etc. need to have a clear understanding on how AI is being used within their business, the potential risks involved and the systems put in place to help govern and monitor it,” she stated in the press release. “AI developers can play a major role in helping educate key stakeholders by inviting them to the vetting process of AI models.”

As the old saying goes, with great power comes great responsibility, Zoldi points out. Considering the power that AI brings, it’s time for companies to bring the same level of responsibility and accountability to their AI processes.

This article first appeared on sister website Datanami. 

About the author: Alex Woodie

Alex Woodie has written about IT as a technology journalist for more than a decade. He brings extensive experience from the IBM midrange marketplace, including topics such as servers, ERP applications, programming, databases, security, high availability, storage, business intelligence, cloud, and mobile enablement. He resides in the San Diego area.