This blog series explored the meaning of business purpose and examined the importance of AI ethical frameworks, as well as the evolution of auditing practices to improve risk management practices and advance the foundations of digital literacy to support the evolution towards smart enterprises.
See Blog One in this five-part series defines business purpose and its importance in building stronger smart businesses. Blog Second in this five-part series, as this blog identifies key AI principles and frameworks or standards to guide directors and CEOs to increase their knowledge in these areas, as well as their C-Suite. Blog 3 identifies key questions a board administrator can ask their CEOs, and in turn, the CEO should be well prepared to answer strategic data management questions to manage data risk and to ensure the realization of data value. Blog Four identifies board and CEO risk management issues relevant to AI.
Directors and CEOs have the primary responsibility to focus on data foundations as a risk priority and to ensure that AI is used without strong and reliable data management foundations firmly in place. This fifth and final blog in this series focuses on what directors and CEOs can ask of their external auditors and explains why audit and control functions play a major role in good AI.
Articles abound on the risks and challenges associated with AI: gender and racial biases in recruitment, credit approval software; chatbots turned racist, inaccuracies in predictive models for public health, and reduced confidence in machine learning models.
Some developments deserve to be known. First, the Algorithmic Accountability Act, proposed by Democratic lawmakers, if passed, would require large corporations/publicly traded companies to assess the accuracy and fairness of their “high-risk automated decision systems.” The EU’s GDPR audit process also covers some aspects of AI, such as a consumer’s right to an explanation when companies use AI algorithms to make automated decisions. Additionally, the Information Commissioner’s Office (ICO) in the UK has come up with an AI audit framework that has a much wider scope and is worth watching as Europe moves faster in its frameworks. privacy and AI legal issues.
The ICO framework has identified eight AI-related risk areas and identifies governance practices such as: leadership engagement, reporting structures, and employee training.
Boards and CEOs can’t wait for all of these legal and regulatory frameworks to be in place, so having 3rd party audits/internal model reviews, making sure the use of AI models is well documented like : document the names of AI model developers, assign risk ratings (data, societal, financial, etc.), in case a model fails. AI audits should deepen the assessment of the quality of training data and ensure that algorithmic methods are robust and fair.
A robust audit process for data management practices should be the way forward to govern AI/ML decisions and ensure alignment with business purpose.
Here are some of the key questions a director or CEO may consider asking an audit firm.
1.) Does your audit practice have proven expertise in data management audits and AI/machine learning areas?
2.) Do you have a trust framework for AI ethics and a set of operating principles or guidelines that can guide our corporate AI initiatives from the perspective of risks?
3.) Does your audit practice apply your AI audit frameworks in your own internal operations and have your internal practices reviewed by an independent auditor?
4.) Have you helped evolve the procurement and legal functions to have risk management practices in relation to black box AI approaches versus trusted (visible) AI approaches?
5.) Do you have a digital literacy program for all your listeners with a foundation in data management and AI?
6.) What lessons have you learned from helping companies implement more reliable AI auditing practices?
7.) Are the resources assigned to your account certified in data management and AI?, and do they have the qualified skills to perform your audit review rather than leveraging the overall company brand. Note: Always know the skills and abilities of the talent assigned to your account (inspect).
This five-part blog series was written to motivate administrators to ensure they have a clearly defined business purpose and that investments in technology, especially with AI and ML methods, are a foundation of data excellence is in place, ethical AI practices and third-party audits. Although many legal frameworks are not yet globally aligned, directors and CEOs need to understand that their most valuable strategic asset is their data, and company valuations will change as audit frameworks and regulatory risk assessment are evolving. Most board audit committees do not have a business or technology risk manager for AI and ethics, or a director with experience in these areas.
I recently completed my ICD.D Board Administrator certification at the Rotman School of Business at the University of Toronto. It was a great program overall, but there were no core AI and ML risk modules, although cybersecurity and data leakage risk were discussed, and nor was there a separate discussion of the company’s purpose and its importance in technology investments. With approximately 8.5% of revenue across all segments, investing in IT, strengthening technology governance and aligning with business purpose.
Newer governance models such as ESG (environment, social, governance) offer new opportunities to integrate AI, ML and business purpose into stronger governance practices. Unfortunately, the majority of ESG consultants have limited expertise in AI/ML/technology, which impacts the integration of AI/ML into these new frameworks.