Summary

Situation Overview: Anthropic CEO Dario Amodei’s We Must Pace the Frontier memo set off alarm bells for artificial intelligence (AI) safety, but also offered a solution well known to regulated banks: independent embedded evaluators. Mr. Amodei’s memo highlights that individual firms do not need to wait for regulatory action to benefit from independent oversight; firms can learn from the banking industry’s lessons by taking their own steps. While his memo was primarily focused on firms that develop AI, his proposed framework of embedded evaluators can be leveraged by any firm using AI.

What: The complexity and risks of AI are changing rapidly, both within the financial services industry and in other areas. Against a backdrop of regulatory uncertainty, industries are left to define their own mechanisms to manage AI risk. Clearly establishing three lines of defense, a well-established banking industry construct, can help identify, manage, and mitigate the risks in a rapidly changing environment.

Who: Financial and non-financial firms using AI.

When: Now.

In Depth

AI creates internal and external risks for all firms. Internal risks stem from a firm’s use of AI regardless of whether that AI is internally developed or sourced from a third party. External risks stem from what others’ use of AI could do to the firm, such as AI-enabled cybersecurity attacks.

To manage risks and align with regulatory expectations, banks have formally established three lines of defense. At a high level, these lines of defense can be summarized as:

1. First Line: The Business and Operational Functions. These typically include revenue-generating teams, such as traders, relationship managers, product development, marketing, and sales, as well as supporting operational units, including information technology, information security, and back-office functions. These first-line-of-defense units own the risk and are responsible for identifying the risks, assessing the risks, designing and implementing controls, and managing and monitoring the residual risks within their firm’s “risk appetite” for that risk type.
2. Second Line: Control Functions. In a bank, these generally include Risk Management and Compliance. These “control functions” are responsible for establishing the firm’s risk management framework, including policies and limits, and, in conjunction with the Board of Directors, defining the firm’s risk appetite. The second-line-of-defense teams are also responsible for independently reviewing and challenging the management of risks by the first line of defense.
3. Third Line: Independent Audit. This function provides assurance, through its independent review and testing activities, to the Board of Directors on the effectiveness of the governance and control activities carried out by the first and second lines of defense.

Mr. Amodei’s memo commits Anthropic to establishing independent embedded evaluators to, among other tasks, “verify safety practices and report incidents,” acting, in a sense, as a second or third line of defense. Should firms that are using AI, such as non-banking financial services firms, or companies in other industries, also take this approach? We would argue that they absolutely should. Certain AI use cases can create material risks for a firm. Managing those risks requires a set of checks and balances that the banking industry has learned can best be accomplished through defined roles and responsibilities for the ownership and management of the risk (by the first line of defense), for the establishment of policies, the setting of limits and undertaking of credible challenge (by the second line of defense), and for the independent assessment of those activities (by the third line of defense).

A few key success factors can be observed in the firms that have been able to implement “three lines of defense” around AI use:

• The formality of the organizational structures to support the delineation of these responsibilities can scale with the materiality of AI use by a given firm, but the key concepts of clearly defined authorities and accountabilities are documented and communicated to all stakeholders.
• The professional expertise and skill sets across the three lines of defense are aligned to achieve the desired outcomes and add value to the firm.

Put Patomak’s Expertise to Work

Patomak is uniquely positioned to help organizations navigate governance and risk management challenges raised by AI. As institutions rapidly adopt generative AI, AI agents, and other AI tools, Patomak can help with scaling and adapting governance structures.

Patomak has extensive expertise in designing and enhancing governance programs tailored to an institution’s size, complexity, and risk profile, ranging from establishing governance frameworks and clarifying three-lines-of-defense responsibilities to implementing targeted improvements. To learn more about how Patomak can support your compliance and strategic goals, please contact Diane Daley at ddaley@patomak.com or Heather Espinosa at hespinosa@patomak.com.