Reimagining Model Risk Management: New Tools and Approaches for a New Era

1. Introduction: The evolving model risk landscape

The rise of the model risk function

Models are integral to the finance industry, underpinning operations across many risk areas and business lines. Nevertheless, financial institutions have long acknowledged that these models, while essential, are inherently susceptible to error. Model risk can spread beyond financial applications to impact nearly every aspect of an organization’s operations. Consequently, institutions’ approach to conceptualizing and managing model risk has evolved into a more structured practice, driven by a combination of regulatory mandates, industry guidelines and supervisory directives.

In response to these evolving dynamics, financial institutions now understand and anticipate the need for a dedicated model risk function. This includes establishing enterprise-wide model risk frameworks, comprehensive policies and well-defined lines of ownership and accountability. Indeed, recent supervisory statements from regulatory authorities (SS 1/23 from the Bank of England’s PRA and E23 from OSFI Canada) now emphasize board-level responsibility for determining an institution’s model risk appetite and framework. Regulators have progressively codified these expectations, ensuring that robust model risk management (MRM), rather than being merely best practice, is now an essential component of operational compliance.

Ongoing market uncertainty

Alongside new standards and regulations, firms must also navigate considerable market volatility caused by several factors: interest rate uncertainty, geopolitical tensions, economic growth, inflation and stress on the banking sector. This volatility puts further pressure on firms’ MRM processes, affecting the performance of their models and making their model parameters more unstable. Market volatility can impact asset valuation modeling, for example; yield curve inversion poses challenges to existing curve analytics. Secondly, mortgage-backed securities (MBS) face particular challenges because of a lack of historical prepayment data that can complicate attempts to price accurately.

The need for automation

With more regulations, market uncertainty and requirements for multifaceted model use, managing model risk has grown increasingly complex in the past few years. Much like other functions, MRM functions feel the pressure to modernize their practice. Many are turning to automation to scale their capacity to manage, mitigate and track model risk. Financial institutions are now developing specialized tools to enable firms to carry out the various phases of model risk governance – broadly inventory management, model lifecycle management and risk threshold tracking. Software vendors saw this gap in the market, building tools for tasks such as assumption and version tracking and performance and stability monitoring, to support firms’ model lifecycle management processes and enable them to collect the data they need to ensure accurate and reliable models. Other key MRM processes that firms are automating include document generation and template population for model development and validation. Indeed, as automation technology advances, the domains of model risk governance and model validation are increasingly converging, offering institutions more sophisticated and integrated solutions for mitigating model risk.

This report explores the current and evolving MRM landscape (see Figure 1), offering an overview of the regulatory environment, examining the growth in automated tools, and considering the best and essential practices that institutions can employ to manage their model risk effectively. It highlights how firms’ selection of validation tools – which must provide robust auditability while complying with stringent regulatory standards – is contingent on the specific context of the models, the underlying theoretical framework, business demands and, critically, the regulatory landscape.

2. The development of MRM: model risk as an essential discipline

MRM has transformed from an informal practice into a structured and formalized discipline, with banks leading the way. Now, sectors such as asset and wealth management, insurance and technology are adopting similarly rigorous approaches to risk operations. This shift is primarily driven by:

• Significant regulatory developments.
• Increasing recognition of MRM’s critical role in the financial industry.

Regulatory catalysts

Examples of regulatory bodies that have been instrumental in shaping modern MRM practices include:

• The Federal Reserve, with its SR 11-7 guidance.
• The Basel Committee on Banking Supervision (BCBS), with its Principles for Effective Risk Data Aggregation and Risk Reporting (BCBS 239).
• The European Central Bank (ECB), with its Targeted Review of Internal Models (TRIM).
• The Bank of England’s PRA, with its SS1/23 guidelines.
• The OSFI E23 guidelines.

Together, guidance and regulations have made independent model validation, continuous monitoring and robust reporting mandatory, fundamentally altering how firms approach model risk.

More recently, the PRA has published a supervisory statement, SS1/23, which came into effect in May 2024. This outlines five key elements of effective MRM:

• Model identification and model risk classification.
• Governance.
• Model development, implementation and use.
• Independent model validation.
• Model risk mitigants.

Embedded validation requirements

Model validation and governance requirements are embedded into a variety of regulations and standards (see Figure 2). The modeling requirements introduced by International Financial Reporting Standard (IFRS) 9 and Current Expected Credit Losses (CECL), for example, have introduced a new suite of more advanced credit risk models into the banking book, helping to reshape the MRM landscape.

IFRS 9 and CECL outline the validation procedures firms need to ensure the accuracy, consistency and robustness of their credit risk models. But because firms must also develop, deploy and maintain appropriate models, they now have considerably more risk parameters to manage. IFRS 9, in particular, has prompted a surge in mathematical modeling, as firms address new requirements around impairment modeling and discounting. Meanwhile, for firms in Europe, the Middle East and Africa (EMEA), modeling of expected credit losses (ECL) is hampered by sparse availability of data, compared with the relatively data-rich CECL compliance environment in the US.

In the trading book, the Fundamental Review of the Trading Book (FRTB) is also driving model validation requirements, as banks grapple with new calculation methodologies that are both operationally and computationally challenging. The FRTB:

• Mandates annual independent validations.
• Expands responsibilities for internal audits.
• Demands granular, desk-level model approvals with stringent back-testing protocols.
• Introduces rigorous one-day value at risk (VaR) calibrations and profit and loss (P&L) attribution tests, both of which require high-quality historical data.

Under FRTB, banks must also identify non-modelable risk factors, develop stress scenarios and calculate capital requirements using the standardized approach, reporting all results to relevant authorities.

With these and other standards, regulations and guidelines, regulators now expect firms to provide detailed audit trails. This means that institutions must maintain thorough and accurate model documentation to ensure consistency, transparency and regulatory compliance. Firms must also track model changes alongside the lineage of their data – a tedious but crucial task.

Responding to regulations

While financial institutions understand the critical importance of MRM and strive to meet the stringent demands of regulatory compliance, managing model risk internally presents significant challenges. The extensive and complex nature of model risk requires validation processes that must be executed by experts on a large scale, creating demand for highly qualified but costly professionals. Moreover, validation reports should incorporate a full assessment of model assumptions, potential faults and benchmarking details, and an evaluation of model performance. The validation process is further complicated by being split into phases (such as internal and external reviews) and different levels (such as code and calculation), each of which can require specialized personnel. All these considerations and approaches must be aligned with an institution’s risk tolerance and internal MRM policies.

However, suitable tools are now available to tackle many of these challenges. More firms are adopting automated document generation and template-population tools, especially as they integrate generative AI (GenAI) to make documentation processes more efficient and accurate. Solutions can also include ongoing model monitoring and customizable templates, which are integral to comprehensive MRM.

Recognizing MRM’s importance

The PRA’s SS1/23 emphasizes the need for board and senior executives to understand model risk’s impact across all areas of the bank, reinforcing the clear responsibilities established in SR 11-7, and underscoring the importance of model risk as an organizational discipline that requires top-level oversight. Similarly, under SS1/23 a senior manager should be responsible and accountable for an overall MRM framework.

In fact, this specific emphasis from regulators reflects a broader industry trend: integrating model risk into overall risk governance frameworks. Firms are now expected to manage model risk with the same rigor as other key risks, blending technical validation and monitoring with strategic oversight and cultural integration at the highest levels of the organization.

Consequently, workflow configurations for validation and governance activities that integrate controls and alerts into the model lifecycle are established features of MRM solutions. For board members to have responsibility for MRM, however, they need solutions that also support explainable risk metrics and that clearly show the capabilities and limitations of most models. Effective dashboarding and reporting tools should illustrate a range of performance, model use and control measures with visualizations, metrics and tests. With these tools, boards can understand the potential risks associated with models and make more informed decisions.

Other market dynamics

Model risk: complex and interdependent

The growing significance of model risk within the broader risk management landscape is further highlighted by its interconnectedness with other risk types, including credit, market, operational and compliance risks. Regulators and industry leaders have also highlighted model risk’s multifaceted nature and extensive scope. In its 2021 Comptroller’s Handbook Model Risk Management, the Office of the Comptroller of the Currency (OCC) stressed the interdependence of various risk categories, identifying eight key risk types: strategic, operational, reputational, compliance, credit, liquidity, interest rate and price. This comprehensive approach encourages financial institutions to understand how risks in one area can affect those in other areas, helping to promote more integrated and robust MRM practices.

3. Transforming MRM: How emerging technologies are changing the practice

Existing tools

The growing complexity of model risk has led to demand for advanced MRM software solution and cross-functional model integration. Modern, increasingly sophisticated models now extend beyond financial risk into other areas such as operational and third-party risk. These models can also span various business areas, including banking, insurance and the buy- and sell-side sectors.

Operational risk, in particular, has evolved from a type of risk largely defined by regulators into a broader risk category that can be managed to some extent with advanced analytics. Indeed, reflecting the need for a holistic and integrated approach to MRM, organizations are now using a range of tools to define, quantify and manage operational risk across the company.

Moreover, model inventory – at both the business line and enterprise level – is a critical way for firms to keep repositories of information about model risk ratings, model owners and the type of models being used. Inventories can also help firms track direct and indirect interdependencies. By mapping relationships between models, firms can better understand how changes or failures in one might affect others.

Finally, firms can assess the complexity and materiality of models using model and risk tiering. These assessments inform the model validation process, covering aspects such as the intensity, frequency and type of tests being performed.

By maintaining detailed model inventories, assessing the complexity and materiality of models and applying the principle of proportionality, firms can better manage the interconnected nature of risks. The concept of proportionality, in particular, can impact the way resources are allocated within a firm, with more focus and controls directed toward higher-risk models. Model materiality, meanwhile, requires continual reassessment, supported by documentation and justifications of criteria.

Emerging tools

As MRM frameworks mature, shaped by regulators’ evolving expectations around model risk, significant advances in technology are helping institutions implement MRM effectively. Digitalization and expanded IT infrastructures have made it easier for financial institutions to monitor models, integrate policies and centralize their pricing and development libraries. Advances in technology, including open-source frameworks and such versatile programming languages as Python and R, have made sophisticated modeling more accessible.

While advances in technology offer powerful tools for improving MRM practices, they also introduce new complexities that require careful governance, as well as new challenges related to governance and control. The increased availability of these tools has led to fragmented and complex modeling environments, and there is a growing expectation among end users that models can be customized. Consequently, institutions must balance the benefits of technological innovation with the need for robust control mechanisms to maintain data integrity and manage model risk effectively.

Regulatory bodies, including the OCC, have highlighted the specific challenges posed by end-user computing tools. They stress the importance of controls on the use of these tools in model implementation, and require firms to assess the impacts and implications of end-user systems on their data integrity and model risk.

Financial institutions must also manage a diverse array of modeling technologies across different sectors. This includes the use of document workflow and data management tools, as well as model inventories and capabilities for market data integration and calibration. And as banks increasingly rely on third-party data sources to support their complex modeling environments, ensuring data integrity has become a critical aspect of MRM processes. Firms must have robust data management practices to maintain the accuracy and reliability of their models and manage risk effectively.

Another dynamic that firms must consider when managing their model ecosystem is interoperability between different types of modeling and programming frameworks. Solutions that provide flexible data models can capture the entire model lifecycle effectively, including development and testing.

Artificial intelligence and machine learning: innovation issues

Technological innovations in artificial intelligence (AI) and machine learning (ML) may support the evolution of MRM frameworks, but they are creating new challenges around model risk. Banks now use AI and ML tools across a wide range of processes and use cases, supported by hardware innovations and less expensive computational capabilities. The EU AI Act – a cross-industry regulation that is technology-specific and directly governs AI models – represents a departure from previous regulations and guidelines in the financial services industry. Its risk-based approach introduces stringent requirements that can overlap with existing regulations governing the financial sector.

Notably, AI systems used to evaluate credit scores, or creditworthiness, are now classified as ‘high-risk’, as are AI systems used in insurance to price life and health policies. Historically, AI techniques that equate to the decision-making component of a process, as well as business areas that require direct interaction with regulators, have often been deemed ‘high-risk’. But the use of statistical AI in retail finance has matured, and the ‘explainability’ of ML models – which demands a strong understanding of the context and data in which a model is deployed – has been a significant focus for users and regulators for some time.

Although other regional regulations are still being developed, and there is some uncertainty about their approach, the expectation is that documenting, testing and ensuring the transparency of all AI systems will continue to be a growing burden for financial institutions.

To navigate this evolving landscape, institutions must adopt robust MRM frameworks that integrate advanced technologies while ensuring compliance with emerging regulations and guidelines. To mitigate the risks associated with the growing complexity and widespread adoption of AI and ML techniques, firms will need effective data management, model monitoring and governance processes. Indeed, governance of AI and ML is now a distinct component of MRM regimes.

4. The future of MRM: industrialization and automation

Key steps in MRM’s continued evolution

More sophisticated validation tools

Sophisticated validation tools, which originated in large banks, have proliferated across the industry, driven largely by growing pressure from regulators, evolving standards and changing business models. Analytical accelerators, for example, are helping to make model development, stress testing and scenario planning more efficient. In addition, advanced automated systems now play a crucial role in ongoing monitoring, using various mechanisms that include:

• Real-time performance monitoring, to continuously track key model metrics and outputs to ensure optimal functionality.
• Customizable threshold alerts, to promptly notify risk managers when predefined limits are breached, allowing for timely intervention.
• Workflow automation of the different stages of model validation, to ensure consistency across testing and task requirements.
• Role-based access/user controls, to facilitate control and traceability throughout the model lifecycle while enabling collaboration among stakeholders.

Crucially, these advanced solutions are designed to integrate seamlessly with existing systems, workflows and data infrastructures without disrupting ongoing processes. Moreover, model risk solutions can now be configured to align with a firm’s specific templates, taxonomies, organizational hierarchies and business lines.

Enhanced model validation techniques

Modern model validation has multiple levels of testing that include conceptual, practical and code-based evaluations. For firms, the methods they use to assess the functionality of their models and to what extent they can be industrialized depends on the theoretical context in which the models operate. Statistical models (such as ECL models), for example, are validated using statistical tests that can often be automated and scaled. However, the time series used in these tests must be representative. Ignoring extreme events can render a model fundamentally inaccurate, which means that firms will have to use sophisticated, industrialized statistical tests for time-series applications and credit risk assessments.

Expanding the ML model validation lifecycle

The lifecycle of ML model validation extends beyond such traditional validation metrics as the area under the receiver operating characteristic curve (AUC/ROC) and cross-entropy loss. Robust ML governance requires a comprehensive evaluation of critical components such as accuracy and performance. Modern MRM solutions and MLOps systems now support advanced visualizations and workflows, enabling users to capture a wide array of data and hyperparameters. Experiment tracking has become an essential feature, allowing developers to compare and manage various model iterations under different training optimizations and hyperparameters. Moreover, for ML validation, firms are increasingly using sophisticated, industrialized statistical tests that highlight the importance of features or provide explanatory approximations.

More industrialization and convergence

Traditionally, model validation and model risk governance have operated as distinct yet interconnected processes. But as automation becomes commonplace in both areas, the trend is now toward a convergence of the two. As a result of the expanding diversity and complexity of models, coupled with end users’ heightened demand for customization, validation practices continue to be industrialized – fueling the convergence.

Because of the resource-intensive nature of validation procedures, organizations are increasingly relying on tools, accelerators and automation components. This shift not only complements the support provided by subject-matter experts, it also helps to streamline firms’ validation efforts. Consequently, the intersection between validation tools and model governance frameworks and workflows is widening, indicating a broader integration of processes.

Increasingly, governance, quantification and validation are becoming components of a unified MRM regime within an organization’s risk function (see Figure 3). In this context, financial institutions must clearly define roles for governance and validation teams. By adopting a more structured approach to MRM, with clear lines of defense and accountabilities that are properly and clearly defined, firms can manage their model risk more effectively, safeguard regulatory compliance and ensure they maintain the integrity of their modeling processes and outputs.

More mature MRM processes can now be characterized by industrialized model validation, which is a key driver in the convergence of governance and validation. To achieve industrialization, firms must:

• Automate and standardize their validation procedures.
• Ensure consistency across validation units.
• Fully integrate validation tools within a wider governance framework.

Inevitably, as MRM’s evolution continues, end users will have to embrace new methods and tools to use and develop technology tools effectively. And as model validation becomes more industrialized, the role of the ‘model validator’ may itself evolve into the ‘model risk manager’, with more responsibility for enterprise model risk.