Many financial firms are increasing their use of AI models because they can represent the real world more accurately, and they can deliver better projections than traditional, rule-based models. But some AI models can add complexity and risk.

You can minimize that risk and also streamline the process of model validation by using IBM Cloud Pak for Data, a data and AI platform that includes IBM Watson Studio, Watson Machine Learning, Watson OpenScale and other services.

Let’s take a closer look at how this platform can help to streamline model risk management.

In a previous post,  I mentioned that financial firms can quickly face substantial business losses if AI models do not perform as intended. AI models need to be monitored and managed for risk, following guidance such as the Federal Reserve Board SR 11-7. This is important across the entire lifecycle of a model, from its proposal through deployment and decommissioning. A sample model life cycle:

Most financial firms have model risk management frameworks in place that have been optimized for traditional, rule-based models. New approaches are needed for the new challenges that AI models present. In addition, it can take months to validate a model, and often the AI expertise that is required is in short supply.

Let’s look at how to address these challenges. Our goals: streamline the model life cycle, shorten time to AI value, reduce operational costs and lower model risk.

Automatically create AI models and challenger models using AutoAI

Developing an AI model can be complex and time-consuming, requiring special data science skills which are not always available or easy to gain. Using AutoAI technology in IBM Watson Studio – included in Cloud Pak for Data – you can automate the entire data preparation, model building and deployment process. Simply select a data set and a field to predict or classify, and AutoAI takes care of the rest. No coding is required.

Here’s how it works: AutoAI uses AI to build AI. It generates several model pipelines which are displayed on the leaderboard. This enables you to easily compare the performance of different candidate models and select the best one based on several automatically calculated metrics. AutoAI can also be used by model validators to generate challenger models – a common practice for in today’s model risk teams – and help ensure the optimum model is chosen in the end.

In addition, AutoAI accelerates the entire model life cycle by automating mundane basic tasks that can take data scientists days or even weeks. The data science team can focus on higher-value work that makes better use of their expertise.

Automate model testing throughout the model lifecycle

Model risk management guidance such as the Federal Reserve SR 11–7 requires that models are validated regularly to ensure that they are performing as expected.

Watson OpenScale, which is part of the Cloud Pak for Data platform, automates active testing of AI models to support their validation and ongoing monitoring. This can help validators reduce the time needed for validation since it minimizes the time needed for generating and executing test scripts.

Our IBM team is excited to announce that new capabilities in Watson OpenScale that enhance support of AI model validation are now generally available. These include the ability to:

• Validate models in pre-production with advanced, customizable tests relevant to AI. These include detecting bias and drift, calculating quality metrics for back testing, and generating model explanations
• Automatically execute tests and generate test reports
• Compare model performance of candidate models and challenger models side-by-side
• Transfer successful pre-deployment test configurations for a model to the deployed version of the model and continue automated testing

More detailed information about these new capabilities are in this blog post.

Synchronize results with governance, risk and compliance solutions

Many model risk management teams are using governance, risk and compliance (GRC) solutions, which typically offer capabilities such as tracking model inventory, supporting customizable business workflows, and documenting the lifecycle of a model. IBM offers a GRC tool called IBM OpenPages Model Risk Governance. Watson OpenScale is fully integrated with OpenPages Model Risk Governance. You can synchronize test results and reports and avoid the need to manually copy or enter data. You can save substantial model validation—especially when multiple models are in development and production.

Learn more

Watch an American Banker webinar in which two model risk management professionals at RBC review MRM challenges, and an IBM expert shows how to automate model validation tests. Register for Automating AI model risk management at financial firms.

Visit this webpage for a more detailed look at how Watson OpenScale simplifies model risk management.

Explore IBM Cloud Pak for Data, a fully-integrated data and AI platform that modernizes how businesses collect, organize and analyze data and infuse AI throughout their organizations. With the announcement of Cloud Pak for Data version 3.0, clients can more seamlessly automate, govern and process data at scale, and achieve cost savings. The Forrester Total Economic Impact of IBM Cloud Pak for Data report found that Cloud Pak for Data delivers a three-year projected return on investment (ROI) ranging from 86 to 158 percent, and it reduces infrastructure management demands by 65 to 85 percent.

Accelerate your journey to AI.