Tracking assets in an AI use case

Track machine learning models or prompt templates in AI use cases to capture details about them in factsheets. Use the information collected in the AI use case to monitor the progress of assets through the AI lifecycle, from request to production.

Define an AI use case to identify a business problem and request a solution. A solution might be a predictive machine learning model or a generative AI prompt template. When an asset is developed, associate it with the use case to capture details about the asset in factsheets. As the asset moves through the AI lifecycle, from development to testing and then to production, the factsheets collect the data to support governance or compliance goals.

Service The required service is not available by default. An administrator must install the watsonx.governance service or the AI Factsheets service, version 4.8.3 or later, on the IBM Cloud Pak for Data platform. To determine whether the service is installed, open the Services catalog, and check whether the service is enabled.

Creating approaches to compare ways to solve a problem

Each AI use case can contain at least one approach. An approach is one facet of the solution to the business problem represented by the AI use case. For example, you might create two approaches to compare by using different frameworks for predictive models to see which one performs best. Or, created approaches to track several prompt templates in a use case.

Approaches also capture version information. The same version number is applied to all assets in an approach. If you have a stable version of an asset, you might maintain that version in an approach and create a new approach for the next round of iteration and experimentation.

This use case includes three approaches for organizing three prompt templates for an insurance claims processing use case:

Multiple approaches for an insurance claim use case

Adding assets to a use case

You can track these assets in an AI use case:

Prompt templates include the prompt input for a foundation model and variables that are defined to make the prompt reusable for generating new output.
Machine learning models that are created by using a Watson Machine Learning tool such as AutoAI or SPSS Modeler.
External models are models that are created in Jupyter Notebooks or models that are created by using a third-party machine learning provider.

Learn more

Use a sample project to try out watsonx.governance features with the tutorial Quick start: Evaluate and track a prompt template.

Parent topic: Governing assets in AI use cases