Known issues and limitations for AI Factsheets

Additional configuration required for external model tracking

If external model tracking is enabled, the Cloud Pak for Data system administrator must set the option for Duplicate asset handling to Allow duplicates in catalogs.

Watson OpenScale database information not published

Database host information from Watson OpenScale is not published for Payload logging or Feedback data.

Applies to: 4.6, 4.6.1, 4.6.2
Fixed in: 4.6.3

Metrics from Watson OpenScale monitors not published

The Watson OpenScale monitor metrics are not published to factsheets when a pre-production subscription has custom monitors configured.

Applies to: 4.6, 4.6.1, 4.6.2
Fixed in: 4.6.3

Common namespace required for related services

For AI Factsheets to log the appropriate information, all supplemental and compatible components must be installed in the same namespace. This includes the following services:

• Watson OpenScale
• IBM OpenPages
• Watson Knowledge Catalog
• Watson Studio
• Watson Machine Learning

Tracking pauses during upgrade process for Watson Knowledge Catalog

If you used AI Factsheets as part of Watson Knowledge Catalog before 4.6 and must install it again as a standalone service, tracking updates will be suspended from the time of the Watson Knowledge Catalog upgrade until AI Factsheets is installed. The facts for the models that are created or updated in that gap will not be tracked by AI Factsheets. You need to update the metadata of the model such as name, description, tags.

See Upgrading AI Factsheets for more details.

Cannot change or update OpenPages instance

Once AI Factsheets is integrated with an OpenPages instance, there is no option to change the instance or to enable integration with another OpenPages instance.

OpenPages model use cases and some external assets owned by System in model inventory

When the AI Factsheets service is integrated with IBM OpenPages, the model use cases that sync with the model inventory are owned by System because they are created by the Factsheets service. Similarly, when external model assets are created with OpenScale evaluation facts, those assets are also owned by System. Catalog users cannot edit them. If you have Administrator access to a catalog, follow these steps to assign a user as owner:

1. Open the asset in the catalog and select the Assets tab.
2. Add the user who will be assigned ownership as a collaborator.
3. Click the Edit icon for Asset owner in the About this asset section.
4. Select the user name to assign as owner.