IBM Cloud Pak® for Data Version 4.6 will reach end of support (EOS) on 31 July, 2025. For more information, see the Discontinuance of service announcement for IBM Cloud Pak for Data Version 4.X.
Upgrade to IBM Software Hub Version 5.1 before IBM Cloud Pak for Data Version 4.6 reaches end of support. For more information, see Upgrading IBM Software Hub in the IBM Software Hub Version 5.1 documentation.
Drift detection overview
Drift is the degradation of predictive performance over time. Watson OpenScale detects and highlights drift so that you can prevent errors in your model.
The Watson OpenScale drift monitor detects the drop in accuracy and the drop in data consistency in your model. A drop in either model accuracy or data consistency can lead to a negative impact on your business outcomes.
How Watson OpenScale identifies drift
To identify drift, the Watson OpenScale drift monitor analyzes the behavior of your model and builds its own model to predict whether your model generates an accurate prediction for a data point. The drift detection model processes the payload data to identify the number of records that your model makes inaccurate predictions for and generates the predicted accuracy of your model. The predicted accuracy is compared to the base accuracy of your model during training to identify the drop in accuracy. Also, the Watson OpenScale identifies the drop in data consistency by analyzing the training data and extracting characteristics to compare to your model at run time.
When you review results, you can view different displays of drift monitor metrics. Particular views depend on whether you are processing data as a batch or not.
Drift analysis for non-batch processing data
On the Watson OpenScale Insights dashboard, you can view a chart that displays the drift monitor metrics. Click a data point on the chart to view specific transactions that contribute to drift. On the transactions page, you can view transactions that are responsible for a drop in accuracy, a drop in data consistency, or both. You can also view the number of transactions that are identified and the features of your model that are responsible for reduced accuracy or data consistency.
When you click the Number of transactions link, you can view the transaction details page to understand how the transactions are evaluated for drift. The transaction details page specifies a reason for the drop in accuracy or drop in data consistency and provides a recommendation to help fix the drift of your model. Each transaction that contributes to drift in your model is specified in a table that provides details about the transactions.
Drift analysis for batch processing data
For the large quantity of data that can be produced by the batch processor, you receive a count of records that contribute to a drop in accuracy, a drop in data consistency, and both. In addition to this summary display, you can run a specialized analysis notebook: Notebook for analyzing payload transactions causing drift.
To calculate data drift successfully, Watson OpenScale must process data sets that can contain more than one-thousand columns. You can configure the batch processor in Watson OpenScale to process these data sets with Apache Hive.
Supported drift metrics
Next steps
Related topics: Understanding Model Drift with IBM Watson OpenScale
Parent topic: Configuring drift detection