What's new in watsonx.data version 1.0.3
This section includes the new features and functions that are included in watsonx.data v1.0.3.
Upgrade
The Presto engine was updated from Version 0.279 to Version 0.282, which includes enhancements and defect fixes.
Security
- Administrators can now grant access to user groups to manage users with similar access requirements to the watsonx.data infrastructure components.
- Db2 Warehouse on OpenShift cluster can now establish connections by authenticating the built-in Hive Metastore (HMS) service on watsonx.data software.
- The coordinator and worker nodes can now communicate securely with TLS in a multi-node or multiple worker configuration of a Presto engine.
- You can now manage certificate renewals by rotating the Presto SSL certificates. The ability to rotate SSL certificates, enable the Presto and HMS service to continue to communicate with each other when one of them is restarted with the new certificate.
Connectors
- You can now use the dereference pushdown feature to push down the parquet dereference expressions in the Iceberg connector. With dereference pushdown, the query performance is improved by accessing the nested fields and by reducing the CPU and query runtime.
- You can now use the custom catalog to create connections to data stores that are not provided by the built-in connectors. For example, you can set up the custom connector that is hosted and managed by AWS Glue.
Integrations
- watsonx.data stand-alone v1.0.3 supports integration with an Analytics Engine serverless instance to run your Spark workloads. For more information, see Integrating with Analytics Engine.
- Tech-preview watsonx.data stand-alone v1.0.3 supports integration with Watson Knowledge Catalog. For more information, see Integrating with Watson Knowledge Catalog.
APIs
A Command Line Interface (CLI) wrapper is now available for watsonx.data APIs to perform the following tasks:
- Work with engine resources.
- Register and work with external databases.
- Register and work with object storage buckets.
- Configure connectivity to the API endpoints.
Performance
You can now enable and configure fragment result caching in watsonx.data. Fragment result caching enables virtualization of data sources and improves performance by caching the partially computed results of SQL queries. For more information, see Setting up fragment result caching.