IBM Integrated Analytics System 1.0.23.2 release notes (September 2020)

Version 1.0.23.2 replaces version 1.0.23.1.

Important:
  1. If you are on IAS 1.0.19.2 or lower, you must upgrade to IAS 1.0.19.7 before you can upgrade to IAS 1.0.2x.x.
  2. On 20 August 2020, the SSL Certificate on IAS and Db2 Warehouse systems expired. IBM has developed an updated SSL Certificate strategy using a self-signed certificate. For SSL database connections to work, you need to trust the certificate from all your clients when you upgrade to 1.0.23.2. For instructions, see Secure Socket Layer (SSL) support.
  3. If you want to preserve SSL certificate files during upgrade, you must specify the old certificates by using SSL environment variables in the dashdb.env file. For more information, see Preserving old certificate files during upgrade.

Version 1.0.23.2 replaces version 1.0.23.1, fixing:

  • HA issues (failure to start, long startup time).
  • Issues with insecure protocols TLS 1.0 and TLS 1.1 in nginx.

    Support for insecure protocols for web/REST connection was removed by setting ssl_protocols TLSv1.2; in /etc/nginx/nginx.conf.

  • AD setup failures.

    A desc': u'Size limit exceeded error appeared during setup. As a solution, a new console container is introduced.

  • Upgrade failure due to an unexpected fabric switch configuration.

    This was caused by a stale NTP configuration detected during the fabric switch hw maintenance. To resolve the issue, a non-essential switch configuration is no longer verified during this mode of sys_hw_config.

  • Problems with connecting to database from the console that were caused by an expired SSL certificate.

    This error appeared: Warning: The console is unable to connect to the database. Common reasons are that the target database is offline or the console is unable to connect to the SSL port. To diagnose the problem, use the console to check the system status. If necessary, consult the Db2 logs..

What's new

DSX
IAS 1.0.23.0 is the last release on DSX.
FSN
Updated the firmware to version 1.5.2.6.
dashDB
Limited dashDb system resources to avoid resource starvation for other containers.
Backup and restore
  • If db_backup fails, it removes all files that were created before exit. This is needed to clean up space. Information about disk usage and disk space consumed by backup is displayed as part of failure.
  • Improved logging messages. Now the backup and restore version is provided.
mounts_count resmgr
Extended the timeout period for mounts_count resmgr, so that when the system is overloaded and the execution time of resmgr reaches about 55 seconds, a timeout doesn't happen.
GPFS
GPFS configuration settings used to be stored and maintained by nodeOS and apupgrade. The settings conf file and RPM creation were moved to nodeOS from apupgrade.
Db2 support tools
db_migrate
  • Added support for loading Netezza INCREMENTAL backup in db_migrate. (FULL backups are already supported in db_migrate.)
  • Added a db_checksum enhancement to calculate partial checksums of columns. This makes it easier to narrow down data corruption. For more information, see Backing up by using the db_backup command.
dbload
  • Enhanced the dbload command so you can now load data into tables with hidden columns.
  • Enhanced the dbload command so now you can load data from standard input of the program.
  • Enhanced the -delim option; it now supports hexadecimal values.
For more information, see dbload command.

Components

IBM® Integrated Analytics System 1.0.23.0 includes the following components:
Db2 Warehouse 11.5.4.0
See What's New in Db2 Warehouse.
Db2 Engine 11.5.4.0
To learn more about the new features and changes that are introduced in Db2 11.5.4.0, read What's new in the Db2 IBM Knowledge Center.
IBM Data Replication for Availability
You can now replicate schemas and tables that are enabled for the Db2 columnar online incremental backup and restore (schema enabled for row modification tracking) feature by setting a unique replication identifier at the source and target databases. The schema enabled for row modification tracking feature allows you to continue DML operations on Db2 column-organized tables during a schema backup and restore operation.

Known issues

DSX
You must reinstall DSX, as all DSX containers are gone after upgrade. During the upgrade items such as unused containers and the DSX core are pruned. The DSX containers are part of the separate DSX installation bundle, not part of the IAS upgrade image.

Workaround:

Back up and restore DSX/ Watson Studio Local as described in https://content-dsxlocal.mybluemix.net/docs/content/SSAS34_current/local/backup.html.

Note: If you want to disable DSX, you have to uninstall it. To uninstall DSX, run ./InstallPackages/utils/uninstall.sh from the directory where install was run. Usually, the installation directory is located in /opt/ibm/appliance/storage/platform. If you can't find the directory, contact IBM Support.
Container-API
Custom settings are not picked up even after updating /opt/ibm/appliance/storage/head/db2_config/db.cfg.
WORKAROUND:
On IAS, the connect_proc parameter is used differently to other Db2 parameters.
You must manually apply connect_proc:
db2 update db cfg for <DATABASE> using connect_proc NULL
Do not add the connect_proc configuration parameter to the custom settings file (Preserving custom Db2 configuration settings between upgrades). This is a one time operation and is retained when you run the apstop or apstart commands.
start_cc_services
After a head node failure/failback, the system is waiting for the node recovery process to complete and is stuck in the Recovering state.
WORKAROUND:
Stop the start_cc_services child process.
Web console
  • When data is edited and saved multiple times in the web console's Call home page (Rules tab page), there are issues with saving drop down fields data. When the console detects the action set is the same as the default one for its type, the console sets to Default type action instead instead of action set.
AD
  • Switching using Multiple RDN in UI is not supported.
    WORKAROUND:
    1. You can switch from backend by running the configure_user_management command.
    2. To access the console after switching by using Multiple RDN, you must restart the web console by running ap apps restart WebConsole.
LDAP
Load file is failing when the system is switched to external LDAP and upgraded.

WORKAROUND:

  1. Run the command from the web console:
    rm -f /scratch/ldap_tmp/sssd.conf
    • If there is file /scratch/ldap_tmp/command.sh in web_console, run it. This resolves the issue.
    • If there is no /scratch/ldap_tmp/command.sh
      1. Copy /etc/sssd/sssd.conf from the dashDB container to web_console:/etc/sssd/sssd.conf.
      2. Run the command:
        systemctl restart sssd
Snapshot job is reported as Failed in the apdr status output
If an automated or a manual snapshot is running on the system and you run apstop -v and apstart -w, the snapshot job is reported as Failed in the apdr status output.
WORKAROUND:
  1. After running the apstart command, delete the .db_in_ws_mode file if it's present
    • cd /opt/ibm/appliance/storage/platform/afm-dr/config/
    • rm -rf .db_in_ws_mode
  2. Run the apdr status command.

    The correct output is displayed.

  3. If a manual snapshot was running previously, rerun the command to run a manual snapshot again. If an automated snapshot was running previously, it is taken whenever it is scheduled next.
Schema enabled for row modification tracking (RMT schema)
When you back up RMT schema tables with the db_backup command on IAS 1.0.21 and restore them to either IAS 1.0.22 or 1.0.23, the SYSROWID is not restarted properly.
IBM Data Replication for Availability
After upgrade, IBM Data Replication for Availability doesn’t work as database communication is broken due to new SSL certificates.

WORKAROUND:

To refresh the Db2 SSL certificate for IBM Data Replication for Availability:
  1. On the source system, in the Db2 container, as su-dsadm:
    1. Ensure that BLUDR_SHARED_DIR is set:
      • On IAS or Db2 Warehouse on Cloud:
        BLUDR_SHARED_DIR=/head/bludr
      • On Db2 Warehouse:
        BLUDR_SHARED_DIR=/mnt/blumeta0/bludr
    2. Import local Db2 SSL certificate to the IBM Data Replication for Availability truststore:
      1. /opt/ibm/java/jre/bin/keytool -delete -keystore ${BLUDR_SHARED_DIR}/certificates/cacerts -storepass "$(base64 -d ${BLUDR_SHARED_DIR}/certificates/truststore_pwd)" -alias db2_ssl -noprompt
      2. /opt/ibm/java/jre/bin/keytool -import -trustcacerts -keystore ${BLUDR_SHARED_DIR}/certificates/cacerts -storepass "$(base64 -d ${BLUDR_SHARED_DIR}/certificates/truststore_pwd)" -alias db2_ssl -noprompt -import -file /mnt/blumeta0/db2/ssl_keystore/rootCA.pem
    3. Import the target Db2 SSL certificate to the IBM Data Replication for Availability truststore.

      The certificate is available at /mnt/blumeta0/db2/ssl_keystore/rootCA.pem inside the database container. You can extract the self-signed root CA certificate to your system. For more information, see the Procedure section.

      1. /opt/ibm/java/jre/bin/keytool -delete -keystore ${BLUDR_SHARED_DIR}/certificates/cacerts -storepass "$(base64 -d ${BLUDR_SHARED_DIR}/certificates/truststore_pwd)" -alias db2_ssl_<target_hostname> -noprompt
      2. /opt/ibm/java/jre/bin/keytool -import -trustcacerts -keystore ${BLUDR_SHARED_DIR}/certificates/cacerts -storepass "$(base64 -d ${BLUDR_SHARED_DIR}/certificates/truststore_pwd)" -alias db2_ssl_<target_hostname> -noprompt -import -file ${BLUDR_SHARED_DIR}/certificates/<target_hostname>_rootCA.pem
    4. As the root or db2inst1 user, import the remote target Db2 root certificate to the local Db2 SSL keystore for Db2CLP:
      opt/ibm/db2/V11.5.0.0/gskit/bin/gsk8capicmd_64 -cert -add -db /mnt/blumeta0/db2/ssl_keystore/bludb_ssl.kdb -stashed -label db2_ssl_rootCA_<target_hostname> -file ${BLUDR_SHARED_DIR}/certificates/<target_hostname>_rootCA.pem -format ascii -trust enable -fips
    5. Verify the Db2 certificate aliases in the IBM Data Replication for Availability truststore :
      keytool -v -list -keystore ${BLUDR_SHARED_DIR}/certificates/cacerts  -storepass "$(base64 -d ${BLUDR_SHARED_DIR}/certificates/truststore_pwd)"  | grep Alias -A10 | grep db2 -A10
    6. Restart the IBM Data Replication for Availability server and web console.

      Run the restart commands as the dsadm user:

      1. The IBM Data Replication for Availability server:
        /opt/ibm/bludr/scripts/bin/bludr-restart.sh
      2. The web console:
        • On Db2 Warehouse:
          /opt/ibm/dsserver/bin/restart.sh 
        • On IAS:
          • Restart the API server:
            systemctl restart apiserver
          • From the IAS console, on the platform
            ap apps disable webconsole
            ap apps enable webconsole 
  2. On the target system, do the same steps as for the source system. Import the source Db2 SSL certificate and local Db2 SSL certificate to the IBM Data Replication for Availability truststore, and import the remote source Db2 root certificate for Db2CLP.
db_migrate_iias
After upgrade, the db_migrate_iias command doesn’t work with ssl.

WORKAROUND:

Run the commands from the source or target machine from which you want to invoke db_migrate_iias.
  1. Extract the self-signed root CA certificate from the other system.

    For more information, see 4. Extract the certificate.

  2. Add the extracted certificate to the db2wh default truststore inside the container:
    [db2inst1@ma-bhpwr5-8 - Db2wh ~]$ /opt/ibm/db2/V11.5.0.0/gskit/bin/gsk8capicmd_64 -cert -add -db /mnt/blumeta0/db2/ssl_keystore/bludb_ssl.kdb -stashed -label ss_bluhel21 -file <fully_qualified_path_to_the_CA_cert_copied>  -format ascii -trust enable -fips
dbload and dbsql
After upgrade, the dbload and dbsql commands don’t work with ssl from clients.

WORKAROUND:

Set up the client to trust the certificate as described in Secure Socket Layer (SSL) support. There's no need to catalog the database.
db_restore
After upgrade, while you're running the db_restore command from web_console, a 'Database restore failed on the web console' error appears. The restore succeeds in backend.

WORKAROUND:

To validate the progress of the restore operation, as bluadmin, run the db2 list utilities show detail command to determine if a restore is running.

If a restore is in progress, the example output for each MLN is:
Restore percentage output:
ID = 1
Type = RESTORE
Database Name = BLUDB
Member Number = 1
Description = db
Start Time = 09/02/2020 09:57:33.923808
State = Executing
Invocation Type = User
Progress Monitoring:
Completed Work = 922976256 bytes
Start Time = 09/02/2020 09:57:33.923813

ID = 1
Type = RESTORE
Database Name = BLUDB
Member Number = 2
Description = db
Start Time = 09/02/2020 09:57:33.904257
State = Executing
Invocation Type = User
Progress Monitoring:
Completed Work = 922976256 bytes
Start Time = 09/02/2020 09:57:33.904264
You can find a log file associated with this restore operation in the /opt/ibm/appliance/storage/scratch/bluadmin_BNR/logs directory directory. Review the logs in the most recent restore log and tracelog for progress details.
db_backup, db_restore
Restore using TSM fails due to: An I/O error occurred. Error code: "-50". Media on which this error.
This error indicates that during restore another activity on the TSM server has consumed all available space. When this failure occurs during a restore, contact IBM Support for recovery steps before you can continue.

WORKAROUND:

Check if there's enough space before you run a backup on the TSM server. Refer to TSM administrators guide for Monitoring the space available in storage pools. It is important to ensure that client operations such as backup and restore can complete successfully.

Resolved issues

  • Memsize issues

    Solved the issue with the IPMI commands taking more time to complete as compared to the typical execution time. Because of this, the session was timing out and an incomplete picture of the memory that was present was logged in. The timeout is extended to 5 minutes now.

  • GPFS

    mmhealth node show used to show a non-optimal GPFS configuration. It is now set to 0. On a running machine, a GPFS restart is needed to enforce the GPFS change. Also, the value persists after GPFS reboot/restart.