Upgrading and finalizing Big SQL

As a step in an IOP to HDP Big SQL migration, you must upgrade and finalize your version of Big SQL.

Before you begin

If you are performing this step as part of a migration, you must follow all the preceding steps described in Migrating from IOP to HDP.

If you are upgrading Big SQL in a non-root installation environment, you must follow the steps in Configuring Ambari agents for non-root access to IBM Big SQL.

The following list describes prerequisites and related information that are required before you can use the Big SQL upgrade script.

  • The Apache Ambari administrator user name and password.
  • If Kerberos security is enabled on the cluster, the Kerberos administrator credentials (principal and password).
  • A user must have the following attributes:
    • Passwordless sudo access on all nodes of the cluster, including the Ambari server itself.
    • The ability to connect passwordlessly through ssh from the Ambari server to all Big SQL nodes.

    This user can be root. If the user is not root, the user name must be passed to the upgrade script with the -a option. The upgrade script must be run with root user privilege. This privilege can be achieved by using the sudo command.

  • The upgrade process has the following disk space requirements:
    • 4 GB at the root of the file system (/)
    • 2 GB in the /usr/ibmpacks directory

    This space is used temporarily by the upgrade process. It is released when the upgrade is complete.

  • The HDFS, Hive, and HBase services, and the services that they depend on, must be up and running in the Apache Ambari server. HDFS should not be running in safe mode.

    If these services are not running, the patch management prerequisite checker warns you and stops the patch process.

  • Apache Ambari configuration groups are not supported by the patch management process. If configuration groups are present, the patch management prerequisite checker warns you and stops the installation process. It is suggested that you disable and remove configuration groups before you initiate the patch management process.
  • The Big SQL High Availability (HA) feature must be disabled on the cluster, and there must be only one head node for the installation to proceed. If HA is enabled, the patch management prerequisite checker warns you and stops the installation process. In addition, YARN and Slider must also be disabled.
  • The Big SQL service must not be in maintenance mode.

Procedure

  1. Use the Upgrade option of the bigsql_upgrade.py script to upgrade your version of Big SQL.
    The upgrade phase upgrades the Big SQL catalog, metadata and configuration information to the new version of Big SQL. To perform the upgrade option of the Big SQL upgrade, run the bigsql_upgrade.py script with the -m option and the value Upgrade. Include any additional options as documented in bigsql_upgrade.py - Big SQL upgrade utility. For example, if you have configured Ambari for non-root access, you should use the -a option. 
    python /usr/ibmpacks/scripts/5.0.2.0/upgrade/bigsql_upgrade.py -m Upgrade

    When the Upgrade phase is complete, the Big SQL service is not visible in the Ambari dashboard. However, the new version of Big SQL is operational and running. It is possible to connect applications to the Big SQL server to run sanity tests before proceeding with the Finalize phase of the upgrade.

    It is not possible to re-run the upgrade phase immediately after it has completed (successfully or not).

  2. In case the previous upgrade step fails, follow these steps:
    1. Consult the script output or the upgrade log located at /var/ibm/bigsql/logs/upgrade.log on the Ambari server host to identify the problem.
    2. Use the Restore option of the bigsql_upgrade.py script to restore to pre-upgrade conditions. The restore phase returns the cluster to a state that allows you to re-run the upgrade phase, by restoring the backup taken in the backup phase.
      python /usr/ibmpacks/scripts/5.0.2.0/upgrade/bigsql_upgrade.py -m Restore
      When the Restore is complete, the Big SQL service is no longer visible in the Ambari dashboard. However, it is operational, but not running. If needed, you can start the service from the command line and use it. In this case, the version is the initial Big SQL version. After the restore phase is complete, proceed to run the upgrade phase again as described above. In case the restore phase fails, consult the script output or the upgrade log located at /var/ibm/bigsql/logs/upgrade.log to identify and resolve the problem. After it is resolved, re-run the restore phase
    3. Repair the upgrade issue that caused the failure.
    4. Re-run the upgrade command as shown in Step 1.
  3. Use the Finalize option of the bigsql_upgrade.py script to finalize your upgrade of Big SQL.
    CAUTION:
    After the upgrade is finalized, the backups of the catalog, metadata and configuration information of Big SQL are cleaned up and are no longer available.
    The finalize phase takes the following actions on all nodes of the cluster:
    1. Registers the Big SQL service in Ambari and applies configuration changes required by the Big SQL service.
    2. Cleans up the binaries of the previous Big SQL version.
    3. Cleans up the backups that were created during the backup phase.
    To perform the finalize phase of the Big SQL upgrade, run the bigsql_upgrade.py script with the -m option and the value Finalize. Include any additional options as documented in the bigsql_upgrade.py - Big SQL upgrade utility. For example, if you have configured Ambari for non-root access, you should use the -a option. 
    python /usr/ibmpacks/scripts/5.0.2.0/upgrade/bigsql_upgrade.py -m Finalize

    When the Finalize phase is complete, the backups no longer exist. The Big SQL service is visible in the Ambari dashboard. The new version of Big SQL is operational and running.

    In case of failure of the finalize phase, consult the script output or the upgrade log located at /var/ibm/bigsql/logs/upgrade.log to identify and resolve the problem. After it is resolved, re-run the finalize phase.

  4. Navigate to Hive > Configs > Advanced > Advanced hive-env template and manually uncomment the following section (if required): 
    # Allow Hive to read Big SQL HBase tables 
if 
  [ -d "/usr/ibmpacks/current/bigsql/bigsql/lib/java" ]; then 
  export HIVE_AUX_JARS_PATH=\ /usr/ibmpacks/current/bigsql/bigsql/lib/java/biga-io.jar,\ 
  /usr/ibmpacks/current/bigsql/bigsql/lib/java/biga-hbase.jar,\ 
  /usr/ibmpacks/current/bigsql/bigsql/lib/java/commoncatalog.jar,\ 
  /usr/ibmpacks/current/bigsql/hive/lib/hive-hbase-handler.jar,\ 
  ${HIVE_AUX_JARS_PATH} 
fi
  5. (optional) Remove old configs that cause database check warnings.
    Note: This step is optional because the database check warnings do not affect the proper functioning of Ambari.
    1. Back up the Ambari server database.
    2. Log in to the Ambari server database shell prompt. Ambari can use various types of database management system (DBSM) that each has its own command to open a shell prompt. For the default PostgreSQL database, as the root user, run psql -U ambari -d ambari in a Linux bash shell. Enter bigdata when asked for a password.
    3. From the database shell prompt, run the following command (replacing ${type_name} and ${version_tag} with the types and versions displayed in the warning):
       DELETE FROM clusterconfig WHERE type_name='${type_name}' AND version_tag='${version_tag}';
      For example, if you see the following warning:
       2017-08-16 07:24:53,209  WARN - You have config(s): bigsql-users-env-version1501346872884, bigsql-head-env-version1502892246 that is(are) not mapped (in serviceconfigmapping table) to any service!
      This indicates two configs which are being flagged as a problem:
      bigsql-users-env-version1501346872884
bigsql-head-env-version1502892246
      In the Ambari database, you need to run two deletes, one for each configuration flagged as problematic. The ${type_name} parameter coincides with everything before the last "-" in the configuration. For example, in bigsql-users-env-version1501346872884, the ${type_name} is bigsql-users-env and the ${version_tag} is version1501346872884. Therefore your delete statement is:
      DELETE FROM clusterconfig WHERE type_name='bigsql-users-env' AND version_tag='version1501346872884';
      For the Configs bigsql-head-env-version1502892246, the delete statement is:
      DELETE FROM clusterconfig WHERE type_name='bigsql-head-env' AND version_tag='version1502892246';
    4. Exit the database shell prompt. If you are using the default PostgreSQL, type \q and hit Enter to exit.
    5. Restart the Ambari server.
  6. If you have previously granted execute privileges on the hcat_cache_sync and hcat_sync_objects routines to a specific set of users (or revoked execute from public on these routines), then reapply these custom privileges after the upgrade is complete.

    During the upgrade of Big SQL, the privileges on these procedures are reset to the defaults.

What to do next

The next step in the migration process is to upgrade DSM. See Upgrading IBM Data Server Manager (DSM) for details.