IBM Support

Known issues while installing an Information Server 11.7.1.0 Service Pack or Fix Pack

Troubleshooting


Problem

Before installing an Information Server 11.7.1.0 Service Pack or Fix Pack, you should ensure that the systems meets the requirements for a successful install. In some cases, this may involve changing the current configuration of the system.

This document covers some of the known Install and Post-installation issues that may be encountered during or after the install.
Note: Information for each issue is listed in the same order, in each of the three sections: Symptom, Cause, and Resolving the problem.
In many cases, you can avoid the "During the Install" issues by doing the Resolution action prior to starting the install.

Symptom

During the Install:

In many cases, you can avoid these issues by doing the Resolution action prior to starting the install.

  1. Error related to host name resolution
    [WARNING FileExisting-crictl]: crictl not found in system path
    See Technote for details.
  2. Upgrade failures due to limited free disk space reported as DiskPressure:
    Sep 27 14:57:36 CO9020091182 kubelet: W0927 14:57:36.204160   13593 eviction_manager.go:142] Failed to admit pod kube-proxy-djsx3_kube-system(e81c5638-c254-11e8-b444-005056941056) - node has conditions: [DiskPressure]
    See Technote for details.
  3. Information Server Enterprise Search (ISES) fails because the Docker service encountered problems during startup. See Technote for details. This issue is also applicable to releases prior to 11.7.0.2.
  4. After ISES 11.7.1.0 installation, or upgrade to 11.7.1.0, some pods are in CrashLoopBackOff state and examining the logs of such pods indicates an error
    "Unable to obtain connection from database (jdbc:db2://iis-db:50000/xmeta) for user 'xmeta': [jcc][t4][10380][11951][4.23.42] Required property "iis-db" is unknown host. ERRORCODE=-4222, SQLSTATE=08001"
    See Technote for details.
  5. Suite Installer and Update Installer GUI panels intermittently exhibit rendering issues.
  6. The Microservices tier pre-requirements checker does not report inadequate free disk space.
  7. Upgrades of the Microservices tier should be run by the same user that initially installed the Microservices tier.
    The Update Installer checks that the specified user has root privileges and access rights for Kubernetes and the Microservices tier install location.
    Sometimes, when root is used to upgrade the Microservices tier using Update Installer version 11.7.1.34 or earlier, you may get the following error even if those conditions are met:
          "Since the root user was used to install the Microservices tier, the current user (root) must have root privileges to patch the installation.".
  8. Note: If you use Update Installer 11.7.1.34 or earlier, you may also encounter this issue while running as root due to issue #7 above that requires sudo to be setup for root users too.
    If a non-root user is used to install/upgrade the Microservices tier, in the /etc/sudoers file, there is a "requiretty" setting that can cause the following messages to appear in the kube deployment log:
                  a. A "sudo: sorry, you must have a tty to run sudo" message when sudo is invoked.
                  b. While "yum install zip" is running, you may get a message
                            "ERROR: With yum install of zip. You can try to install requirement manually"
                       This error can appear even if zip is installed as needed.
  9. If the Watson Knowledge Catalog Professional plan was installed during the install of Information Server 11.7.1.0 then while installing an Information Server 11.7.1.0 Service Pack, the Update Installer GUI panel will display a patch validation error. See Technote for details.
  10. If the microservices tier installation fails with below mentioned error when installing IIS and microservices from Windows server with SSH passwordless authentication mode.
    "ssh: Could not resolve hostname C Name or service not known"
  11. A NumberFormatException is intermittently reported in a number of install scenarios, using various browsers and browser versions:
    java.lang.NumberFormatException: For input string: "50.400001525878906"
    at java.lang.NumberFormatException.forInputString(NumberFormatException.java:76)
  12. Applying 11.7.1.1 to an installation that has a Microservices tier, and uses WebSphere Application Server Network deployment (WAS ND) intermittently fails after upgrade of the Microservices tier indicates "Success".
     
    The install log contains the following error messages:
    020-05-14T01:37:37.845, INFO: CDIIN2951I: Action: PkgScript File: /opt/IBM/InformationServer/UGDockerDeployment/bin/LaunchUGDeployment.sh lin-test0716-int.fyre.ibm.com root false false false 1 ... Success
    2020-05-14T01:37:37.845, INFO: Stopping the Application Server
    .....
    Server stop request issued. Waiting for stop status.
    ADMU3111E: Server stop requested but failed to complete.
    ADMU0111E: Program exiting with error:
    com.ibm.websphere.management.exception.AdminException: ADMU3060E:
    Timed out waiting for server shutdown.
    ....
    2020-05-14T01:58:59.846, SEVERE: com.ibm.is.install.update.installer.Updater^M
    java.lang.Exception: CDIIN4559E: Could not stop WebSphere Application Server profile: /opt/IBM/WebSphere/AppServer/profiles/InfoSphere.
     
  13. Applying 11.7.1.1 to an installation that has a Microservices tier, and uses WebSphere Application Server Network deployment (WAS ND) intermittently fails during upgrade of the Microservices tier.
     
    The install log contains the following error messages:
    RUNNING HANDLER [com/ibm/ugi/iis/iis-external : Ensure IIS classic launchpad is accessible] ***
    Wednesday 13 May 2020 02:32:57 -0700 (0:00:06.749) 0:46:21.769 *********
    FAILED - RETRYING: Ensure IIS classic launchpad is accessible (10 retries left).
    .....
    fatal: [deployment_coordinator]: FAILED! => {"attempts": 10, "changed": false, "content": "", "elapsed": 30, "msg": "Status code was -1 and not [200]: Connection failure: ('The read operation timed out',)",
    "redirected": false, "status": -1, "url": "https://11711-msfs-int.fyre.ibm.com/ibm/iis/launchpad/"}
    ....
    com/ibm/ugi/iis/iis-external : Ensure IIS classic launchpad is accessible - 386.73s
    Load all container images --------------------------------------------- 258.11s
    com/ibm/ugi/kubeplatform/kubernetes : Wait for Cluster DNS service to become available - 191.92s
  14. In case of Microservices Tier deployment (install/upgrade) NO special character is supported for IIS users and passwords like isadmin, xmeta, dsadm, Kafka SASL, Solr etc.

  15. Applying 11.7.1.1 to an installation that has a Microservices tier, and uses WebSphere liberty (LWAS) intermittently fails after install/upgrade of the Microservices tier indicates "Success".

    Exception:
    Either The install log contains the following error messages:
    server stop executed successfully.
    Its failed to start the server.its taking more than 60 secs to start the LWAS server.

    InstallAction (_id = start.lwas.on.windows.ug, _installUnitModelId = UGDockerDeploymentServices)
    _action = START
    _serviceName = InfoSvr
    _seconds = 60
    _numberOfRetries = 1
    )
    OR Below exception
    C:\Users\Administrator\AppData\Local\Temp\2\ibm_is_temp.2020.06.22.09.51.06\scripts\service.js start InfoSvr 60 1, env = null, workingDir =null
    Jun 22, 2020 1:13:29 PM com.ibm.is.install.core.CommonProcess
    INFO: Exit Code: 1, Info: action = start
    serviceName = InfoSvr
    Service state for InfoSvr after attempt number 1 is Start Pending
    Unable to start service InfoSvr, service state = Start Pending
    Jun 22, 2020 1:13:29 PM com.ibm.is.install.core.SuiteLogger logError
    SEVERE: com.ibm.is.install.engine.runtime.InstallEngine
    com.ibm.is.install.exception.ActionException: CDIIN5123E: WindowsServiceStopStartAction failed: the process returned error code 1.:CDIIN5106I: Details:
    WindowsServiceStopStartAction (CDIIN5106I: Details:
    InstallAction (_id = start.lwas.on.windows.ug, _installUnitModelId = UGDockerDeploymentServices)
    _action = START
    _serviceName = InfoSvr
    _seconds = 60
    _numberOfRetries = 1
    )

  16. Intermittent Weave network issue while upgrading  11.7.1.0  or 11.7.1.1  installations on RHEL 7 to version 11.7.1.2 on RHEL 8 may result in multiple pods getting into an error state.
    a. Locate Weave pod name in the output of
               kubectl get pods -n kube-system
    b. Display Weave container logs
        kubectl logs -n kube-system <weave_pod_name> -c weave
    c. The problem is present if the logs contain the message:
              ipset v7.2: Set cannot be destroyed: it is in use by a kernel component

                                                                           

Post-installation issues:

  1. Login to Information Server launchpad applications using the "Automatically log in" checkbox fails even when valid login credentials are provided. You may see the following in the WebSphere log: "The Response Message is: Error 500: java.lang.RuntimeException: Message is larger than modulus".
  2. Microservices tier installation needs to be cleaned up, and re-installed.
  3. Uninstall of Information Server 11.7.1.0 on Windows does not cleanup Db2. In C:\Windows\System32\drivers\etc the Db2 ports are still listed. The Db2 services are still running and do not shut down.
  4. Clicking the Administration console and Metadata Asset manager icons on the Information Server microservices launchpad does not go through the HTTP server, and is redirected to Node 1 of the WebSphere Network Deployment cluster.
  5. From the InfoSphere Information Server Launchpad, trying to launch Information Governance Catalog(IGC) via the IGC icon returns the error:

    Error 404: com.ibm.ws.webcontainer.servlet.exception.NoTargetForURIException: No target servlet configured for uri: /ibm/iis/igc
    In the Websphere Application Server(WAS) Console, the following applications are 'stopped':
         igc-web, igc-rest, igc-rest-explorer

  6. Unable to login to the Information Server Administration (web) console after installing 11.7.1 Fix Pack 1.

Cause

During the Install:

  1. The host name in the /etc/hostname file, and/or relevant entries within the uginfo.rsp file was not specified as the fully qualified domain name (FQDN).
  2. When disk space usage is above the maximum threshold, Kubernetes garbage collection can trigger removal of various artifacts which in turn can cause pod failures.
  3. Docker keeps configuration flags and storage configuration options in multiple places. When the values specified are conflicting, it causes a problem during startup. See Technote for details.
  4. Kubernetes receives the short name when retrieving the host name of the XMETA database.
  5. The release of Firefox used to run the install.
  6. The default disk space requirements on the Microservices tier for User data, Docker images, and installation directory is 600GB. The pre-requirements checker checks available space for each of these but does not account for them being located on the same partition.
  7. Update Installer has incorrectly coded a check for the user that initially installed the Micro services tier.
  8. A "requiretty" setting for sudo actions causes sudo execution during the install to fail.
  9. Installation of the initial release of Information Server 11.7.1.0 with Watson Knowledge Catalog Professional plan updates the Version.xml file with an invalid version format. The respin release of Information Server 11.7.1.0 does not have this problem.
  10. The problem is with the scp command on this Windows environment. scp command expect 'C' to be the host name (as it is before the :) and scp didn't manage to locate hostname.      So Installer is unable to copy the following files from IIS Windows server to microservices tier master host  uginfo.rsp, kubeselfextract.bsx, buildinfo.txt, and tokenservicepublic.cer
  11. The cause of this failure is not fully understood. It is affected by the version of Windows and the browser used.
  12. The WAS ND server takes a long time to stop and the resulting timeout error results in the Installer stopping.
  13. The WAS ND server takes a long time to stop and the resulting timeout error results in the Installer stopping.
  14. Customers might experience installation issues,if they do not follow the documented requirements while creating application users. Customers must follow the listed standards for passwords to avoid installation failures.
  15. The LWAS server takes a long time to stop and the resulting the server start error causing  the Installer action to fail.
  16. Behavioral differences of the Weave network between RHEL 7 and RHEL 8.


Post-installation issues:

  1. Incorrect information in the repository causes the "Automatically log in" setting to be interpreted incorrectly.
  2. Cleanup of the Microservices tier master node host may be needed for any reason.
  3. Db2 appears to be busy and does not shutdown.
  4. The yaml file configuration is not configured with the HTTP server host name.

  5. This is a known issue when using WebSphere Application Server 9.0.0.4 since the Update Installer deletes the cache directory.
    This issue is often encountered when using the Update Installer to apply Service Pack 1 for 11.7.1,  since the installer redeploys the war files.

  6. Erroneous coding results in the session not receiving the information it needed.

Environment

Information Server 11.7.1 installation. Some issues may be platform specific.

Diagnosing The Problem

Examine logs or system behavior for the problems listed above and note the corresponding symptoms.
For some issues, further details may be found in associated Technotes mentioned below.

Resolving The Problem

During the Install:

  1. Prior to installation, ensure that the host name in the /etc/hostname file, and relevant entries within the uginfo.rsp file are specified as the fully qualified domain name (FQDN). See Technote for details.
    Note that in version 11.7.1.1 the restriction is relaxed; /etc/hostname or hostnamectl may use either a short hostname or an FQDN.
     
  2. Ensure that adequate free space is available on the system. For details on diagnosing and resolving the problem, see Technote.
     
  3. Configure Docker to have non-conflicting settings. For details on diagnosing and resolving the problem, see Technote.
     
  4. The iis_db.yaml file must be configured to specify the long host name for the XMETA database. See Technote for details of the steps to be done.
     
  5. Install specific releases of Firefox; see Technote for details.
     
  6. Prior to installation, ensure that there is at least 300 GB available in the Persistent Storage location, 200 GB in /var/lib/docker, and 100 GB in the installation directory. If two or more of these locations are on the same partition the free space required should be summated.
     
  7. Enable NOPASSWD for the root user. Use visudo to append " ALL=NOPASSWD: ALL" to root's entry in the /etc/sudoers file, or add a new entry for root.
                The resulting line should be
                        root ALL=(ALL) NOPASSWD:ALL
     
  8. You will have to disable "requiretty" globally or at a user/group/command level:
            Replace "Defaults requiretty" with "Defaults !requiretty" in your /etc/sudoers file. This will impact your global sudo configuration.
            Alternatively, you can change this configuration at a per user, per group or per command basis. For example,
                    Defaults!/path/to/my/bin !requiretty
                    Defaults:myuser !requiretty
     
  9.  Use the respin release of Information Server 11.7.1.0 if you are installing Information Server with Watson Knowledge Catalog Professional plan. This will prevent the issue from occurring.
    If you have already installed Information Server and used the initial release image, perform the following steps.
     Take a backup of the Version.xml file located in your Information Server installation folder.    
     Change the following entry in the Version.xml file:
             <Product productId=“WKCPro” version=“11.7.1"/>
                      to
             <Product productId=“WKCPro” version=“11.7.1.0"/>
    You can now start the Update Installer to install the Service Pack.

    See Technote for details.

10. Should  copy files from Windows host to microservices host with SCP command as below(removing C: from the path)

scp \IBM\InformationServer\UGDockerDeployment\installer\uginfo.rsp root@<microservices_hostname>:/tmp/uginfo.rsp
Continue and complete the IIS installation. And then modify paths in <IIS_INSTALL_HOME>\UGDockerDeployment\bin\LaunchUGDockerDeployment.bat file and
execute with the appropriate parameters as mentioned in the LaunchUGDockerDeployment.bat
11. There are several workarounds for this problem. See Technote for details.

12. Stop the server:
                 /opt/IBM/InformationServer/ASBServer/bin/MetadataServer.sh stop
       After the server has stopped, use the installer's "continue" option.
       On continuing, the installer will start the server, and the upgrade will complete as expected.
 
13.  Check whether the log file has the errors mentioned.
        Do NOT exit from the installer.
        Start the server:
                   /opt/IBM/InformationServer/ASBServer/bin/MetadataServer.sh run
        Verify that the IIS launchpad is working.
        If the launchpad is working, choose the "retry" option of the installer; upgrade of the Microservice tier will resume from the point where it had stopped.
        If the launchpad is not working, contact IBM customer support.
14. In case of Microservices Tier deployment (install/upgrade) NO special character is supported for IIS users and passwords like isadmin, xmeta, dsadm,Kafka SASL, Solr etc.
15. After the server has started, use the installer's "retry" option.
On retrying, the installer will start the server, and the upgrade will complete as expected.
16. To bring the pods back up you can apply a workaround by running the following command:
             kubectl patch ds -n kube-system weave-net --patch '{"spec":{"template":{"spec":{"containers":[{"name":"weave","command":["/bin/sh","-c","sed '"'"'/ipset destroy weave-kube-test$/ i sleep 1'"'"' /home/weave/launch.sh | /bin/sh"]}]}}}}'
        Wait for a few minutes for pods to come back up.


Post-installation issues:

  1. See Technote for details of resolving the problem related to usage of Automatic login.
     
  2. To cleanup a Microservices tier master node installation, execute
          /opt/IBM/UGinstall/cleanUGInstall.sh /tmp/uginfo.rsp
     
  3. Examine C:\Windows\System32\drivers\etc. Check whether DB2 services are running.
    From your is-suite folder, uninstall Db2 using the -force option and reboot the system.
     
  4. Perform the following steps:
            a. login to Microservices tier master node
            b. Edit <Install Location>/manifests/ug_services/iis_svc.yaml
            c. Search for <node1> host name and replace it with HTTP server hostname in all locations
            d. kubectl apply -f iis_svc.yaml
     
  5. Upgrade WebSphere Application Server (WAS) to version 9.0.0.8 or higher. See Technote for details on diagnosing and resolving the issue, and alternative mitigation steps.
     

  6. Clear the browser cache and cookies and again try to login to the Administration console.

Document Location

Worldwide

[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSZJPZ","label":"IBM InfoSphere Information Server"},"Component":"","Platform":[{"code":"PF002","label":"AIX"},{"code":"PF016","label":"Linux"},{"code":"PF033","label":"Windows"}],"Version":"11.7.1.0","Edition":"","Line of Business":{"code":"LOB10","label":"Data and AI"}}]

Document Information

Modified date:
05 October 2020

UID

ibm10888305

Page Feedback