Body

IBM Tivoli Composite Application Manager for Transactions
~ How to do stuff ~
By Kevin Baldwin -

How to.....

Identify and fix ITM Framework and GSKit related issues

This blog describes problems sometimes seen after installing ITM and/or ITCAMfT components and provides information to help resolve those problems.

In most cases, the underlying cause is 32-bit versions of the ITM Framework and/or GSKit not being the same as the 64-bit versions.

Check these examples which describe the type of problem that might be experienced to see if the links provided in the "Useful information and links" section might apply to you.

In all cases, the problem was overcome by upgrading 32-bit ITM framework which in some cases was needed to also upgrade 32-bit GSKit.
 

• Error installing Web Response Time agent
  
  We receive the following error message that the libgsk7sys.so library is missing:
  
  "KDY1008E The agent action SETCONFIG failed with a return code of 4 for product code t5.
  The command /opt/IBM/ITM/tmaitm6/lx8266/bin/kdy_xa -p -pc t5 produced the following error text: CandleExecute        : installer level 630 / 100.
  CandleExecute        :  running lx8266 jre.
  /opt/IBM/ITM/lx8266/t5/bin/t5_xa.bin: error while loading shared libraries: libgsk7sys.so: canno."
  
  cinfo output from the WRT sgent:
  ax   IBM Tivoli Monitoring Shared Libraries   li6263 06.23.05.00
  ax   IBM Tivoli Monitoring Shared Libraries   lx8266 06.30.03.00
  gs   IBM GSKit Security Interface             lx8266 08.00.50.20
   
• WRT Agent won't start
  
  Error in <hostname>_t5_*.log:
  exec(): 0509-036 Cannot load program kt5agent because of the following errors:
  0509-022 Cannot load module
  /opt/IBM/ITM/tmaitm6/aix533/lib/libkglbase.a.
  0509-150   Dependent module libgsk7cms.so could not be loaded.
  0509-022 Cannot load module libgsk7cms.so.
  0509-026 System error: A file or directory in the path name does not exist.
  0509-022 Cannot load module kt5agent.
  0509-150   Dependent module /opt/IBM/ITM/tmaitm6/aix533/lib/libkglbase.a could not be loaded.
  0509-022 Cannot load module .
  
  This occurs because there is a 6.23 framework, but GSKit has been updated to version 8:
  
  ax   IBM Tivoli Monitoring Shared Libraries  aix523  06.30.02.00
  ax   IBM Tivoli Monitoring Shared Libraries  aix526  06.30.06.00
  ax   IBM Tivoli Monitoring Shared Libraries  aix533  06.23.05.00
  ax   IBM Tivoli Monitoring Shared Libraries  aix536  06.23.05.00
  gs   IBM GSKit Security Interface            aix523  08.00.50.05
  gs   IBM GSKit Security Interface            aix526  08.00.50.36
   
• No WRT/T5 data in the TEP
  
  The problem could be that the ITM infrastructure was upgraded, and now the WRT agent has a problem communicating with the TEMS.
  
  The cinfo shows that the Tivoli Framework used by the kt5agent process is at 6.23.5.0:
  
  ax IBM Tivoli Monitoring Shared Libraries
  li6263  Version: 06.23.05.00
  lx8266  Version: 06.30.04.00
  
  gs IBM GSKit Security Interface
  li6243  Version: 07.40.47.00
  lx8266  Version: 08.00.50.20
  
  This can cause problems with agent to TEMS communications.
   
• No ISM data in the TEP
  
  The ISM agent not reporting to the TEPS server.
  
  Information seen in the latest ISM agent logs show that the agent is failing to connect to the TEMS and GSKit errors:
  
  (2016/02/15,01:46:51.6001-11:kdebeal.c,81,"ssl_provider_open") GSKit error 420: GSK_ERROR_SOCKET_CLOSED
  (2016/02/15,01:46:51.6002-F:kdepdpc.c,62,"KDEP_DeletePCB") 485007E9: KDEP_pcb_t deleted
  (2016/02/15,01:46:51.6003-F:kdcc1sr.c,485,"rpc__sar") Connection failure: "ip.spipe:#111.22.33.444:3660", 1C010001:1DE00045, 0, 5(2), FFFF/1, D140831.1:1.1.1.13, tms_host:d3086a
  (2016/02/15,01:46:51.6004-F:kdcl0cl.c,142,"KDCL0_ClientLookup") status=1c020006, "location server unavailable", ncs/KDC1_STC_SERVER_UNAVAILABLE
  (2016/02/15,01:46:51.6005-F:kraarreg.cpp,1781,"LookupAndRegisterWithProxy") Unable to connect to broker at ip.spipe:host03.sysdomain.local: status=0, "success", ncs/KDC1_STC_OK
  (2016/02/15,01:46:51.6006-F:kraarreg.cpp,1801,"LookupAndRegisterWithProxy") Exit: 0x0
  (2016/02/15,01:46:51.6007-F:kraarreg.cpp,1599,"LookupAndRegisterWithProxy") Entry
  (2016/02/15,01:46:51.6008-F:kdepnpc.c,138,"KDEP_NewPCB") 111.22.33.444: 487007E8, KDEP_pcb_t @ 574018B8 created
  (2016/02/15,01:46:51.6009-12:kdebeal.c,81,"ssl_provider_open") GSKit error 420: GSK_ERROR_SOCKET_CLOSED
  (2016/02/15,01:46:51.600A-F:kdepdpc.c,62,"KDEP_DeletePCB") 487007E8: KDEP_pcb_t deleted
  (2016/02/15,01:46:51.600B-F:kdcc1sr.c,485,"rpc__sar") Connection failure: "ip.spipe:#111.22.33.444:3660", 1C010001:1DE00045, 0, 5(2), FFFF/2, D140831.1:1.1.1.13, tms_host:d3086a
  (2016/02/15,01:46:51.600C-F:kdcl0cl.c,142,"KDCL0_ClientLookup") status=1c020006, "location server unavailable", ncs/KDC1_STC_SERVER_UNAVAILABLE
  (2016/02/15,01:46:51.600D-F:kraarreg.cpp,1781,"LookupAndRegisterWithProxy") Unable to connect to broker at ip.spipe:host04.sysdomain.local: status=0, "success", ncs/KDC1_STC_OK
  (2016/02/15,01:46:51.600E-F:kraarreg.cpp,1801,"LookupAndRegisterWithProxy") Exit: 0x0
  (2016/02/15,01:46:51.600F-F:kraarreg.cpp,1453,"ConnectProxyUsingCMSLIST") Unable to find running CMS on CT_CMSLIST <ip.spipe:host03;ip.spipe:host04>
  (2016/02/15,01:46:51.6010-F:kraarreg.cpp,1461,"ConnectProxyUsingCMSLIST") Exit: 0x0
  (2016/02/15,01:46:51.6011-F:kraarreg.cpp,1121,"ConnectToProxy") CMS lookup: try #1 failed.
  (2016/02/15,01:46:51.6012-F:kraarreg.cpp,1123,"ConnectToProxy") Sleeping 600 seconds.
  (2016/02/15,01:46:51.6013-F:kraaulog.cpp,752,"IRA_OutputLogMsg") Active RAS1 Classes: EVERYT EVERYE EVERYU
  (2016/02/15,01:46:51.6014-F:kraaulog.cpp,755,"IRA_OutputLogMsg") CMS lookup failed.
   
• AMC / AMCE depot communication problems
  
  When attempts are made to launch the AMCE, the error message; "T3 depot agent could not be contacted" is displayed.
  
  There is a problem with GSkit in host02_t3*.log:
  
  (56AF8C62.0004-1:kglcry.c,2810,"initializeICC") ICC initialization: failed to load ICC DLL Library /opt/IBM/ITM/li6243/gs/lib/libgsk8iccs.so.
  
  Before that in the log, it shows that GSKit 7 libraries are loaded:
  +56AF8C60.0009 /opt/IBM/ITM/tmaitm6/li6263/lib/libgsk7iccs.so
  
  The next problem is this error:
  (56AF8CDA.000B-F:khdxdacl.cpp,598,"routeExportRequest") Export for object <AMC_Transaction> (table T3SNTRANS appl KT3) failed in createRouteRequest, Status = 8.
  
  This indicates that the T3 agent cannot communicate with the Warehouse Proxy.

Useful information and links:

Problems such as those described above have been seen when there are differences in version between 32-bit and 64-bit ITM Framework and/or GSKit libraries.

ITCAMfT agents are 32-bit so they use the 32-bit version of ITM Framework and when appropriate, the 32-bit version of GSKit. It's therefore important that the 32-bit version is at the same code level.
 

If any of the above examples look like the problem you're experiencing, even if it is for a different ITCAMfT agent, you may find the following links and information useful.
 

• Installing the 32-bit libraries is an ISM installation prerequisite as seen in the following link:
  
  ITCAMfT  7.4.0.1 > Installation and Configuration Guide > Planning your ITCAM for Transactions installation > Planning to install Internet Service Monitoring > Installation prerequisites
  
  Where it states:
  Tivoli Enterprise Management Agent Framework. If you are installing on 64-bit Windows, UNIX, or Linux systems with IBM Tivoli Monitoring V6.3.0.1 or later components installed, install the 32-bit Tivoli Enterprise Management Agent Framework from the IBM Tivoli Monitoring installation media before installing ITCAM for Transactions agents.
  
  Please note: This requirement is only documented for ISM but it should be a requirement for all ITCAMfT agents
   
• The "tacmd updateFramework" technote describes the IBM Tivoli Monitoring 'tacmd updateFramework' command which is mentioned several times in the following links
   
• Requirements for installing existing ITM agents into a new 6.3 environment
  This blog with help you with installing an existing ITM Agent into a new 6.3 environment.
   
• Upgrading Shared Components for IBM Tivoli Monitoring Agents
  Describes how to upgrade the shared components that are a part of IBM Tivoli Monitoring agents
   
• This technote describes how to upgrade the framework using the depot on the TEMS:
  IBM 32-bit agent logs error loading libgsk7sys.so
   
  If the appropriate fixpack files aren't in the depot you can use the OS agent installation media files to install the 32-bit Shared Libraries locally.
  
  The command to upgrade the Tivoli Framework on the local agent is:
  <MEDIA>/install.sh -h <CANDLEHOME> -q -p <MEDIA>/unix/tf<PLAT>.txt
  
  Here is an example, where the install media is unpacked in /tmp/6.3.0-TIV-ITM-FP0006:
  /tmp/6.3.0-TIV-ITM_TMV-Agents-FP0006/install.sh -h /opt/IBM/ITM -q -p /tmp/6.3.0-TIV-ITM_TMV-Agents-FP0006/unix/tflx8263.txt
  
  The following technote has information to help upgrade the framework using the Linux OS Agent install media:
  IBM ITCAM  T6 RRT Agent will not run with GSKit 8
  
  Despite this technote being published for the T6 agent the information also applies to other ITCAMfT agents.
  
   

Visit the ITCAM for Transactions ~ How to do stuff ~ Homepage for more blogs in this series
 

Subscribe and follow us for all the latest information directly on your social feeds: