Fixes are available
IBM Spectrum Control V5.2.8 (December 2015)
IBM Spectrum Control V5.2.9 (February 2016)
IBM Spectrum Control V5.2.10 (May 2016)
IBM Spectrum Control V5.2.10.1 (July 2016)
IBM Spectrum Control V5.2.11 (August 2016)
IBM Spectrum Control V5.2.12 (November 2016)
IBM Spectrum Control V5.2.13 (March 2017)
IBM Spectrum Control V5.2.14 (May 2017)
IBM Spectrum Control V5.2.15 (August 2017)
IBM Spectrum Control V5.2.15.2 (November 2017)
IBM Spectrum Control V5.2.16 (March 2018)
IBM Spectrum Control V5.2.17 (May 2018)
APAR status
Closed as program error.
Error description
An internal timer thread utilized by TPC-R code crashes without detection. When invoking certain operations (such as "Refresh Config"), the following errors appear in the csmTrace logs: '''''''''''''''' 2015-08-18 16:06:49.044+1000 CSMCOM-E28B RepMgr D com.ibm.csm.server.hw.ess.ccw.ESSElementCatalogPopulator refresh() TRACE: (DS8000:BOX:xxxx.xxxxx) Element Catalog Populator Refresh has been issued.; isWorking=false 2015-08-18 16:06:49.044+1000 CSMCOM-E28B RepMgr D com.ibm.csm.util.thread.SchedulerService cancel TRACE: looking for com.ibm.csm.server.hw.ess.ccw.ESSElementCatalogPopulator@7b83b93 e [DS8000:BOX:xxxx.xxxxx] and found task=com.ibm.csm.server.hw.ess.ccw.ESSElementCatalogPopulator@7b 83b93e [DS8000:BOX:xxxx.xxxxx] 2015-08-18 16:06:49.044+1000 CSMCOM-E28B RepMgr E e com.ibm.csm.server.communication.ClientRequest processCommandParcel java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [...] at java.lang.Thread.run(Thread.java:804) Caused by: java.lang.IllegalStateException: Timer already cancelled. at java.util.Timer.sched(Timer.java:408) at java.util.Timer.schedule(Timer.java:219) [...] 2015-08-18 16:06:49.044+1000 CSMCOM-E28B RepMgr I com.ibm.csm.server.communication.ClientRequest processCommandParcel IWNR2999E [18/08/2015 4:06:49 PM] A Tivoli Storage Productivity Center for Replication internal error occurred. '''''''''''''''' In this condition, TPC-R results in error IWNR2999E "internal error occurred" when performing several operations such as: - Refresh Config - HA disconnect/reconnects
Local fix
restart TPC-R service.
Problem summary
**************************************************************** * USERS AFFECTED: * * Users of the product who experience a crash of the Timer * * thread which is used by all pollers, loggers, work tasks, * * and other functions that require schedule. * **************************************************************** * PROBLEM DESCRIPTION: * * | fix pack | 5.2.8-TIV-TPC-FP0000 - target 4Q 2015 | * * * * * * http://www-01.ibm.com/support/docview.wss?&uid=swg21320822 * * * * The target dates for future fix packs do not represent a * * formal * * commitment by IBM. The dates are subject to change without * * notice. * * * * This apar addresses an unexpected crash of the Timer thread * * that is used by many functions in the TPC for Replication * * product. The thread crash will cause pollers, loggers, work * * tasks, and other functions that require schedule not to be * * able to be scheduled. Some of the problems that can occur if * * the thread is not running will not be seen directly but may * * manifest in problems with running commands, session may get * * stuck in preparing, or the storage / copyservice changes * * caused by External failover may not work properly. The * * particular problem that this surfaced with was an internal * * error while running the refresh config. Although outOfMemory * * issues are very rare, this maintenance should be applied as * * soon as possible to avoid future problems. Customers should * * ensure that the TPC server has the recommended memory * * http://www-01.ibm.com/support/docview.wss?uid=swg27039550 * * and amount of attached storage and copysets meets the heap * * memory recommendation described in the infocenter * * http://www-01.ibm.com/support/knowledgecenter/SSU9FR_5.2.7/c * * om.ibm.tpcrz.doc_5.2.7/frc_r_verifying_the_java_heap_size.ht * * ml?lang=en * **************************************************************** * RECOMMENDATION: * * Ensure that the server meets the requirements listed * * previously. * ****************************************************************
Problem conclusion
The fix will detect the condition in which the timer thread has crashed and restart it.
Temporary fix
Restart the TPC-R server.
Comments
APAR Information
APAR number
IT11056
Reported component name
TPC
Reported component ID
5608TPC00
Reported release
521
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt
Submitted date
2015-09-03
Closed date
2015-10-09
Last modified date
2015-10-09
APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:
Fix information
Fixed component name
TPC
Fixed component ID
5608TPC00
Applicable component levels
R520 PSY
UP
[{"Line of Business":{"code":"LOB26","label":"Storage"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SS5R93","label":"IBM Spectrum Control"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"521"}]
Document Information
Modified date:
22 February 2022