APAR status
Closed as program error.
Error description
Customer observes a slow leak in his WebSphere controller region. It was found that there is a storage leak in the security code when there is an exception while handling a message in context CSIv2 message. The controller region was run for a length of time with the LE Runtime Options HEAPCHK(ON,,,,10),HEAPPOOLS(ON) Analysis of the resulting Heap Pool Trace for Heap Pool 1 showed two instances of Heap Pool 1 elements getting allocated and not freed, that had 0x00000007 in the first word: ================================================================ Timestamp: 2009/09/03 14:32:28.878538 Type: GET Cell Address: 30C0F220 Cpuid: 02 Tcb: 007BBE88 CALL NAME CALL ADDRESS CALL OFFSET CredBucket::getCredential(CredKey*,CredType) 1E2C1EB0 000003BE CSIv2MIC(ORB_Request*,CredKey*) 1E2887B0 000000AA SecurityManager::receiveInbound(ORB_Request*) 1E289798 0000575A ORB_Request::call_inbound_request_interceptors(unsigned char,unsigned char) 1EA681A8 0000005A ORB_Request::process_inbound_request(acrwObj*,GIOP_Message*) 1EA687B0 00000298 ORB_Request::comm_inbound_request(acrwObj*) 1EA66758 00000F24 ACR_ExecutionThread::ProcessInboundRequest(acrwObj*,ThreadCleanU p*,BOSS_Object_Key&,Interna... 1EAFB200 000001B8 ACR_ExecutionThread::RemoveAndProcessWork(ThreadCleanUp*) 1EB0A848 000006EA ACR_ExecutionRoutine 1EB0C5C0 0000052A CEEVROND 06BB9898 00000000 ================================================================ Timestamp: 2009/09/03 14:32:28.866843 Type: GET Cell Address: 30C10740 Cpuid: 00 Tcb: 007BBE88 CALL NAME CALL ADDRESS CALL OFFSET CredBucket::addCredential(CredKey*,CredType,long,void*,long long) 1E287790 00000434 SecurityManager::receiveInbound(ORB_Request*) 1E289798 00019038 ORB_Request::call_inbound_request_interceptors(unsigned char,unsigned char) 1EA681A8 0000005A ORB_Request::process_inbound_request(acrwObj*,GIOP_Message*) 1EA687B0 00000298 ORB_Request::comm_inbound_request(acrwObj*) 1EA66758 00000F24 ACR_ExecutionThread::ProcessInboundRequest(acrwObj*,ThreadCleanU p*,BOSS_Object_Key&,Interna... 1EAFB200 000001B8 ACR_ExecutionThread::RemoveAndProcessWork(ThreadCleanUp*) 1EB0A848 000006EA ACR_ExecutionRoutine 1EB0C5C0 0000052A CEEVROND 06BB9898 00000000 In an other instance of the problem, we saw the Control region run out of native memory and the held storage elements looked like below. Heappools are: ON HP# ElmtSize %Primary PoolSize PoolMeg #Pools TotalMeg 1 24 1 000CCCD0 0.8 132 105.6 ElementSize SizeInHex ElementCnt VirtualMeg 536870920 20000008 1 512.0 838864 000CCCD0 132 105.6 **** 7553032 00734008 6 43.2 36909616 02333230 1 35.2 262152 00040008 88 22.0 8388616 00800008 2 16.0 16777224 01000008 1 16.0 150520 00024BF8 106 15.2 16 00000010 676645 10.3 **** 9226672 008CC9B0 1 8.8 3355216 00333250 2 6.4 Reviewing the storage , we found that the instances of Heap Pool 1 elements getting allocated and not freed, that had 0x00000007 in the first word. 6E88E6A0 6E616000 000CCCD0 C5E7F3F1 6E7BA9C0 |na..........n{ 6E88E6B0 00000001 00000001 00000007* 00000000 |................ 6E88E6C0 49424D12 00000008 00000000 14200001 |IBM.......... 6E88E6D0 00000001 00000001 00000007* 00000000 |................ 6E88E6E0 00010100 D3D4D9C5 C3404040 00010000 |.........@@@.... 6E88E6F0 00000001 00000001 00000007* 00000000 |................ 6E88E700 53656E64 696E6743 6F6E7465 78742F43 |SendingContext/C
Local fix
Problem summary
**************************************************************** * USERS AFFECTED: All users of IBM WebSphere Application * * Server V6.0.1 * **************************************************************** * PROBLEM DESCRIPTION: For WebSphere Application Server for * * z/OS, there is a slow memory leak in * * the security code. The memory leak * * happens in Heap Pool 1 where the * * first word of the leaked storage * * contents is 0X00000007. * **************************************************************** * RECOMMENDATION: * **************************************************************** For WebSphere Application Server for z/OS, there is a memory leak in Heap Pool 1. The leak occurs when there is any error handling a CSIv2 message in context message for which the saved credential is no longer valid. When LE options are enabled as follows, HEAPCHK(ON,,,,10),HEAPPOOLS(ON) the trace produced is similar to Timestamp: 2009/09/03 14:32:28.878538 Type: GET Cell Address: 30C0F220 Cpuid: 02 Tcb: 007BBE88 CALL NAME CALL ADDRESS CALL OFFSET CredBucket::getCredential(CredKey*,CredType) 1E2C1EB0 000003BE CSIv2MIC(ORB_Request*,CredKey*) 1E2887B0 000000AA SecurityManager::receiveInbound(ORB_Request*) where it shows that storage is allocated due to the getCredential function. This storage is NOT released and its contents start with 0X00000007.
Problem conclusion
The security code was modified to correctly release storage allocated even when there is a failure handling CSIv2 message in context messages. APAR PK96881 is currently targeted for inclusion in Service Level (Fix Pack) 6.0.2.41 of WebSphere Application Server V6.0.1. Please refer to URL: //www.ibm.com/support/docview.wss?rs=404&uid=swg27006970 for Fix Pack availability.
Temporary fix
Comments
APAR Information
APAR number
PK96881
Reported component name
WEBSPHERE FOR Z
Reported component ID
5655I3500
Reported release
601
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt
Submitted date
2009-09-22
Closed date
2010-01-22
Last modified date
2010-05-04
APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:
Fix information
Fixed component name
WEBSPHERE FOR Z
Fixed component ID
5655I3500
Applicable component levels
R601 PSY UK55832
UP10/04/16 P F004
Fix is available
Select the PTF appropriate for your component level. You will be required to sign in. Distribution on physical media is not available in all countries.
[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SS7K4U","label":"WebSphere Application Server for z\/OS"},"Component":"","ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"6.0.1","Edition":"","Line of Business":{"code":"LOB45","label":"Automation"}}]
Document Information
Modified date:
10 February 2022