IT36044: STORAGE SYSTEM REMOVED PORT ALERTS TRIGGERED ERRONEOUSLY FOR DS8000 STORAGE.

APAR status

• Closed as program error.

Error description

• After the daily probe portion of the data collection ran, an
  alert for Storage System Removed Port occurred for every FC
  port in a particular DS8000 storage device running
  7.8.1.51.159. The ports were not actually removed and there was
  no impact to the SAN. Similar alerts were not generated for
  other storage devices.
  
  The probe logs showed that the probe
  had been collecting FC ports and hosts info and then no longer
  could on a subsequent
  probe.
  
  Working...
  [EP_DS_CLI_Probe.getLSSandStorageVolumes]
  [LOG] HWN021733I 4,574 Volumes
  Found.
  [EP_DS_CLI_Probe.getIOPorts] [LOG] HWN021713I Collecting
  fibre channel ports for storage system
  2107-xxxxxxx.
  [EP_DS_CLI_Probe.getVolGroups] [LOG] HWNEP0015I
  Collecting volume groups on storage system
  2107-xxxxxxx.
  [EP_DS_CLI_Probe.getConnectedHosts] [LOG]
  HWNEP0016I Collecting hosts on storage system
  2107-xxxxxxx.
  [EP_DS_CLI_Probe.getConnectedHosts] [LOG]
  HWNEP0017I 110 Hosts
  Found.
  
  
  Failing...
  [EP_DS_CLI_Probe.getLSSandStorageVolumes]
  [LOG] HWN021733I 4,532 Volumes
  Found.
  [EP_DS_CLI_Probe.getIOPorts] [LOG] HWN021713I Collecting
  fibre channel ports for storage system
  2107-xxxxxxx.
  [EP_DS_CLI_Probe.getVolGroups] [LOG] HWNEP0015I
  Collecting volume groups on storage system
  2107-xxxxxxx.
  [EP_DS_CLI_Probe.getConnectedHosts] [LOG]
  HWNEP0016I Collecting hosts on storage system
  2107-xxxxxxx.
  [EP_DS_CLI_Probe.getConnectedHosts] [LOG]
  HWNEP0017I 0 Hosts Found.
  
  The ESSNI niTransmission_0.log shows
  "Successful: false;" occurring when requesting the data rather
  than "Successful: true;".
  
  NIPhysicalConfigurationService.getFib
  reChannelPorts(IBM.2107-xxxxxxx):Successful: false; Return
  code: be740006; Resource: null"
  NIHostConfigurationService.getHo
  sts(IBM.2107-xxxxxxx):Successful: false; Return code: be740006;
  Resource: 110 instances of
  com.ibm.storage.ess.ni.resource.NISCSIHostImpl"
  
  This is caused
  by an ESSNI memory leak and required recycling ESSNI so that
  IBM Storage Insights could collect the FC ports and hosts
  again. The fix for the issue is expected to be in later
  microcode: R8.5 SP5.2 - 88.55.9.2.
  https://www.ibm.com/support/pages/ds8000-code-recommendation
  
  
  No fix for IBM Storage Insights is needed.
  

Local fix

• Recycle ESSNI
  

Problem summary

• ****************************************************************
  * USERS AFFECTED:                                              *
  * IBM Spectrum Control & Storage Insights users monitoring     *
  * DS8000 storage systems                                       *
  ****************************************************************
  * PROBLEM DESCRIPTION:                                         *
  * For DS8000 storage systems that run microcode levels         *
  * earlier than R8.5 SP5.2 - 88.55.9.2, after a probe you       *
  * might receive alerts that fibre channel ports were removed.  *
  * In most cases, the alerts are invalid and the ports were not *
  *                                                              *
  * actually removed. This problem occurs because of a           *
  * memory leak in the ESSNI network layer.                      *
  *                                                              *
  *                                                              *
  *                                                              *
  *                                                              *
  ****************************************************************
  * RECOMMENDATION:                                              *
  ****************************************************************
  

Problem conclusion

• To fix this problem when it occurs, contact IBM Support to
  help recycle ESSNI before the next probe.
  
  For a permanent fix, update your DS8000 to the latest
  microcode level.
  
  For information about recommended microcode levels for
  DS8000, see:
  https://www.ibm.com/support/pages/ds8000-code-recommendation.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  STORAGE INSIGHT

• Fixed component ID

  5608TPCSI

Applicable component levels