IT34283: RANDOM PERFORMANCE MONITOR SAMPLES SKIPPED, LOGGING HWNPM4302E

APAR status

• Closed as program error.

Error description

• Storage Insights occasionally marks the monitoring status as
  ?degraded? due to periodic missed performance data
  samples.
  
  This can lead the user to believe that there?s a
  problem with the device, when in fact this degraded status is
  the result of the data collector agent?s scheduler function,
  polling for sample data at incorrect intervals.
  
  An example of
  this message from the performance monitoring logs visible in
  the Storage Insights user interface:
  
  HWNPM4302E New
  performance data is not yet available for the device.
  
  Error
  	Sep 15, 2020 	04:37:21.847	HWNPM4302E	New performance data is
  not yet available for the device. Statistics with time stamps
  later than 09/15/2020 04:32:22:000 CDT could not be
  found.
  Information	Sep 15, 2020 	04:37:21.907	HWNPM2145I	The
  data is being collected by the data collector:
  hostname.company.com.
  
  The Performance Monitoring scheduler
  which runs within the Data Collector agent has been observed to
  vary the Performance Monitoring interval, which should be a
  consistent 5 minute interval. It has been observed to poll at
  inconsistent length intervals - for example, polling at 4
  minutes, 13 seconds once, then 5 minute, 20 seconds the next
  time, etc.
  
  Confirmation of this issue requires Storage
  Insights support reps to view the performance monitor tracing
  logs, and observe incorrect polling interval timestamps, as
  illustrated here:
  
  sample #1 at 12:27:54
  2020-09-09
  12:27:54.261-0400 xxx-000000000000...XXX StatsCollectorSvcNi
  queryDateTimeInfo EXIT return value = Timestamp= 20200909
  122754 -4:00
  ...2020-09-09 12:27:54.671-0400
  xxx-000000000000...XXX StatsCollectorSvcNi selectFileSet EXIT
  return value = MoreCnt: 0, Selected:
  [Nn_stats_78E00MY-1_200909_122312,...
  
  sample #2 at 12:33:15 (
  5min 21 sec)
  2020-09-09 12:33:15.988-0400
  XXX-000000000000...XXX StatsCollectorSvcNi queryDateTimeInfo
  EXIT return value = Timestamp= 20200909 123315
  -4:00
  ....2020-09-09 12:33:16.394-0400 XXX-000000000000...XXX
  StatsCollectorSvcNi selectFileSet EXIT return value = MoreCnt:
  0, Selected: [Nn_stats_78E00MY-1_200909_123313,...
  
  sample #3
  at 12:37:58 ( 4min 43sec.)
  2020-09-09 12:37:58.813-0400
  XXX-000000000000...XXX StatsCollectorSvcNi queryDateTimeInfo
  EXIT return value = Timestamp= 20200909 123758
  -4:00
  ...2020-09-09 12:37:59.204-0400 XXX-000000000000...XXX
  StatsCollectorSvcNi selectFileSet EXIT return value = MoreCnt:
  0, Selected: [Nn_stats_78E00MY-1_200909_123313,...
  
  sample #4
  at 12:42:39 (4min 41sec)
  2020-09-09 12:42:39.301-0400
  XXX-000000000000...XXX StatsCollectorSvcNi queryDateTimeInfo
  EXIT return value = Timestamp= 20200909 124239
  -4:00
  ...2020-09-09 12:42:39.707-0400 XXX-000000000000...XXX
  StatsCollectorSvcNi selectFileSet EXIT return value = MoreCnt:
  0, Selected: [Nn_stats_78E00MY-1_200909_123814,...
  
  sample #5
  at 12:47:42 (5min 03sec)
  2020-09-09 12:47:42.221-0400
  XXX-000000000000...XXX StatsCollectorSvcNi queryDateTimeInfo
  EXIT return value = Timestamp= 20200909 124742
  -4:00
  ...2020-09-09 12:47:42.690-0400 XXX-000000000000...XXX
  StatsCollectorSvcNi selectFileSet EXIT return value = MoreCnt:
  0, Selected: [Nn_stats_78E00MY-1_200909_124315,...
  
  
  The result
  of this is that the Data Collector will poll the monitored
  storage device, before it is ready to provide the sample data
  based on the previous polling time. When samples are not
  collected, the data collection status changes to degraded,
  which can lead to undue concern and confusion.
  
  
  
  
  
  ____________________________________________________
  
  The
  defect is against component: 5608TPCSI
  Server/Manager
  build/release (TPC): 5.4.0
  
  Problem as described by customer:
  Degraded monitoring status in Storage Insights
  Initial customer
  impact (low/med/high): med
  

Local fix

• Temporary relief can be achieved by restarting the monitoring
  tasks for affected storage devices.
  

Problem summary

• ****************************************************************
  * USERS AFFECTED:                                              *
  * IBM Storage Insights users                                   *
  ****************************************************************
  * PROBLEM DESCRIPTION:                                         *
  * Storage Insights occasionally marks the monitoring status    *
  * as "Degraded" due to periodic missed performance data        *
  * samples.                                                     *
  *                                                              *
  * This can lead the user to believe that there is a            *
  * problem with the device, when in fact this degraded          *
  * status is the result of the data collector agent's scheduler *
  *                                                              *
  * function, polling for sample data at incorrect intervals.    *
  *                                                              *
  * The Performance Monitoring scheduler which runs within       *
  * the Data Collector agent has been observed to vary the       *
  * Performance Monitoring interval, which should be a           *
  * consistent 5 minute interval. It has been observed to poll   *
  * at inconsistent length intervals - for example, polling at   *
  * 4 minutes, 13 seconds once, then 5 minute, 20 seconds        *
  * the next time, etc.                                          *
  *                                                              *
  * The result of this is that the Data Collector will poll the  *
  * monitored storage device, before it is ready to provide the  *
  * sample data based on the previous polling time. When         *
  * samples are not collected, the data collection status        *
  * changes                                                      *
  * to Degraded, which can lead to undue concern and confusion.  *
  *                                                              *
  * Confirmation of this issue requires Storage Insights Support *
  *                                                              *
  * team to view the performance monitor tracing logs, and       *
  * observe incorrect polling interval timestamps to review and  *
  * confirm the issue.                                           *
  *                                                              *
  ****************************************************************
  * RECOMMENDATION:                                              *
  ****************************************************************
  

Problem conclusion

• The fix for this APAR is targeted for the following release:
  
  IBM Storage Insights 2Q21  [ 54X-IBM-SI ]
  ( release target 2Q 2021 / June )
  
  The target dates for future releases do not represent a formal
  commitment by IBM. The dates are subject to change without
  notice.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  STORAGE INSIGHT

• Fixed component ID

  5608TPCSI

Applicable component levels