IT35694: SNAPSHOT CONSOLIDATION FAILS AFTER SUCCESSFUL VM BACKUPS WITH 8.1.11

APAR status

• Closed as program error.

Error description

• After successful virtual machine (VM) backups using IBM
  Spectrum Protect for Virtual Environments - Data Protection for
  VMware version 8.1.11.0 the VMware vCenter warns that the
  consolidation for some but not all of the backed up VM disks
  fails.
  For example, a warning event is listed by the VMware vSphere
  Client monitor stating:
  
  Virtual machine disks consolidation failed.
  
  Subsequent backups of the affected VMs will be successful but
  then also cause the following warning to be written to the data
  mover node error log:
  
  ANS2718W The virtual machine 'VM_Name' requires snapshot
           consolidation.
  
  A following manual snapshot consolidation for these disks using
  VMware administrative interfaces like the VMware vSphere Client
  may again fail if dsmc or dsmagent processes from the VM backup
  operation are still running.
  
  
  =======================
  Diagnosing the issue:
  
  The vmware.log file for the affected guests will show messages
  like the following for the snapshot consolidation:
  
  [DATE]T[TIME]Z.......|.....: DISK:
   Failed to open disk for consolidate
   '/vmfs/volumes/.................................../VM_Name/DISK
  _NAME-SNAPID.vmdk'
   : Failed to lock the file (.....) ....
  
  In an IBM Spectrum Protect Client service and vddk trivia trace
  there will be messages for opening the VM disks to backup.
  For example, there will be a message like the following for
  each disk:
  
  [date] [time] [012345] [6789] :
   ..\..\common\vm\vmvddksdk.cpp(1950): vddksdkOpen: Opening disk
   '[datastore_name] vm_name/disk_name.vmdk'
  
  Towards the end of the backup operation there should be
  messages for closing the VM disks, for example for each disk:
  
  [date] [time] [012345] [6789] :
   ..\..\common\vm\vmvddksdk.cpp(1368):
   VixDiskLib: VixDiskLib_Close: Close disk.
  
  If the number of "VixDiskLib_Close: Close disk" messages is
  less than the number of "vddksdkOpen: Opening disk" messages,
  then this APAR applies.
  
  
  =======================
  Affected product versions:
  IBM Spectrum Protect for Virtual Environments
  - Data Protection for VMware version 8.1.11.0
  on Windows and Linux
  
  
  =======================
  Initial Impact: Medium
  
  
  =======================
  Additional Keywords:
  tsm tsmve TS004612112
  | MDVREGR 8.1.11.0 5725TVEVM |
  

Local fix

• Follow the "Resolving The Problem" section in the article
  "VMware snapshot consolidation failure due to virtual machine
  file lock" (https://www.ibm.com/support/pages/node/267727)
  which describes a similar symptom due to a different root
  cause.
  Manually remove the lock, for example by rebooting the data
  mover host and contact IBM support.
  

Problem summary

• ****************************************************************
  * USERS AFFECTED:                                              *
  * Data Protection for VMware 8.1.11  running on Windows and    *
  * Linux platforms.                                             *
  ****************************************************************
  * PROBLEM DESCRIPTION:                                         *
  * See ERROR DESCRIPTION                                        *
  ****************************************************************
  * RECOMMENDATION:                                              *
  * This issue is projected to be fixed in the Data Protection   *
  * for VMware version 8.1.12  on Windows x64 and Linux x86      *
  * platforms.                                                   *
  * Note that this is subject to change at the discretion of     *
  * IBM.                                                         *
  ****************************************************************
  

Problem conclusion

• Now all handles of disks are closed correctly.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Modules/Macros

• dsmc
  

Fix information

• Fixed component name

  TSM FOR VE DP V

• Fixed component ID

  5725TVEVM

Applicable component levels