IBM Support

Frequent "ANS9921E virtual machine disk verification check failed" and "ANS9919E Failed to find expected control files" errors in Data Protection for VMware versions 8.1.11 and 8.1.12.

Flashes (Alerts)


Abstract

Data Protection for VMware incremental backups of VMware virtual machines (VMs) are subject to spurious ANS9921E and ANS9919E errors. Interim fix version 8.1.12.1. is available from IBM support. This problem is documented by APAR IT37049.

Content

25 June 2021 update:
Interim fix version 8.1.12.1 is available. See Related Information on this page for links to the interim fix.
Background information
During incremental forever backup of every VM, Data Protection for VMware verifies that the number of control files that are stored on the IBM Spectrum Protect server match the expected number of control files. This verification occurs for each virtual machine disk (VMDK) to ensure the integrity of the incremental forever backup chain.
If the verification fails, then the following error messages are displayed, and a one-time full backup of the VM is needed to establish a new incremental forever backup chain:

ANS9921E Virtual machine disk, vm_name (Hard Disk n), verification check failed (xxxxx/yyyyy). If the disk is on an NFS datastore and the disk size was recently changed or the disk was moved from non-NFS datastore to NFS datastore, the verification failure is expected and a full backup is required.
ANS9919E Failed to find the expected control files for vm_name.
ANS4174E Full VM backup of VMware Virtual Machine vm_name failed with RC=7327 mode=Incremental Forever - Incremental, target node name=node_name, data mover node name=dm_node_name
Data Protection for VMware version 8.1.11.0 introduced a performance enhancement that leverages a feature introduced in vSphere 6.7 to back up only allocated blocks. This feature accelerates full backups, especially of large virtual disks, and assists incremental or differential backups.
The problem
An error in the Data Protection for VMware control file verification algorithm, related to the new performance enhancement, causes spurious verification failures during incremental backup. The problem occurs for random VMs. It can occur for one, some, or many VMs. Although the error messages signify an integrity problem, there is no data loss. VM backups affected by this defect are intact and can be restored.
The problem can lead to the following consequences:
  • Backup for each affected VM is skipped unless VMVERIFYIFACTION FORCEFULL is configured in the data mover options file. The default setting is VMVERIFYIFACTION FAILBACKUP.
  • A full backup is required for each affected VM to reestablish the incremental forever backup sequence. If VMVERIFYIFACTION FORCEFULL is configured, then the full backup occurs automatically when the error is detected. If you do not want full backups to occur during the normal backup schedule, run a manual full backup of each affected VM.
  • Full backups can lead to exceeded backup windows, increased utilization of network and storage resources, and impact to other non-backup workload.
  • Even if a full backup is done for an affected VM, it is possible for the same VM to be affected during a subsequent incremental backup.
This problem is documented in APAR IT37049.
Levels affected
IBM Spectrum Protect for Virtual Environments: Data Protection for VMware 8.1.11.0 and 8.1.12.0.
Immediate action
Install Data Protection for VMware interim fix version 8.1.12.1.
After the interim fix is installed, normal incremental backup processing occurs for all VMs.
Fix
The problem is fixed in Data Protection for VMware 8.1.12.1

[{"Type":"MASTER","Line of Business":{"code":"LOB26","label":"Storage"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SSERB6","label":"IBM Spectrum Protect for Virtual Environments"},"ARM Category":[{"code":"a8m3p000000hAaNAAU","label":"Data Protection->Virtual Environments (VE)"}],"Platform":[{"code":"PF016","label":"Linux"},{"code":"PF033","label":"Windows"}],"Version":"8.1.11;8.1.12"}]

Document Information

Modified date:
25 June 2021

UID

ibm16457031