IT34012: VM RESTORE FROM ARCHIVE PRODUCES UNUSABLE DATA

APAR status

• Closed as program error.

Error description

• When a VM is restored from archive with IBM Spectrum Protect
  PLus in either test, clone, or production node, the restored VM
  can fail to start.
  Attempting to start up the VM in test mode results in the
  vSphere server reporting the error :
  
    "Cannot open the disk <vmdk_path>"
  
  When attempting to move the test mode VM to production, vMotion
  stops with the message :
  
    "The virtual disk is either corrupted or not a supported
  format."
  
  Similarly, when attempting to restore a database from archive in
  either test or production mode, the restore job can fail with
  errors indicating an invalid or corrupt file.
  
  The exact error message can vary depending on the type of the
  database.
  
  Restoring the same data from a vSnap or Cloud recovery point is
  successful.
  
  The problem can be confirmed by comparing the valid data
  restored from vSnap versus the invalid data restored from
  archive.
  Files restored from archive will be smaller in size than
  expected.
  
  The failure occurs because the data restored from archive is
  incomplete.
  
  During the restore process, the vSnap server downloads the
  compressed data objects into a temporary local volume.
  If download of some segments fails due to network conditions and
  has to be retried, the vSnap server may not correctly handle the
  retried segments during decompression.
  
  This can cause the stream to be interrupted resulting in an
  incomplete restored file.
  
  IBM Spectrum Protect Plus Versions Affected:
  IBM Spectrum Protect
  Plus 10.1.4 and higher
  
  Initial Impact: High
  
  Additional
  Keywords: SPP, SPPLUS, TS003901560, restore, corruption,
  archive, S3, cloud
  

Local fix

Problem summary

• ****************************************************************
  * USERS AFFECTED:                                              *
  * IBM Spectrum Protect Plus level 10.1.5 and 10.1.6.           *
  ****************************************************************
  * PROBLEM DESCRIPTION:                                         *
  * See ERROR DESCRIPTION                                        *
  ****************************************************************
  * RECOMMENDATION:                                              *
  * Apply fixing level when available. This problem is currently *
  * projected to be fixed in IBM Spectrum Protect Plus level     *
  * 10.1.7. Note that this is subject to change at the           *
  * discretion of IBM.                                           *
  ****************************************************************
  

Problem conclusion

• During restore from archive, vSnap downloads the compressed
  archive, extracts it, and writes it to the local pool. The
  compressed stream is piped directly to the extraction process.
  Under specific network conditions where the download stream is
  interrupted and has to be retried, the pipe to the extraction
  process breaks resulting in a failed download. The exact reasons
  are still under investigation because the problem is not
  reproducible outside of one specific customer environment. To
  resolve this issue, a workaround has been implemented. An
  optional alternate restore method is available which fully
  downloads the compressed data to disk before extracting it. The
  alternate restore method can be enabled by setting a preference
  on the vSnap server: vsnap system pref set --name
  archiveDownloadBeforeExtraction --value true.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  SP PLUS

• Fixed component ID

  5737SPLUS

Applicable component levels