IT36302: VIRTUAL MACHINE BACKUP FAILS AFTER REMOVING THE VM FROM A SLA AND ADDING IT BACK TO THE SAME SLA

APAR status

• Closed as program error.

Error description

• Error Description:
  The backup of a Virtual Machine in IBM Spectrum Protect Plus can
  fail if the VM has ever been backed up successfully to an SLA,
  then is removed from this SLA and added to the same SLA again at
  a later stage.
  
  The Joblog will show the backup failure without any further
  detail:
  ERROR,<id>,<time>,,CTGGA2649,Backup of VM <VM-NAME> failed due
   to an unexpected error. Contact support for further assistance.
   Error: Did not find object with key.
  ...
  SUMMARY,<id>,<time>,,CTGGA2384,VM: <VM-NAME>; Proxy: null;
   vSnap(s): vsnap.host.name; Type: Incremental; Transport: null;
   Transferred: null; Throughput: 0 B/s; Queue Time: 0 second;
   Protected VMDKs: 0 of 1; Status: failed
  
  The Virgo log will show the following:
  
  [timestamp] INFO  thread-id1 .. Initializing backup of vm:
  <VM-NAME> (host: esxi.host.name) data center: <DC-NAME>
  [timestamp] INFO  thread-id1 .. VM <VM-NAME> storageMapping key:
  c7de3ce5156c50ea6c8a258bc1fde53b ->
  spp_1070_2153_174696369d1__group9_92
  [timestamp] INFO  thread-id1 .. matched volumeId for disk Hard
  disk 1 for VM <VM-NAME>
  [timestamp] INFO  thread-id1 .. Error in
  ManagedObjectLoaderImpl(). Did not find object with key: <KeyId>
  [timestamp] INFO  thread-id1 .. VMware exception:
  com.catalogic.ecx.serviceprovider.common.hypervisor.exception.Hy
  pervisorObjectNotFoundException: Did not find object with key.
   at
  com.catalogic.ecx.serviceprovider.common.hypervisor.vmware.Manag
  edObjectLoaderImpl.getObject(ManagedObjectLoaderImpl.java:78)
   at
  com.catalogic.ecx.serviceprovider.common.hypervisor.vmware.Vsphe
  reBackupVmUpdaterImpl.getDatacenter(VsphereBackupVmUpdaterImpl.j
  ava:153)
   at
  com.catalogic.ecx.serviceprovider.common.hypervisor.vmware.Vsphe
  reBackupVmUpdaterImpl.toVsphereResult(VsphereBackupVmUpdaterImpl
  .java:337)
   at
  com.catalogic.ecx.serviceprovider.common.hypervisor.vmware.VMwar
  eVirtualMachine.backup(VMwareVirtualMachine.java:1304)
   at
  com.catalogic.ecx.serviceprovider.protection.hypervisor.cbt.CbtB
  ackupWorker.runBackup(CbtBackupWorker.java:595)
   at
  com.catalogic.ecx.serviceprovider.protection.hypervisor.cbt.CbtB
  ackupWorker.call(CbtBackupWorker.java:1076)
   at
  com.catalogic.ecx.serviceprovider.protection.hypervisor.cbt.CbtB
  ackupWorker.call(CbtBackupWorker.java:137)
   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   at
  java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExec
  utor.java:1149)
   at
  java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExe
  cutor.java:624)
   at java.lang.Thread.run(Thread.java:748)
  
  The issue happens because the VM was previously protected in the
  same SLA so it had backup results saved in the IBM Spectrum
  Protect Plus catalog.
  When it is added back to the SLA some of that backup result data
  related to these previous backups is no longer matching the
  actual inventory.
  This is confirmed in the Virgo log with the message "Did not
  find object with key: <KeyId>"
  
  IBM Spectrum Protect Plus Versions Affected:
  IBM Spectrum Protect Plus 10.1.x
  
  Initial Impact: Medium
  
  Additional Keywords: SPP, SPPlus, TS004928712
  

Local fix

• Add the VM in a new SLA
  

Problem summary

• ****************************************************************
  * USERS AFFECTED:                                              *
  * IBM Spectrum Protect Plus level 10.1                         *
  ****************************************************************
  * PROBLEM DESCRIPTION:                                         *
  * See Error Description.                                       *
  ****************************************************************
  * RECOMMENDATION:                                              *
  * Apply the fixing level when available. This problem is       *
  * projected to be fixed in IBM Spectrum Protect Plus 10.1.8.   *
  * Note that this is subject to change at the discretion of     *
  * IBM.                                                         *
  ****************************************************************
  

Problem conclusion

• This issue has been fixed by ignoring old, invalid metadata from
  previous backups.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  SP PLUS

• Fixed component ID

  5737SPLUS

Applicable component levels