IT31377: SPP BACKUP JOB SHOWS FAILURE TO MOUNT NFS SHARE WITH MESSAGE "NO SUCH FILE OR DIRECTORY"

APAR status

• Closed as program error.

Error description

• Error Description:
  
  SPP backup job fails for one or more VMs. The error message
  indicates a vSnap NFS path that failed to mount with the
  message: No such file or directory.
  
  
  Sample log message:
  
  Command finished with error: exit status 32  desc (mount.nfs:
  mounting <vsnap_address>:/vsnap/vpoolXX/fsYY failed, reason
  given by server: No such file or directory) (Mounting failed)}
  
  
  This occurs when the internal filesystem mount point on the
  vSnap server is  missing even though the ZFS filesystem thinks
  it is mounted. The issue can be confirmed by running the
  following commands on the vSnap server for the specified mount
  point:
  
  
  zfs get mounted /vsnap/vpoolXX/fsYY
  
  df | grep "/vsnap/vpoolXX/fsYY"
  
  If the first command reports "yes" but the second command
  shows no results, This problem is confirmed. It indicates a
  mismatch between the ZFS file system thinking the path is
  mounted and the Linux OS thinking it is not mounted.
  
  Spectrum Protect Plus Versions Affected:
  
  10.1.4 10.1.5
  
  Customer/L2 Diagnostics (If Applicable)  N/A
  
  Initial Impact: Low|Medium|High                Medium
  
  Additional Keywords: ZFS MOUNTPOINT MISSING
  

Local fix

• Reboot the vSnap server while no jobs are active.
  

Problem summary

• ****************************************************************
  * USERS AFFECTED:                                              *
  * IBM Spectrum Protect Plus level 10.1.4 and 10.1.5.           *
  ****************************************************************
  * PROBLEM DESCRIPTION:                                         *
  * See Error Description.                                       *
  ****************************************************************
  * RECOMMENDATION:                                              *
  * Apply fixing level when available. This problem is currently *
  * projected to be fixed in IBM Spectrum Protect Plus level     *
  * 10.1.5 patch1 and 10.1.6. Note that this is subject to       *
  * change at the discretion of IBM.                             *
  ****************************************************************
  

Problem conclusion

• When a vSnap server boots up, the storage pool is mounted during
  startup, and filesystems/volumes in the pool are also mounted. A
  race condition existed in the mounting logic that caused an
  inconsistency between the mount status recorded in the pool
  metadata and the actual operating system mount table. The result
  was that the volume was marked as mounted in the pool metadata
  but was not actually mounted at the operating system layer. This
  then resulted in job failures when vSnap attempted to share the
  volume to external clients. The issue has been resolved by
  fixing the race condition in the mounting logic during startup.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  SP PLUS

• Fixed component ID

  5737SPLUS

Applicable component levels