APAR status
Closed as program error.
Error description
When using IBM Spectrum Protect Plus to backup Linux guests : - having some or all filesystems managed with the Logical Volume Manager (LVM) and - with the agent option "Catalog file metadata" enabled, the job log will display warning messages as seen in the following example : SUMMARY,<timestamp>,CTGGA2399,Starting job for policy BACKUP with job name <SLAName> (ID:xxxx). id -> <JobID>. IBM Spectrum Protect Plus version 10.1.7-3043. ... DETAIL,<timestamp>,CTGGA0671,Backing up VM (<VMName>) from remote proxy(IP: <vADPName> Host name: <vADPHostName>) INFO,<timestamp>,CTGGA2110,Will do incremental backup for VM <VMName> ... WARN,<timestamp>,CTGGA4077,[<VM_IP>] Detected duplicate UUIDs for one or more physical volumes. Aborting discovery of volume group information. WARN,<timestamp>,CTGGA4075,[<VM_IP>] Failed to discover volume group information. WARN,<timestamp>,CTGGA4084,[<VM_IP>] Failed to determine covering disks for volume /dev/mapper/fs1. WARN,<timestamp>,CTGGA4084,[<VM_IP>] Failed to determine covering disks for volume /dev/mapper/fs2. DETAIL,<timestamp>,CTGGA4088,[<VM_IP>] Collect file metadata under mountpoint /<non-lvm-filesystem>. DETAIL,<timestamp>,CTGGA4090,[<VM_IP>] Collected metadata for yyy file path(s) under mountpoint /<non-lvm-filesystem> WARN,<timestamp>,CTGGA4084,[<VM_IP>] Failed to determine covering disks for volume /dev/mapper/fs3. WARN,<timestamp>,CTGGA4084,[<VM_IP>] Failed to determine covering disks for volume /dev/mapper/fs4. ... WARN,<timestamp>,CTGGA4084,[<VM_IP>] Failed to determine covering disks for volume /dev/mapper/fs<z>. DETAIL,<timestamp>,CTGGA4002,[<VM_IP>] Discovered files and directories from 1 mounted volume(s) on 1 disk(s) in <xx>s. INFO,<timestamp>,CTGGA2219,[<VM_IP>] Discovery on Host <VMName> completed with status success ... SUMMARY,<timestamp>,CTGGA2384, VM: <VMName>; Proxy: <vADPHostName>; vSnap(s): <vSnapHostName>; Type: Incremental; Transport: [hotadd]; Transferred: xx GB; Throughput: yy MB/s; Queue Time: 0 second; Protected VMDKs: z of z; Status: complete ... SUMMARY,<timestamp>,CTGGA2340,Job policy <SLAName> completed in <elapsed time> and ended with status COMPLETED. id <JobID> The application log found in the job log bundle will display the following as seen for one of the guest file systems : [<timestamp>] JOBLOG pid:<PID> MainThread main: <CTGGA4001> Hostname: <VMHostName> / Operating System: Red Hat Enterprise Linux Server 7.9 x86_64 / Python: 3.6.9 [<timestamp>] JOBLOG pid:<PID> MainThread discovery: Paths excluded from file discovery: /mnt/spp; /tmp; /usr/bin; /bin; /sbin [<timestamp>] JOBLOG pid:<PID> MainThread discovery: <CTGGA4160> Discovering disks and mounted volumes [<timestamp>] INFO pid:<PID> MainThread linux_populate_blockdevice_info: Collecting block device information [<timestamp>] WARN pid:<PID> MainThread linux_populate_volume_group_info_pvs: <CTGGA4077> Detected duplicate UUIDs for one or more physical volumes. Aborting discovery of volume group information. [<timestamp>] WARN pid:<PID> MainThread populate_volume_group_info: <CTGGA4075> Failed to discover volume group information. [<timestamp>] INFO pid:<PID> MainThread populate_mounted_volume_info: Collecting mounted volume information [<timestamp>] INFO pid:<PID> MainThread get_volume_discovery_json: Traceback (most recent call last): File "/opt/IBM/SPP/agents_root/agtcomm/diskutil.py", line 911 , in get_volume_discovery_json File "/opt/IBM/SPP/agents_root/agtcomm/diskutil.py", line 813 , in get_covering_disks Exception: No physical volumes found for <lvm_volume_group_name> [<timestamp>] WARN pid:<PID> MainThread get_volume_discovery_json: <CTGGA4084> Failed to determine covering disks for volume /dev/mapper/fs1. [<timestamp>] INFO pid:<PID> MainThread get_volume_discovery_json: Traceback (most recent call last): File "/opt/IBM/SPP/agents_root/agtcomm/diskutil.py", line 911 , in get_volume_discovery_json File "/opt/IBM/SPP/agents_root/agtcomm/diskutil.py", line 813 , in get_covering_disks Exception: No physical volumes found for <lvm_volume_group_name> Regardless of whether the VM backup is successful or not, the cataloging of the file metadata will fail. Verifying the guest disks Universal Unique Identifiers (UUID) will not display any duplicates. IBM Spectrum Protect Plus queries the guest defined volume groups with the in-guest command : 'pvs'. In this occurrence, the output does provide the information but with a warning message. Example of a guest with two defined volume groups : WARNING: Failed to connect to lvmetad. Falling back to device scanning. PV VG Fmt Attr PSize PFree /dev/sdb2 vg1 lvm2 a-- 480.00g 4.00m /dev/sdc1 vg2 lvm2 a-- 300.00g 50.00g /dev/sdd1 vg2 lvm2 a-- 600.00g 100.00g When this command has a warning message, it is incorrectly parsed and results with the misleading message seen above in the application log : <CTGGA4077> Detected duplicate UUIDs for one or more physical volumes. The above 'pvs' command warning means it is trying to use the 'lvmetad' service that is not active. Disabling lvmetad is documented in the IBM Spectrum Protect Plus file indexing and restore requirements : "If data resides on LVM volumes, the lvm2-lvmetad service must be disabled, as it can interfere with the ability of IBM Spectrum Protect Plus to mount and resignature volume group snapshots and clones. To disable the service, complete the following steps: - Run the following commands: systemctl stop lvm2-lvmetad systemctl disable lvm2-lvmetad - Edit the /etc/lvm/lvm.conf file and specify the following setting: use_lvmetad = 0" This APAR addresses the incorrect warning message parsing. IBM Spectrum Protect Plus Versions Affected: IBM Spectrum Protect Plus 10.1.3 to 10.1.7 when enabling "Catalog file metadata" option for Linux guests using LVM. | MDVREGR 10.1.2 5737SPLUS | IT24930 Initial Impact: Medium Additional Keywords: SPP, SPPLUS, TS004585641, LVM, UUID
Local fix
Problem summary
**************************************************************** * USERS AFFECTED: * * IBM Spectrum Protect Plus levels 10.1.3 to 10.1.7. * **************************************************************** * PROBLEM DESCRIPTION: * * See Error Description. * **************************************************************** * RECOMMENDATION: * * Apply the fixing level when available. This problem is * * projected to be fixed in IBM Spectrum Protect Plus levels * * 10.1.8 patch1 and 10.1.9. Note that this is subject to * * change at the discretion of IBM. * ****************************************************************
Problem conclusion
When a Linux VM is configured to use lvmetad, a specific error message will be displayed indicating lvm is configured to use lvmetad and instructions on how to fix this. Additionally, test and configure will now check if lvm is configured to use lvmetad and if so flag it as an error.
Temporary fix
Comments
APAR Information
APAR number
IT35264
Reported component name
SP PLUS
Reported component ID
5737SPLUS
Reported release
A17
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt / Xsystem
Submitted date
2020-12-14
Closed date
2021-05-26
Last modified date
2021-05-26
APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:
Fix information
Fixed component name
SP PLUS
Fixed component ID
5737SPLUS
Applicable component levels
[{"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SSNQFQ","label":"IBM Spectrum Protect Plus"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"A17","Line of Business":{"code":"LOB26","label":"Storage"}}]
Document Information
Modified date:
31 January 2024