IBM Support

lsvg shows VIOS rootvg hdisk with PV STATE "missing" or "removed"

Troubleshooting


Problem

lsvg shows VIOS rootvg hdisk in missing or removed state
$ lsvg -pv rootvg

PV_NAME           PV STATE          TOTAL PPs   FREE PPs    FREE DISTRIBUTION
hdisk0            active            745         688         148..118..124..149..149
hdisk1            missing           745         710         148..140..124..149..149

Symptom

If the volume group is mirrored, the LV STATE of the logical volumes for which I/O has failed might be stale.
$ lsvg -lv rootvg

rootvg:
LV NAME             TYPE       LPs     PPs     PVs  LV STATE      MOUNT POINT
hd5                 boot       1       2       2    closed/stale  N/A
hd6                 paging     2       4       2    open/syncd    N/A
hd8                 jfs2log    1       2       2    open/stale    N/A
hd4                 jfs2       1       2       2    open/stale    /
hd2                 jfs2       5       10      2    open/stale    /usr
hd9var              jfs2       1       2       2    open/stale    /var
hd3                 jfs2       5       10      2    open/stale    /tmp
hd1                 jfs2       10      20      2    open/stale    /home
hd10opt             jfs2       1       2       2    open/stale    /opt
hd11admin           jfs2       1       2       2    open/syncd    /admin
lg_dumplv           sysdump    3       3       1    open/syncd    N/A
livedump            jfs2       1       2       2    open/syncd    /var/adm/ras/livedump
lg_dumplv2          sysdump    1       1       1    closed/syncd  N/A
lv_mkbackup         jfs2       5       10      2    open/stale    /mkbackup
VMLibrary           jfs2       20      20      1    open/syncd    /var/vio/VMLibrary

Environment

VIOS 3.1

Diagnosing The Problem

Log in to VIOS as padmin, and run the following commands.  In this example, hdisk1 is the disk in question.
1. Determine the disk state and disk type
$ lsdev -type disk|grep hdisk1
hdisk1 Available     NVMe 4K Flash Disk
2. Determine the disk's parent adapter
$ lsdev -dev hdisk1 -parent
parent
nvme1
3. Determine the disk physical location field
$ lsdev -dev hdisk1 -field name physloc
name             physloc
hdisk1           00-00 U78D2.001.WZS0XYJ-P2-C3-L1 NVMe 4K Flash Disk
If the disk is in Defined state, carefully review the VIOS errlog to determine if the disk, its parent adapter or LVM layer is logging errors
$ errlog|grep hdisk                                   
$ errlog|grep <parent_adapter_name_in_step_2>     e.g. 'errlog|grep nvme1'

Resolving The Problem

Determine if the disk is accessible
$ oem_setup_env
# lquerypv -h /dev/hdisk# 80 10
# exit                     (back to padmin)
If the disk is accessible, lquerypv command will return one line of output where the 2nd and 3rd column make up the Physical Volume Identifier (PVID)
To list the disk PVID, run:
$ lspv                      
If the hdisk in question is not listed, it is indicative the disk is not accessible.
When PV STATE is "missing"
If the disk state is "Available" and lquerypv returns output, try activating the volume group:
$ activatevg rootvg
This will cause LVM to probe the disks in rootvg.
When PV STATE is "removed"
$ oem_setup_env
# chpv -va hdisk#
Depending on the disk condition, the activatevg or chpv operation can make the PV state Active or the I/O attempt might trigger disk, adapter, or LVM  errors if there is an underlying hardware problem.
Check the PV STATE
$ lsvg -pv rootvg
If the hdisk state remains the same, re-evaluate the errlog.
For assistance with disk errors for storage that is internal to the managed system or adapter errors, contact your local Hardware Support representative.  If the storage is SAN-attached, contact to SAN administrator to assess things on the SAN (storage and switch) side.

Document Location

Worldwide

[{"Type":"MASTER","Line of Business":{"code":"LOB57","label":"Power"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SSPHKW","label":"PowerVM Virtual I\/O Server"},"ARM Category":[{"code":"a8m0z0000001i2sAAA","label":"PowerVM VIOS-\u003ELVM"}],"ARM Case Number":"TS012620939","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"3.1.2;3.1.3;3.1.4"}]

Document Information

Modified date:
19 April 2023

UID

ibm16983983