VIO server and LPAR client relationship and storage mapping

System Administrator might be confused about this relationship and how to discover a dedicated storage device for troubleshooting .
This document will detail the easiest way to identify client-vios storage mapping

A. Introduction of VSCSI and NPIV Architecture

 

For More information regarding VSCSI and NPIV implementation:

https://www.ibm.com/support/knowledgecenter/en/8284-22A/p8hat/p8hat_virtualscsi.htm

https://www.ibm.com/support/knowledgecenter/8284-22A/p8hat/p8hat_vfc.htm



B. VSCSI-NPIV (Whole Disk mapped to client)

Let's discover the mapping between the client lpar and vio server via CLI, when whole disk is mapped from vios to client

On AIX lpar client, let's start by identifying the lpar name and lpar id with "uname -L" command.

[aix_7.1test:root:/home/root:]> uname -L
5 aix_7.1test

Lpar name >>>>>>>>>> aix_7.1test
Lpar id >>>>>>>>>>>>> 5

Then "lsmpio" command is used to identify the parent adapters:

[aix_7.1test:root:/home/root:]> lsmpio

name path_id status path_status parent connection
hdisk0 0 Enabled Sel,Opt fscsi0 5005076802205316,0
hdisk0 1 Enabled Non fscsi0 5005076802205317,0
hdisk0 2 Enabled Opt fscsi1 5005076802305316,0
hdisk0 3 Enabled Non fscsi1 5005076802305317,0
hdisk1 0 Enabled Clo fscsi0 5005076802205316,1000000000000
hdisk1 1 Enabled Clo fscsi0 5005076802205317,1000000000000
hdisk1 2 Enabled Clo fscsi1 5005076802305316,1000000000000
hdisk1 3 Enabled Clo fscsi1 5005076802305317,1000000000000
hdisk2 0 Enabled Clo fscsi0 5005076802205316,2000000000000
hdisk2 1 Enabled Clo fscsi0 5005076802205317,2000000000000
hdisk2 2 Enabled Clo fscsi1 5005076802305316,2000000000000
hdisk2 3 Enabled Clo fscsi1 5005076802305317,2000000000000
hdisk3 0 Enabled Clo fscsi0 5005076802205316,3000000000000
hdisk3 1 Enabled Clo fscsi0 5005076802205317,3000000000000
hdisk3 2 Enabled Clo fscsi1 5005076802305316,3000000000000
hdisk3 3 Enabled Clo fscsi1 5005076802305317,3000000000000
hdisk4 0 Enabled Clo vscsi0 810000000000
hdisk5 0 Enabled Clo vscsi0 820000000000


As you note the hdisk0, hdisk1, hdisk2 and hdisk3 have fscsi0 and fscsi1 as parent adapters, while hdisk4 and hdisk5 are disks provided through vscsi connection .
To confirm that above fscsi# adapters are NPIV, check that Description field is "Virtual Fibre Channel Client Adapter" by using "lsdev -Cc adapter"
 

B.1 Mapping Discovering from AIX Lpar client side

To discover virtual scsi server and virtual fiber channel adapters with vio server name relation, use below commands :

# echo "cvai" |kdb| grep vscsi >>>>>>>>>for vscsi mapping

# echo "vfcs" |kdb| grep fcs >>>>>>>>>for fcs mapping

OR

[aix_7.1test:root:]> kdb

START END <name>
0000000000001000 0000000005840000 start+000FD8
F00000002FF47600 F00000002FFDF9C8 __ublock+000000
000000002FF22FF4 000000002FF22FF8 environ+000000
000000002FF22FF8 000000002FF22FFC errno+000000
F1000F0A00000000 F1000F0A10000000 pvproc+000000
F1000F0A10000000 F1000F0A18000000 pvthread+000000
read vscsi_scsi_ptrs OK, ptr = 0xF1000000C020BE20

(0)> cvai;vfcs >>>>>>>>>for vscsi and fcs
NAME STATE CMDS_ACTIVE ACTIVE_QUEUE HOST
vscsi0 0x000007 0x0000000000 0x0 bladevio1->vhost3

NAME ADDRESS STATE HOST HOST_ADAP OPENED NUM_ACTIVE
fcs0 0xF1000A00001E8000 0x0008 bladevio1 vfchost7 0x01 0x0000
fcs1 0xF1000A00001EA000 0x0008 bladevio1 vfchost8 0x01 0x0000

(0)> exit
[aix_7.1test:root:]>

OR

If you gathered "snap -ac" collection, vscsi and vfc mapping information are stored there :

/tmp/ibmsupt/vfc_client_collect/fcs#.state
/tmp/ibmsupt/client_collect/vscsi#.state

EX.

[aix_7.1test:root:/home/root:]>more /tmp/ibmsupt/client_collect/vscsi0.state

START END <name>
0000000000001000 0000000005840000 start+000FD8
F00000002FF47600 F00000002FFDF9C8 __ublock+000000
000000002FF22FF4 000000002FF22FF8 environ+000000
000000002FF22FF8 000000002FF22FFC errno+000000
F1000F0A00000000 F1000F0A10000000 pvproc+000000
F1000F0A10000000 F1000F0A18000000 pvthread+000000
read vscsi_scsi_ptrs OK, ptr = 0xF1000000C020BE20

(0)> cvai vscsi0; cvdi vscsi0; cvcrq vscsi0
unit_id: 0x30000002 partition_num: 0x5 partition_name: aix_7.1test
capability_level: 0x0 location_code:
priv_cap: 0x1 host_capability: 0x0 host_name: vhost3 host_location:
heart_beat_enabled: 0x0 sample_time: 0x0 ping_response_time: 0x3C
rw_timeout: 0x0
host part_number: 0x1 os_type: 0x3 host part_name: bladevio1


-Understanding the vscsi mapping:

[aix_7.1test:root:/home/root:]> echo "cvai" |kdb| grep vscsi

read vscsi_scsi_ptrs OK, ptr = 0xF1000000C020BE20

vscsi0 0x000007 0x0000000000 0x0 bladevio1->vhost3

The vscsi0 is mapped to vhost3 on vios bladevio1

-For fcs mapping

[aix_7.1test:root:/home/root:]> echo "vfcs" |kdb| grep fcs

(0)> vfcs

fcs0 0xF1000A00001E8000 0x0008 bladevio1 vfchost7 0x01 0x0000

fcs1 0xF1000A00001EA000 0x0008 bladevio1 vfchost8 0x01 0x0000

so :

fcs0 on the client is mapped to vfchost7 on vio server bladevio1

fcs1 on the client is mapped to vfchost8 on vio server bladevio1

B.2 Mapping discovering from VIO server side

 

Now, we move to the virtual i/o server ( on Bladevio1 in this example, with padmin account )

For vscsi mapping:

$ lsmap -vadapter vhost3

SVSA Physloc Client Partition ID

--------------- -------------------------------------------- ------------------

vhost3 U8406.70Y.06FB98A-V1-C31 0x00000005

VTD vtscsi1
Status Available
LUN 0x8100000000000000
Backing device hdisk7
Physloc U78A5.001.WIHC7F8-P1-C11-T1-W5005076802205316-L5000000000000
Mirrored false

VTD vtscsi2

Status Available
LUN 0x8200000000000000
Backing device hdisk8
Physloc U78A5.001.WIHC7F8-P1-C11-T1-W5005076802205316-L6000000000000
Mirrored false

Note :

- Client Partition ID is in hexadecimal and need to be converted to decimal

- Client partition ID is 0 when the client partition is inactive

- VTD is the virtual target device and can have a specific name as the following:

$ mkvdev -vdev hdiskx -vadapter vhost# -dev VTD_name

- LUN is the logical unit number

- We can identify the backing device by checking the matching of the LUN ID between child device on the client and correspondent LUN ID under vhost3

In this ex.

The hdisk4 on the client has hdisk7 as backing device on vio server bladevio1 with LUN 0x8100000000000000

[aix_7.1test:root:/home/root:]> lsdev -l hdisk4 -F 'name parent connection type'

hdisk4 vscsi0 U8406.70Y.06FB98A-V5-C2-T1-L8100000000000000 vdisk

NOTE: The more reliable way to see identifier's matching between the disks of the client and vios is the unique_id method :

[aix_7.1test:root:/home/root:]> odmget -q "name=hdisk4" CuAt |grep -p unique

CuAt:
name = "hdisk4"
attribute = "unique_id"
value = "4833332136005076802808187980000000000000C04214503IBMfcp05VDASD03AIXvscsi"
type = "R"
generic = ""
rep = "n"
nls_index = 0

On virtual i/o server (from padmin)

# lsattr -El hdisk7 |grep unique_id

unique_id 332136005076802808187980000000000000C04214503IBMfcp Unique device identifier False

The hdisk5 on the client has hdisk8 as backing device on vio server bladevio1 with LUN 0x8200000000000000

[aix_7.1test:root:/home/root:]> lsdev -l hdisk5 -F 'name parent connection type'

hdisk5 vscsi0 U8406.70Y.06FB98A-V5-C2-T1-L8200000000000000 vdisk

Same unique_id between hdisk5 on the client and hdisk8 on vios :

[aix_7.1test:root:/home/root:]> odmget -q "name=hdisk5 " CuAt |grep -p unique

CuAt:
name = "hdisk5"
attribute = "unique_id"
value = "483333213600507680280818798000000000000B904214503IBMfcp05VDASD03AIXvscsi"
type = "R"
generic = ""
rep = "n"
nls_index = 0

On vio server,from padmin

# lsattr -El hdisk8 |grep unique_id

unique_id 33213600507680280818798000000000000B904214503IBMfcp Unique device identifier False

For FCS mapping:

$ lsmap -all -npiv

Name Physloc ClntID ClntName ClntOS
------------- ---------------------------------- ------ -------------- -------
vfchost7 U8406.70Y.06FB98A-V1-C32 5 aix_7.1test AIX

Status:LOGGED_IN
FC name:fcs0 FC loc code:U78A5.001.WIHC7F8-P1-C11-T1
Ports logged in:3
Flags:a<LOGGED_IN,STRIP_MERGE>
VFC client name:fcs0 VFC client DRC:U8406.70Y.06FB98A-V5-C5

Name Physloc ClntID ClntName ClntOS

------------- ---------------------------------- ------ -------------- -------
vfchost8 U8406.70Y.06FB98A-V1-C33 5 aix_7.1test AIX

Status:LOGGED_IN
FC name:fcs1 FC loc code:U78A5.001.WIHC7F8-P1-C11-T2
Ports logged in:3
Flags:a<LOGGED_IN,STRIP_MERGE>
VFC client name:fcs1 VFC client DRC:U8406.70Y.06FB98A-V5-C6

- The vfchost7 in bladevio1 with ID 1 and slot 32 (V1-C32 ) is mapped to VFC client name:fcs0

aix client ID 5 slot number 5

- The vfchost8 in bladevio1 with ID 1 and slot 33 (V1-C33 ) is mapped to VFC client name:fcs1

aix client ID 5 slot number 6

C. Mapping Overview with IVM or HMC Command

 

You can also get information about the mapping from IVM or HMC

it is very useful, especially when we have non AIX client lpar like IBMi or LINUX partition.

Discover the mapping from IVM CLI:

For vscsi mapping:

we can filter the output with partition id or partition name :

$ lshwres -r virtualio --rsubtype scsi --level lpar --filter lpar_names=aix_7.1test

$ lshwres -r virtualio --rsubtype scsi --level lpar --filter lpar_ids=5

$ lshwres -r virtualio --rsubtype scsi --level lpar --filter lpar_names=aix_7.1test

lpar_name=aix_7.1test,lpar_id=5,slot_num=2,state=1,adapter_type=client,remote_lpar_id=1,remote_lpar_name=bladevio1,remote_slot_num=31,is_required=1

NOTE : For HMC, specify option -m <Managed system> because an HMC can handle more than one managed system.

# lshwres -m <Managed System> -r virtualio --rsubtype scsi --level lpar --filter lpar_names=aix_7.1test

client information

lpar name= aix_7.1test

lpar id=5

slot number=2

adapter_type = client

vios(bladevio1) information

remote_lpar_id = 1 >>>>>>>>> With IVM management the vios should be the first partition so ID=1

remote_lpar_name = bladevio1

remote_slot_num= 31

To understand the output:

on client:

[aix_7.1test:root:/home/root:]> lsslot -c slot |grep C2

U8406.70Y.06FB98A-V5-C2 Virtual I/O Slot vscsi0

[aix_7.1test:root:/home/root:]> lsdev -p vscsi0 >>>>>>>>>To discover the child of vscsi0

hdisk4 Available Virtual SCSI Disk Drive

hdisk5 Available Virtual SCSI Disk Drive

[aix_7.1test:root:/home/root:]> lsdev -c disk4 -F 'name parent connection'

hdisk4 vscsi0 810000000000

[aix_7.1test:root:/home/root:]> lsdev -c disk5 -F 'name parent connection'

hdisk5 vscsi0 820000000000

on vios:

$ lsdev -slots |grep "C31"

U8406.70Y.06FB98A-V1-C31 Virtual I/O Slot vhost3

$lsmap -vadapter vhost3

You can identify the backing device by checking the matching of LUN ID between child device on client and the corresponding LUN ID under vhost3

For FCS mapping:

we can filter the output with partition id or partition name :

$lshwres -r virtualio --rsubtype fc --level lpar --filter lpar_names=aix_7.1test

$lshwres -r virtualio --rsubtype fc --level lpar --filter lpar_ids=5

$ lshwres -r virtualio --rsubtype fc --level lpar --filter lpar_names=aix_7.1test

lpar_name=aix_7.1test,lpar_id=5,slot_num=5,adapter_type=client,remote_lpar_id=1,remote_lpar_name=bladevio1,remote_slot_num=32,is_required=0,

"wwpns=C0507604831E003A,C0507604831E003B",state=1

lpar_name=aix_7.1test,lpar_id=5,slot_num=6,adapter_type=client,remote_lpar_id=1,remote_lpar_name=bladevio1,remote_slot_num=33,is_required=0,

"wwpns=C0507604831E003C,C0507604831E003D",state=1

First Fiber adapter

Client information: Vios information:

lpar_name=aix_7.1test remote_lpar_name=bladevio1

lpar_id=5 remote_lpar_id=1

slot_num=5 remote_slot_num=32

adapter_type=client wwpns=C0507604831E003A,C0507604831E003B

------------------------------------------

second fiber adapter

Client information: Vios information:

lpar_name=aix_7.1test remote_lpar_name=bladevio1

lpar_id=5 remote_lpar_id=1

slot_num=6 remote_slot_num=33

adapter_type=client wwpns=C0507604831E003C,C0507604831E003D

To understand the output

----------------------------

On the client:

[aix_7.1test:root:/home/root:]> lsslot -c slot |egrep 'C5|C6'

U8406.70Y.06FB98A-V5-C5 Virtual I/O Slot fcs0

U8406.70Y.06FB98A-V5-C6 Virtual I/O Slot fcs1

[aix_7.1test:root:/home/root:]> echo "vfcs" |kdb| grep fcs

(0)> vfcs

fcs0 0xF1000A00001E8000 0x0008 bladevio1 vfchost7 0x01 0x0000

fcs1 0xF1000A00001EA000 0x0008 bladevio1 vfchost8 0x01 0x0000

On the vios :

$lsdev -slots |grep "C32"

U8406.70Y.06FB98A-V1-C32 Virtual I/O Slot vfchost7

$ lsdev -slots |grep "C33"

U8406.70Y.06FB98A-V1-C33 Virtual I/O Slot vfchost8

So fcs0 on the client is mapped to vfchost7 on bladevio1 ,fcs1 on the client is mapped to vfchost8 on bladevio1 and to discover which physical fiber port run :

$lsmap -all -npiv

D. VSCSI (LV, TAPE, Virtual Media Repository and CD-ROM)

Now, take a look on different Backing Device type (lv, cd-rom, tape and virtual media repository) mapped on dual vios through HMC

D.1 Mapping Discovering from AIX Lpar client side

On AIX lpar client, let's start by identifying the lpar name and lpar id with "uname -L" command.

# uname -L

10 lpar1-149-nim-7.1.4

Lpar name>>>>>>>>> lpar1-149-nim-7.1.4
Lpar id>>>>>>>>>>>> 10

To discover the parent of all child devices (disk, cdrom, rmt, lv or virtual media repository) the below command is useful :

# lsdev -Sa -F 'name parent physloc type' |egrep 'cd|disk|rmt'

cd0 vscsi0 U8202.E4C.0625C1T-V10-C10-T1-L8200000000000000 vopt
cd3 vscsi0 U8202.E4C.0625C1T-V10-C10-T1-L8500000000000000 vopt
hdisk0 vscsi0 U8202.E4C.0625C1T-V10-C10-T1-L8100000000000000 vdisk
hdisk1 vscsi1 U8202.E4C.0625C1T-V10-C5-T1-L8100000000000000 vdisk
hdisk2 vscsi1 U8202.E4C.0625C1T-V10-C5-T1-L8200000000000000 vdisk
rmt0 vscsi0 U8202.E4C.0625C1T-V10-C10-T1-L8600000000000000 scsd

To identify such Backing Devices type, use sub-command kdb "cvai"

# echo "cvai" |kdb| grep vscsi

read vscsi_scsi_ptrs OK, ptr = 0xF1000000C0124E20

vscsi0 0x000007 0x0000000000 0x0 vios1->vhost0

vscsi1 0x000007 0x0000000000 0x0 vio2->vhost8

D.2 Mapping Discovering from VIO server side

 

From virtual i/o server vios1, run lsmap command

$ lsmap -vadapter vhost0

SVSA Physloc Client Partition ID

--------------- -------------------------------------------- --------------------------

vhost0 U8202.E4C.0625C1T-V1-C10 0x0000000a

VTD vtopt2
Status Available
LUN 0x8200000000000000
Backing device /var/vio/VMLibrary/AIX_7.2_TL_72000000_BI_DVD_2OF2.iso
Physloc
Mirrored N/A
-------------------------

VTD vtopt6
Status Available
LUN 0x8500000000000000
Backing device cd0
Physloc U78AA.001.WZSJ7PA-P2-D9
Mirrored N/A
-------------------------

VTD vtscsi0

Status Available
LUN 0x8100000000000000
Backing device lpar1-rootvg
Physloc
Mirrored N/A
-------------------------

VTD vttape0

Status Available
LUN 0x8600000000000000
Backing device rmt0
Physloc U78AA.001.WZSJ7PA-P1-D1
Mirrored N/A

-vtopt2 is a virtual media repository with an ISO image under /var/vio/VMLibrary as backing device

For more information how to create and map virtual media repository, consult the following URL:

http://www-01.ibm.com/support/docview.wss?uid=isg3T1013047

-vtopt6 is a VTD for a physical CD-ROM virtualized to the client

-lpar1-rootvg is a Logical volume

-vttape0 is VTD for a physical tape virtualized to the client

Checking the matching of LUN id between client and vios1 to know the backing device.

NOTES:

lpar1-rootvg is a logical volume

$ lslv lpar1-rootvg |grep GROUP

LOGICAL VOLUME: lpar1-rootvg VOLUME GROUP: data

$ lsdev -dev lpar1-rootvg
name status description
lpar1-rootvg available Logical volume


$ lslv lpar1-rootvg |grep IDENTIFIER
LV IDENTIFIER: 00f825c100004c0000000152a635de6f.2 PERMISSION: read/write

$ lsvg data |grep IDENTIFIER
VOLUME GROUP: data VG IDENTIFIER: 00f825c100004c0000000152a635de6f

Now proceed with above command on virtual i/o server vios2

$ lsmap -vadapter vhost8

SVSA Physloc Client Partition ID
--------------- -------------------------------------------- ------------------
vhost8 U8202.E4C.0625C1T-V2-C5 0x0000000a

VTD vtscsi2
Status Available
LUN 0x8100000000000000
Backing device nim2
Physloc
Mirrored N/A
-----------------------

VTD vtscsi3

Status Available
LUN 0x8200000000000000
Backing device prova
Physloc
Mirrored N/A
-------------------------------------------------------------------------------------------------------------------

nim2 and prova are logical volume:

$ lsdev -dev nim2

name status description
nim2 Available Logical volume

$ lslv nim2 |grep IDENTIFIER
LV IDENTIFIER: 00f825c100004c0000000152a63534ff.2 PERMISSION: read/write

$ lslv nim2 |grep GROUP
LOGICAL VOLUME: nim2 VOLUME GROUP: data

$ lsvg data |grep IDENTIFIER
VOLUME GROUP: data VG IDENTIFIER: 00f825c100004c0000000152a63534ff

$ lsdev -dev prova

name status description
prova Available Logical volume

$ lslv prova |grep GROUP
LOGICAL VOLUME: prova VOLUME GROUP: rootvg

$ lslv prova |grep IDENTIFIER
LV IDENTIFIER: 00f825c100004c0000000152a1f2b259.16 PERMISSION: read/write

$ lsvg rootvg |grep IDENTIFIER

VOLUME GROUP: rootvg VG IDENTIFIER: 00f825c100004c0000000152a1f2b259