Zoning requirements for the use of SAN heartbeat communication

Answer

TARGET AUDIENCE:

Users running PowerHA taking advantage of the SAN heartbeat functionality

OBJECTIVE:

Provide information for proper setup and verification of SAN zones being created for exchanging SAN heartbeat information.

OVERVIEW:

The available documentation for SAN heartbeat functionality is spread across a number of different documents in the AIX and PowerHA knowledge centers. As there is also some misleading information in the developerWorks® web resource it is necessary to provide some clarification about zoning requirements for SAN heartbeat communication.

PROCEDURES:

The AIX knowledge center states the following: "During storage area network (SAN) port configuration you must verify that your server interfaces are connected to the SAN fabric ports in the same zone." This implies that only the use of a single fabric to exchange SAN heartbeat information is supported. The use of multiple fabrics to exchange SAN heartbeat information was never tested and is not supported.
There are architectural reasons to limit SAN heartbeat to a single fabric. SAN heartbeat packets are exchanged using direct and indirect communication paths, and when using multiple fabrics the number of possible communication paths will exceed the limits of the architecture. As SAN heartbeat has been designed as an additional interface to exchange heartbeat information in a cluster, and usually multiple adapters are being used to exchange these heartbeats this should not create any exposure to the cluster.
To limit the number of possible paths to a given cluster node and to limit the efforts required for error recovery during a hardware event such as an adapter outage it is also recommend to limit the number of physical FC adapters ports that have target mode enabled (or, in other words, have the TME attribute set to "yes").
When zoning the TME enabled adapters please follow these rules:

• Create a single zone that contain the WWPNs of all TME enabled adapters.
• Do not add WWPNs of adapters that do not have TME enabled to that zone to avoid long boot delays.
• Rather then using port zoning zone the WWPNs of the adapters.

Use IBM's devscan tool to verify that all target mode enabled adapters are in the same zone. In a configuration with 4 FC adapters being used to exchange SAN heartbeats an example of the devscan output would look like the following:
Processing FC device:
    Adapter driver: fcs1
    Protocol driver: fscsi1
    Connection type: fabric
    Link State: up
    Current link speed: 8 Gbps
    Local SCSI ID: 0xa10c00
    Local WWPN: 0x10000090fa2b9b91
    Local WWNN: 0x20000090fa2b9b91
    Device ID: 0xdf1000f114108a03
    Microcode level: 203305

SCSI ID LUN ID           WWPN             WWNN
-----------------------------------------------------------
a10c00  0000000000000000 10000090fa2b9b91 20000090fa2b9b91
    START failed with errno EINVAL

a10700  0000000000000000 10000090fa2b94df 20000090fa2b94df
    Vendor ID: IBM          Device ID: AIX Node Rev: 0001 NACA: no
    PDQ: Connected          PDT: Processor
    Name:     No ODM match  VG:       None found
    Device already SCIOLSTARTed    Dynamic Tracking Enabled
    VPD Inquiry Page 0x0 failed with errno EIO
    VPD Inquiry Page 0x0 got SCSI status:
        Check Condition (sense key: (0x05) ILLEGAL_REQUEST; ASCQ: (0x240
        0) INVALID FIELD IN CDB)

410c00  0000000000000000 10000090fa26ffff 20000090fa26ffff
    Vendor ID: IBM          Device ID: AIX Node Rev: 0001 NACA: no
    PDQ: Connected          PDT: Processor
    Name:     No ODM match  VG:       None found
    Device already SCIOLSTARTed    Dynamic Tracking Enabled
    VPD Inquiry Page 0x0 failed with errno EIO
    VPD Inquiry Page 0x0 got SCSI status:
        Check Condition (sense key: (0x05) ILLEGAL_REQUEST; ASCQ: (0x240
        0) INVALID FIELD IN CDB)

510600  0000000000000000 10000090fa3b26b9 20000090fa3b26b9
    Vendor ID: IBM          Device ID: AIX Node Rev: 0001 NACA: no
    PDQ: Connected          PDT: Processor
    Name:     No ODM match  VG:       None found
    Device already SCIOLSTARTed    Dynamic Tracking Enabled
    VPD Inquiry Page 0x0 failed with errno EIO
    VPD Inquiry Page 0x0 got SCSI status:
        Check Condition (sense key: (0x05) ILLEGAL_REQUEST; ASCQ: (0x240
        0) INVALID FIELD IN CDB)

4 targets found, reporting 3 LUNs,
3 of which responded to SCIOLSTART.
Elapsed time this adapter: 00.171515 seconds

This output should be similar on all servers with adapters being used for SAN heartbeat communication. Please note that it is expected that the local adapter is listed as a target too (even though a STARTINITR command will fail with EINVAL) .



REFERENCES:

• IBM devscan tool
  http://www-01.ibm.com/support/docview.wss?uid=aixtoolsc9e095f
• "Defining a virtual Ethernet adapter" in the IBM Knowledge center for AIX 7.1, Cluster communication

https://www.ibm.com/support/knowledgecenter/ssw_aix_71/com.ibm.aix.clusteraware/claware_san_information.htm

CATEGORY:

AIXIOD

SUPPORT:

If additional assistance is required after completing all of the instructions provided in this document, please follow the step-by-step instructions below to contact IBM to open a service request (PMR) for software under warranty or with an active and valid support contract.  The technical support specialist assigned to your support call will confirm that you have completed these steps.

 a.  Document and/or take screen shots of all symptoms, errors, and/or messages that might have occurred

b.  Capture any logs or data relevant to the situation

c.  Contact IBM to open a support call (PMR):

• For electronic support, please visit the web page:

https://www-947.ibm.com/support/servicerequest/newServiceRequest.action
• For telephone support, please visit the web page:

http://www.ibm.com/planetwide
• Please visit the IBM Support Portal web page for additional resources:

https://www-947.ibm.com/support/entry/myportal/support

d.  Provide a good description of your issue and reference this technote

e.  Upload all of the details and data to your support call (PMR):

Please visit this web page for instructions:  https://www.secure.ecurep.ibm.com/app/upload

FEEDBACK:

Quality documentation is important to IBM and its customers.  If you have feedback specific to this article, please send an detailed message to the email address:

• aix_feedback@wwpdl.vnet.ibm.com

- This email address is monitored for feedback purposes only. 
- No support for any IBM products or services will be provided through this email. 
- To receive support, please follow the step-by-step instructions in the above "SUPPORT" section.