White Papers
Abstract
Understanding of SCSI Asymmetric Logical Unit Access (ALUA) enabled devices is critical for ensuring a properly working multi-path storage configuration. The purpose of this document is to explain the extended 'path_status' values of the 'lsmpio' command to help system administrators understand if a storage path is either "selected" or "non-selected". ALUA is important as it allows for storage arrays to advertise which paths are available and preferred for host I/O.
Content
Command Description
The 'lsmpio' command displays information that is related to AIX® MPIO storage devices. This command works only for devices that are controlled by path-control modules (PCMs) that are enabled for lsmpio support. As an example, to show the summary details of a disk device named 'hdisk1234', enter the following command:
# lsmpio -l hdisk1234
name path_id status path_status parent connection
===============================================================================
hdisk1234 0 Enabled Opt,Sel,Deg,Rsv fscsi0 500a098186a7d4ca,0008000000000000
hdisk1234 1 Enabled Non fscsi0 500a098196a7d4ca,0008000000000000
hdisk1234 2 Enabled Opt,Sel fscsi1 500a098186a7d4ca,0008000000000000
hdisk1234 3 Enabled Non fscsi1 500a098196a7d4ca,0008000000000000
If the lsmpio command is run without any flags or with the -l flag, it displays the path operational status.
Further information regarding the command can be found here.
Objective Description
The Path Status column reflects the status reported by the 'lspath' command, displaying values such as Enabled, Disabled, Failed, or Missing. In addition, the Extended Status field may include one or more three-letter abbreviations offering more detailed information on path status. Certain values might not clearly provide the information about the status meaning particularly in determining whether a path is selected or non-selected. When dealing with ALUA, clarifying the decision-making process for path selection remains a challenge.
Subject Exploration
In general there are three types of storage servers:
- Fully symmetrical storage array, where all LUNs receive service simultaneously through every storage port
- ALUA storage arrays like IBM’s Storage Virtualize family (IBM FlashSystem and IBM SAN Volume Controller), that can service host reads and writes on all paths but have the concept of preferred and non-preferred paths for load balancing and to improve cache efficiency
- Servers with fail-over controllers (like the old DS4k)
ALUA-based storage arrays use the SCSI ALUA messages to advertise which paths the multipath driver should use when reading and writing data. Depending on the configuration and implementation of the storage array, ALUA may be used to load-balance between ports or controllers or to ensure the multipath driver is using the ports with the best response time.
Essentially, the ALUA protocol offers a mechanism for ALUA-based storage devices to communicate their operational status to hosts. Hosts can then leverage this information to prioritize paths and make decisions related to failover and load balancing. To enable initiators to discern the most suitable targets for I/O operations, ports are organized into target port groups (TPGs). These TPGs are commonly utilized in conjunction with arrays that manage load balancing and failover within their array controllers.
Arrays provide the capability of path grouping, where each port within the same TPG shares a common port state. The potential port states include Active/Optimized, Active/Non-optimized, Standby, Unavailable, and In-Transition.
The Significance of Storage Controllers
The asymmetric access nature of the array dictates that the most efficient path to a LUN (Logical Unit or Volume Number) is channeled through one controller (or a controller subnet). This stands in contrast to the symmetrical access model, where multiple controllers can potentially assume ownership of a LUN, or alternatively, advertise the unavailability of certain paths. An optimal path is characterized by an I/O route to the LUN with minimal processing overhead through the one controller.
In contrast, the non-managing controller, also referred to as the proxy controller, is tasked with receiving I/O requests for a LUN it doesn't manage. In such cases, it becomes necessary for the proxy controller to forward the request to the managing controller for execution. After that, the managing controller transfers this response data to the non-managing controller’s cache so that the response data can be returned to the host through the controller/host ports to which the host initially sent the request.
When it comes to ALUAs managing controllers, there are two distinct models that dictate ownership. These models, known as implicit and explicit ALUA modes, have their own unique perspectives and functions.
- In the implicit transition mode, the array can assign and change the managing controller for the Logical Unit Number (LUN).
- Alternatively, in explicit transition mode, the responsibility lies with the host driver, granting it the capability to establish or modify ownership.
When using an ALUA-aware host alongside AIX PCM (MPIO), all the paths to a LUN are categorized as active preferred/non-preferred, or a device may indicate itself as a non-capable ALUA device. Multipath employs load balancing across active paths to a LUN and will resort to utilizing non-preferred paths if all active preferred paths fail.
Below are the possible values for the path_status field and an overview of the path selection process:
Beginning with the "Selected" (Sel) status, it indicates that the MPIO disk path has been designated for I/O operations when the 'lsmpio' command was executed. When a path is optimized (Opt), it means that it is connected to a preferred controller, and typically, this path is selected. If the status is non-optimized (Non), it implies that the path is communicating through a non-preferred controller and is selected if all optimized paths fail.
In a situations involving both active and passive controllers, we encounter (Act) Active and (Pas) Passive states, signifying whether a path is active or passive. In such scenarios, the PCM prioritizes and selects the Active path over the Passive one.
Changes in access characteristics resulting in statuses such as changing (Chg), transitioning (Trn), or standby (Sby) prevent path selection until the status is altered. (Rsv) indicates that the MPIO disk path experienced an unexpected reservation conflict. This value is observed on selected paths and might indicate a usage or configuration error, especially when multiple hosts access the same disk.
During Enhanced Error Handling (EEH) events, the PCM temporarily avoids the associated MPIO disk path. This precaution is particularly critical because, when the path is already marked as degraded (Deg) due to previous failures, any additional errors could escalate the situation, potentially resulting in a failed (Fai) status. In such cases, PCM avoids selecting these paths, emphasizing the importance of its strategy to prevent potential issues.
The status of the ports and the network link is also reflected in the path_status column, all of which identify a non-selected/preferred path:
- (Pfa) indicates a situation where the remote port has failed, leading the PCM to designate the path as (Fai) failed.
- (PCn) points to network congestion on the remote port used by the MPIO disk path.
- (PDg) signifies a degraded remote port.
- (LCn) indicates congestion on the link associated with the adapter due to a significant amount of data transmission.
These last three are self-explanatory and do not involve any selection process:
- (Deferred) This state occurs when a new path is added to a disk that is already open, but the path becomes accessible only after the disk is reopened.
- (Nav) This state occurs when a disk is unavailable, and its associated path cannot be utilized.
- (Clo) Indicates that the MPIO disk path is closed. A device is considered closed if all MPIO disk paths that are connected to the device are closed.
After reviewing this document, one should have received enough information about ALUA and AIX’s PCM (MPIO) selection process. It's essential to note that configuring ALUA isn't within the typical responsibilities of AIX administrators; instead, this task is within the scope of the Storage admins, who are responsible for its implementation and ongoing management.
Related Information
[{"Type":"MASTER","Line of Business":{"code":"LOB08","label":"Cognitive Systems"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SWG10","label":"AIX"},"ARM Category":[{"code":"a8m0z000000cvzSAAQ","label":"Device Configuration"},{"code":"a8m0z000000cw0FAAQ","label":"IO Device Drivers"}],"ARM Case Number":"","Platform":[{"code":"PF002","label":"AIX"}],"Version":"All Versions"}]
Was this topic helpful?
Document Information
Modified date:
19 March 2024
UID
ibm17124296