Planning for GPUDirect Storage
IBM Spectrum Scale support for GPUDirect Storage (GDS) enables a direct path between GPU memory and storage. You need to ensure that certain conditions are met before you start installing the feature.
For more information on GDS, see GPUDirect Storage support for IBM Spectrum Scale.
The following list provides the prerequisite for installing and using GDS:
- IBM Spectrum Scale version: Requires IBM Spectrum Scale 5.1.2. If you need to configure GDS for a remote file system, IBM Spectrum Scale 5.1.2 or later is required on all NSD servers of the remote cluster.
- Data path: GDS requires the NSD path for storage access. The storage must not be locally attached to the GDS clients.
- Storage: ESS and non-ESS storage servers are supported. The storage servers can be in the same or different cluster as the GDS clients.
- InfiniBand fabric: GDS requires Mellanox RDMA over InfiniBand between GDS clients and storage servers. RoCE is not supported.
- Adapters: Melannox ConectX-5 or newer adapters are required to use GDS with RoCE.
Supported hardware
The following list provides the hardware requirements:
- GDS clients: Intel x86 with a GPU model that supports GDS. For more details, see NVIDIA GDS documentation.
- Network: EDR or HDR InfiniBand.
- InfiniBand adapter: Mellanox CX4, CX5 or CX6. The CX4 firmware must be 12.27.4000 or higher.
Supported software for GDS clients
For information about the supported versions of the required software for GDS clients, see Components required for GDS in IBM Spectrum Scale FAQ in IBM® Documentation.
Supported software for GDS servers
The following list provides the software requirements:
- Any operating system that supports IBM Spectrum Scale can be used.
- MOFED: Use the Mellanox OFED stack for the OS. For details about driver versions, see Components required for GDS in IBM Spectrum Scale FAQ in IBM Documentation.