Summary of changes

• Added the afmResyncVer2 parameter to improve the replication performance. For more information, see AFM resync version 2.
• Added AFM NFSv4 support. For more information, see AFM Network File System version 4 support.
• Added a parameter to query uncached files on an AFM fileset. For more information, see the mmafmctl command.
• Added a parameter to query dirty files on an AFM fileset. For more information, see the mmafmctl command.
• Added parameters to query replication statistics and estimated time by using the --read-stats and --write-stats. For more information, see the mmafmctl command.
• Documented errors in the AFM error logs with a description. For more information, see Active File Management error logs.
• Both an AFM-DR secondary fileset and an SW fileset on a home target will not have any extra files. Because when the afmResyncVer2 parameter is enabled, the extra files are deleted. For more information, see Resync on SW filesets.
• With the updated afmRPO parameter you can disable afmRPO. For more information, see mmchfileset command.
• Introduced support of Ubuntu 22.04 as AFM Gateway. For more information, see AFM and AFM DR limitations.

• AFM to Cloud Object Storage read performance improvements by using multiple gateway nodes and mapping for large files. For more information, see AFM to cloud object storage parallel read data transfer.
• An access key and a secret key can contain special characters such as hyphen (-) or period (.). For more information see, mmafmcoskeys command.
• Added the --fast-readdir option and the afmObjectFastReaddir parameter for the read-dir performance improvement. For more information see, mmafmcosconfig command and mmchfileset command.

Display the fully qualified domain name (FQDN) of the specified nodes

Introduced the --fqdn parameter in the mmgetstate command to display the daemon fully qualified domain name (FQDN) of the specified nodes. The FQDN is the full name that is known to IBM Spectrum Scale at the time the node is added to the cluster. For more information, see mmgetstate command.

NUMA-aware RDMA adapter selection and NUMA-aware RDMA thread affinity

You can use the verbsNumaAffinity parameter of the mmchconfig command to enable NSD servers to select a NUMA-aware RDMA adapter for RDMA-based communication with NSD clients. This option also sets the RDMA thread's affinity to CPUs on the NUMA node to which the associated RDMA device is attached. For more information, see mmchconfig command.

Changing file system name is restricted if file audit logging or clustered watch folder is enabled for the file system

You cannot use the mmchfs command to change the file system name if file audit logging or clustered watch folder is enabled for the file system. For more information, see mmchfs command.

Support for GPUDirect Storage

Introduced the support for NVIDIA's GPUDirect Storage (GDS). This feature allows achieving low latencies and high throughput when reading data directly from an NSD server to a GPU buffer. It requires an InfiniBand fabric with GDS capable hardware and the CUDA environment that is installed on the GDS clients.

Both IBM Spectrum Scale client and NSD server must be running on IBM Spectrum Scale 5.1.2 to enable the support for GDS. For more information, see GPUDirect Storage support for IBM Spectrum Scale.

IBM Spectrum Scale 5.1.2 supports GDS Read operations. Writes (cuFileWrite) are supported in compatibility mode.

CCR is required to run the mmchconfig release=LATEST command

Cluster configuration by using primary and secondary server is discontinued. If the Cluster Configuration Repository (CCR) is not enabled, you cannot run the mmchconfig release=LATEST command. Use the mmchcluster --ccr-enable command to enable CCR in the cluster. For more information about the mmchconfig release=LATEST command, see mmchconfig command.

Support 4K disk sectors for IBM Spectrum Scale on Windows

Introduced 4K disk sector support for IBM Spectrum Scale on the Windows platform. Removed the restriction from the IBM Spectrum Scale FAQ. For more information about the updated FAQ question, see IBM Spectrum Scale support for 4K disk sectors.

Repair functionality of mmfsck command in online mode is deprecated

The repair functionality of mmfsck command in online mode is no longer available. The report-only operation still works in the online mode. For more information, see mmfsck command.

--metadata-block-size parameter of mmcrfs command is being deprecated

The --metadata-block-size parameter of the mmcrfs command can be used to specify a metadata block size that is different from the data block size. This option is being deprecated because it is no longer required to use for performance improvements for file systems with file system format 5.0.0 or later. This parameter will be removed in a future release. For more information, see mmcrfs command.

Display size of the LROC devices in the mmlsnsd command output

The mmlsnsd -L command output now shows the Size (MB) column for LROC devices. For more details, see mmlsnsd command.

File creation performance improvement

The performance of file creation has been improved by reducing the context switch between kernel and user space, by implementing the atomic_open inode operation.

mmap performance improvement

The performance of mmap when pre-fetching file data blocks has been improved.

DIO vectored read performance improvement

The performance of vectored reads in direct-IO mode has been improved with more efficient memory pool buffer locking.

New diskDown callback to inform about the reason for a disk failure

The diskDown callback is triggered on the file system manager when the status of a disk in a file system changes to down. The diskDown callback is similar to the diskFailure callback but provides the reason for triggering the diskDown callback. For more information, see mmaddcallback command.

Support for IBM Security Guardium Key Lifecycle Manager (GKLM) 4.1.0.1 (IF01)

IBM Spectrum Scale 5.1.2 supports IBM Security Guardium Key Lifecycle Manager (GKLM) 4.1.0.1 (IF01) or later. The older versions of GKLM are referred to as IBM Security Lifecycle Manager or SKLM in the documentation. The configuration information is the same for both GKLM and SKLM. For more information, see mmkeyserv command.

Clear QoS configuration automatically when system objects are deleted

You do not need to use the special option QOSSkipValidate=1 to override the QoS configuration validation. The following commands automatically clean up QoS data when those system objects are deleted:

• mmdelfileset (filesets)
• mmdelnode (nodes)
• mmdelnodeclass (node classes)
• mmauth (remote cluster names)
• mmdeldisk (pools)

For more information, see mmqos command.

O_TMPFILE support (Kernel version greater than 3.15 required)

IBM Spectrum Scale 5.1.2 now supports the use of the O_TMPFILE open flag. The O_TMPFILE is used in the open(2) call. This allows the creation of an unnamed temporary regular file. Anything written to the resulting file is lost when the last file descriptor is closed unless the file is given a name. To link the temporary file, linkat(2) or gpfs_linkat() calls can be used.

gpfs_linkat API (Linux only)

The gpfs_linkat API is defined in the gpfs.h header file. The gpfs_linkat() function links a file to a directory name, and atomically replaces the destination if the file exists. The gpfs_linkat API has a similar interface to the linkat(2) system call with the following differences:

• When a new path specifies an existing file, it is atomically replaced; EEXIST is never returned.
• AT_EMPTY_PATH does not require CAP_DAC_READ_SEARCH.

For more information, see gpfs_linkat() subroutine.

Fine-Grained Directory Locking (FGDL) for link

When concurrent hard links are created in a single directory from multiple nodes, FGDL is enabled to improve performance of link creation.

New parameters to retain the user changes in mmsysmon.conf parameters after an RPM update

Introduced a set of parameters in the mmchconfig command to ensure that the changes in some of the mmsysmon.conf parameters do not disappear after an RPM update. For more information about how to set value for these parameters, see mmchconfig command.

• POST Filesystems/{filesystemPath}/loopbackDevice/attach/ {fileName}/{nodeName}
• DELETE Filesystems/{filesystemPath}/loopbackDevice/detach/{fileName}/{nodeName}
• GET Filesystems/{filesystemPath}/loopbackDevice/{fileName}/{nodeName}
• GET filesystems/{filesystemName}/directory/{path}

Removed the 'The CTDB nodes file for CES' name from the IBM Spectrum Scale other configurations list mentioned underneath the sub-section Updating IP addresses or host names in other configurations.

Replaced option '--all' with '-a' that is used in mmces service command statement to failback or restore the SMB protocol configuration. For more information see, Failback or restore steps for the SMB protocol.

• Upgraded CES NFS-Ganesha to NFS V3-stable version. This upgrade adds more stability, fixes defects, and improves the NFS performance.
• Updated limitations for NFS Ganesha protocol. For more information, see General CES NFS Linux limitations.
• Introduced configuration parameters:DIR_CHUNK, READDIR_RES_SIZE, and SERVER_SCOPE. For more information, see mmnfs command.

From IBM Spectrum Scale 5.1.2, a security feature is introduced to limit the connections of performance monitoring sensors to a list of known IBM Spectrum Scale node IPs. For more information, see Configuring the performance monitoring tool.

For compatibility reasons on IBM Spectrum Scale 5.1.0.x and later on Linux 7.x (7.7 and later), a few Python 2 files are packaged and they might trigger dependency-related messages. In certain scenarios, Python 2.7 might also be required to be installed. Multiple versions of Python can co-exist on the same system. For more information, see the entry about mmadquery in Guidance for Red Hat Enterprise Linux 8.x on IBM Spectrum Scale nodes.

The Python code in IBM Spectrum Scale 5.0.y or earlier continues to be in Python 2.

• Added new IP addresses for esupport.ibm.com to consider during firewall configurations. For more information, see Firewall recommendations for call home and Installing call home.
• 

  Added a new option --temporary to the mmhealth command. For more information, see the mmhealth command.

  

• Added a new component GDS in the parameter list of the mmhealth node show and mmhealth cluster show commands. For more information, see the mmhealth command.

  For more information, see GPUDirect Storage support for IBM Spectrum Scale

• Added a new component, HEALTHCHECK, to improve the experience of users with Call Home capability by providing them a direct support from IBM. For more information, see Proactive system health alerts.

• New API sub-command added to the mmperfmon delete command, which can be used to delete expired API keys. For more information, see mmperfmon command and Configuring performance monitoring API keys.

• New events added for the following:
  • Call Home events
  • CES network events
  • File Audit Logging events
  • File system events
  • GPFS events
  • GUI events
  • Local Cache events
  • Network events
  • Watchfolder events

  For more information, see Events

• New commands are added to scheduled data upload list. For more information, see Scheduled data upload.

The following changes are made to the IBM Spectrum Scale Erasure Code Edition on IBM Z:

• Support for the IBM Spectrum Scale Erasure Code Edition on IBM Z15 server.

• Support for four storage nodes that are running in native LPARs.

• Support for up to three NVMe drives with a U.2 form factor per storage node.

• Support for HiperSockets network connection.

• Automatic mapping from the server’s drive slots to PCIe buses. There is no need to manually create an Enclosure Descriptor File (EDF).

For more information, see IBM Spectrum Scale ECE 5.1.2.

New commands

mmxcp

New structures

There are no new structures.

New subroutines

gpfs_linkat()

New user exits

There are no new user exits.

Changed commands

• gpfs.snap
• mmlsquota
• mmchfs
• mmaddcallback
• mmafmconfig
• mmafmcosconfig
• mmafmctl
• mmaudit
• mmbackup
• mmchcluster
• mmchconfig
• mmchfileset
• mmchfs
• mmcrcluster
• mmcrfs
• mmfsck
• mmgetstate
• mmhealth
• mmkeyserv
• mmlsnsd
• mmnetverify
• mmnfs
• mmperfmon
• mmqos
• mmsetquota
• mmsdrrestore
• mmuserauth
• mmxcp
• spectrumscale

Changed structures

There are no changed structures.

Changed subroutines

gpfs_fcntl()

Deleted commands

mmcesdr

Deleted structures

There are no deleted structures.

Deleted subroutines

There are no deleted subroutines.

New messages

6027-1829, 6027-1830, 6027-2417, 6027-2418, 6027-2419, 6027-3265, 6027-3335, 6027-4303,6027-4304, 6027-4305, 6027-4306, 6027-984, 6027-3411, 6027-3412, 6027-3413, 6027-3516, 6027-3951, 6027-3952, 6027-3953, 6027-4020, 6027-4021, 6027-4022, 6027-4023, 6027-4024, 6027-4212, 6027-4307, 6027-4308, 6027-4309, 6027-4310, and 6027-4311

Changed messages

6027-475, 6027-477, 6027-1311, 6027-1312, 6027-1323, 6027-1324, 6027-1325, 6027-2050, 6027-3245, 6027-3263, 6027-3265, and 6027-3328

Deleted messages

6027-1922

List of documentation changes in product guides and respective IBM Documentation sections

The following is a list of documentation changes including changes in topic titles, changes in placement of topics, and deleted topics:

Table 1. List of changes in documentation

Guide	IBM Documentation section	List of changes
Problem Determination Guide	Monitoring	Applied the following changes: Re-arranged the list of first three chapters: Monitoring system health by using IBM Spectrum Scale GUI Monitoring system health by using the mmhealth command Performance monitoring Changed following section headings: Set up event notifications to Event notifications Monitoring tip events to Monitoring tips by using GUI Monitoring thresholds to Monitoring thresholds by using GUI
	Troubleshooting	Removed the following topics: Protocols cluster disaster recovery issues Message queue events
		Message number changes: 6027-4305 changed to 6027-4304 6027-4306 changed to 6027-4305 6027-4307 changed to 6027-4306
Administration Guide	Configuring	Changed the topic title from Configuration parameters for AFM to Configuration parameters for AFM and AFM-DR. Merged all AFM and AFM-DR configuration parameters tables into a single table. Removed the following topics: Parallel data transfer configuration parameters for AFM Configuration parameters for AFM-based DR Parallel data transfer configuration parameters for AFM-based DR
	Administering	Removed the following topics: Creating the inband disaster recovery setup Creating the outband disaster recovery setup Performing failover for protocols cluster when primary cluster fails Performing failback to old primary for protocols cluster Performing failback to new primary for protocols cluster Backing up and restoring protocols and CES configuration information Updating protocols and CES configuration information
Command and Programming Reference Guide	Command reference	Removed mmcesdr command.
	Programming reference	Added the following API documents: PUT health/config/monitor/resume/{server} PUT health/config/monitor/pause/{server}
Concepts, Planning, and Installation Guide	Product overview	Changed the topic title from File audit logging events' descriptions to File audit logging events.