Summary of changes

This topic summarizes changes to the IBM Spectrum Scale licensed program and the IBM Spectrum Scale library. Within each topic, these markers ( Start of change End of change ) surrounding text or illustrations indicate technical changes or additions that are made to the previous edition of the information.

Summary of changes
for IBM Spectrum Scale 5.1.3
as updated, April 2022

This release of the IBM Spectrum Scale licensed program and the IBM Spectrum Scale library includes the following improvements. All improvements are available after an upgrade, unless otherwise specified.

Important: IBM Spectrum Scale 5.1.x supports Python 3.6 or later. It is recommended that Python 3.6 is installed through the OS package manager (For example, yum install python3). If you install Python 3.6 by other means, unexpected results might occur, such as failure to install gpfs.base for prerequisite checks, and workarounds might be required.

AFM and AFM DR-related changes

Introduced support of afmObjKeyExpiration and afmMUAutoRemove parameters. For more information, see Configuration parameters for AFM and AFM-DR.

AFM to Cloud Object Storage

Introduced AFM to cloud object storage manual-updates (MU) mode to support manual replication of the files by using ILM policies or user provided file list. The MU mode works with all the supported cloud object storage backends. For more information, see Fileset operation modes.
Added support to control the timeout of cloud object storage key expiration which automatically refreshes the keys from /var/mmfs/etc/mmuid2keys file after the expiration timeout value. Introduced a new cluster level parameter, afmObjKeyExpiration. For more information, see mmchconfig command.
Added support of Azure Blob object storage by using MinIO as S3 gateway AFM to cloud object storage fileset. AFM to cloud object storage does not support Azure blob storage directly but through S3 gateway in between. For more information, see AFM to cloud object storage by using Azure Blob storage and Configuring AFM to cloud object storage for Azure Blob storage by using MinIO as S3 gateway.
Support to enable manual eviction of cached data from partially fetched files by using --file parameter with the mmafmctl command. For more information, see Cache eviction.
Added support to use STS token for S3, IBM COS and AWS S3. For more information, see Configuring AFM to cloud object storage fileset by using use-keys and STS token.
Support of generating statistics, report of AFM to cloud object storage download and upload activities. For more information, see AFM to COS upload and download statistics.
The afmDIO parameter default value is set based on the protocol used between the AFM cache and the AFM home cluster. With this change, now the user is not expected to set it to the proper value. For more information, see The backend protocol - NFS versus NSD.

Big data and analytics changes

For information on changes in IBM Spectrum Scale Big Data and Analytics support and HDFS protocol, see Big Data and Analytics - summary of changes.

Clustered watch folder changes

Upgraded librdkafka version to 1.8.0 for producing to the remote Kafka sink.
Added new attributes for clustered watch folder. For more information, see JSON attributes in clustered watch folder.

File audit logging changes

Added new attributes for file audit logging. For more information, see JSON attributes in file audit logging.
File audit logging now supports updates to the filesets under audit using the new mmaudit update --enable-filesets and --disable-filesets. There is a limit of 20 filesets for file systems prior to 5.1.3 (27.0 file system version). For more information, see Enabling or skipping filesets with file audit logging.

IBM Spectrum Scale Container Storage Interface driver changes

For information on changes in the IBM Spectrum Scale Container Storage Interface driver, see IBM Spectrum Scale Container Storage Interface driver - Summary of changes.

IBM Spectrum Scale Erasure Code Edition changes

For information on changes in the IBM Spectrum Scale Erasure Code Edition, see IBM Spectrum Scale Erasure Code Edition - Summary of changes.

File system core improvements

Automatically flush buffers to disk when files are closed: You can set buffers to be automatically flushed to disk when closing files that were opened for write operations. For more information, see mmcrfs command, mmchfs command, and mmlsfs command.
Descriptions updated for special values DELETE and DEFAULT: The mmchconfig command supports DELETE and DEFAULT as special values for the Attribute=Value pair configuration. For more information, see mmchconfig command.
Support extended for 64-bit inode count variables: The gpfs_quotactl() subroutine is enhanced to support 64 bit inode count variables with the gpfs_quotaInfo64_t structure and the Q_INODE64BITS flag. Both mmsetquota and mmedquota allow setting the inode limit to >2B. For more information, see gpfs_quotactl() subroutine.
--filesetdf enhanced to report inode capacity and usage at independent fileset level: In IBM Spectrum Scale 5.1.3 if quota is enabled and limits are not defined then the df command reports both inode space capacity and usage at the independent fileset-level. For more information, see mmcrfs command, mmchfs command, and mmlsfs command.
Fine-grained fileset quiesce: The commands that are used to unlink and change fileset are enhanced with a change in their quiesce granularity, which is now done at the independent fileset level and not for the whole file system.
Fine-grained snapshot quiesce: The snapshot creation and deletion commands against individual fileset are enhanced with a change in their quiesce granularity. The fileset snapshot quiesce is done at the independent fileset level and not for the whole file system.
GDS improvements: IBM Spectrum Scale supports Nvidia's GDS over RDMA over Converged Ethernet (RoCE). It allows to achieve low latencies and high throughput when reading data from an NSD server pagepool into a GPU buffer. It requires a high-speed Ethernet fabric with GDS capable hardware and the CUDA environment must be installed on GDS clients.; IBM Spectrum Scale supports recovery for failing GDS RDMA operations by returning the failed read request to CUDA and CUDA retries the failed read request in the compatibility mode.; The mmdiag command is enhanced to print diagnostic information for GDS.
Resolved CCR limitation for running the mmfsctl command: Removed the restriction for running the mmfsctl command on file system disks that are configured as CCR tiebreaker disks. For more information, see mmfsctl command.
Support for IBM Security Guardium Key Lifecycle Manager (GKLM) 4.1.1: IBM Spectrum Scale 5.1.3 supports IBM Security Guardium Key Lifecycle Manager (GKLM) 4.1.1. The older versions of GKLM are referred to as IBM Security Lifecycle Manager or SKLM in the documentation. The configuration information is the same for both GKLM and SKLM. For more information, see mmkeyserv command.
New attribute to improve the write performance of workloads for appending data: The mmchconfig command is enhanced with the appendShipEnabled attribute to improve the write performance of workloads that involve appending small chunks of data concurrently to a shared file. For more information, see mmchconfig command.
New attribute to specify whether the ts* commands are allowed to use the remote TCP/IP connection: The mmchconfig command is enhanced with the tscCmdAllowRemoteConnections attribute to specify whether the ts* commands are allowed to use the remote TCP/IP connections when communicating with the local or other mmfsd daemons.
Valid values are yes and no. For new clusters with a minimum release level of 5.1.3 or later, the default value of this parameter is no. Otherwise, the default value of this parameter is yes. The system administrator can change it to no when the minimum release level is changed to 5.1.3 or higher. The recommended value for the tscCmdAllowRemoteConnections parameter is no.

The scope of the tscCmdAllowRemoteConnections parameter is local to a cluster and it has the same value on all the nodes in the cluster. However, when the parameter value is set to no, it can impact the communication between the ts* commands and the mmfsd daemons in remote clusters that service the requests.
Warning: Set the tscCmdAllowRemoteConnections parameter value to yes in the home cluster if any remote cluster is running a version of IBM Spectrum Scale older than 5.1.3.
This value is useful to avoid the Operation not permitted errors from mm* commands that are invoked on the remote cluster’s nodes redirected to mmfsd daemons running on nodes in the home cluster. For more information, see mmchconfig command.

New permission inherit mode for filesets: The mmcrfileset and mmchfileset commands are enhanced with the option --allow-permission-inherit to control how the permissions of a newly created object are inherited from its parent directory that has an NFSv4 ACL. For more information, see mmcrfileset command and mmchfileset command.

Installation toolkit changes

Support for parallel offline upgrade of all nodes in the cluster to reduce the upgrade duration compared to doing upgrades in a serial manner. For more information, see Parallel offline upgrade with the installation toolkit.
Support for Ansible 2.10.x. For more information, see Preparing to use the installation toolkit.

Management API changes

The following endpoints are added:

GET gpfs/trace/list
GET gpfs/trace/status
GET gpfs/trace/snapPath
GET filesystems/{filesystemName}/filesets/{filesetName}/snapshots/latest
GET Nodes/{name}/info/deadlock:
GET Nodes/{name}/info/memory
GET Nodes/{name}/info/iohist:
GET Nodes/{name}/info/waiters
PUT filesystems/{filesystemName}/restripefs
PUT gpfs/trace/set
PUT gpfs/trace/stop
POST gpfs/trace/start
DELETE gpfs/trace/{snapPath}
GET filesystems/{filesystemName}/directory/{path}

The following endpoints are modified:

PUT /scalemgmt/v2/health/config/monitor/pause/{node}
PUT /scalemgmt/v2/health/config/monitor/resume/{node}
DELETE filesystems/{filesystemName}/directory/{path}
DELETE filesystems/{filesystemName}/filesets/{filesetName}
PUT smb/shares/shareName

For more information, see IBM Spectrum Scale management API endpoints.

Management GUI changes

The Cache table on the Active File Management page, includes columns that provides information on the estimated time that is needed to refresh or flush queues during recovery or resync operations.
Truststore CA Certificates no longer require manual backups to be done.

SMB changes

Important: SMB 4.15 is updated with major design changes with respect to the security context from its earlier releases. The software design for the file and directory access in a file system is changed gradually across all the modules by replacing the path based access to handle based access by using O_PATH implementation. Therefore, this design change might cause some observable latency thus affecting the performance. In the upcoming service packs, the latency is going to be considerably reduced and the performance is going to be improved.

Introduced force user and force group parameter on SMB shares in mmsmb command. For more information, see mmsmb command.
Introduced hosts allow and hosts deny parameters on SMB shares in the mmsmb command. For more information, see mmsmb command.
Introduced a fix to resolve an error message in listing smb exports with SMB 4.15 and IBM Spectrum Scale 5.1.3. Updated the field smb encrypt to server smb encrypt. For more information, see Planning for protocol data security.

NFS changes

Updated limitations for NFS Ganesha protocol. For more information, see General CES NFS Linux limitations. End of change

Protocols authentication changes

Introduced support for anonymous bind for LDAP file authentication. Updated the --enable-anonymous-bind parameter in the mmuserauth command. For more information, see mmuserauth command.

Python-related changes

From IBM Spectrum Scale release 5.1.0, all Python code in the IBM Spectrum Scale product is converted to Python 3. The minimum supported Python version is 3.6.

For compatibility reasons on IBM Spectrum Scale 5.1.0.x and later on Red Hat Enterprise Linux 7.x (7.7 and later), a few Python 2 files are packaged and they might trigger dependency-related messages. In certain scenarios, Python 2.7 might also be required to be installed. Multiple versions of Python can co-exist on the same system. For more information, see the entry about mmadquery in Guidance for Red Hat Enterprise Linux 8.x on IBM Spectrum Scale nodes.

The Python code in IBM Spectrum Scale 5.0.y or earlier continues to be in Python 2.

Tip:

IBM Spectrum Scale 5.1.x.x uses Python 3 code, and it runs best with operating systems that also use Python 3 internally such as Red Hat Enterprise Linux 8.x, SLES 15, and Ubuntu 20.04.
IBM Spectrum Scale 5.0.x.x uses Python 2 code, and it runs best with operating systems that also use Python 2 internally, such as Red Hat Enterprise Linux 7.x.

System health changes

Added a new enhancement to the call home feature as the Unified call home feature wherein a user can configure the call home settings on IBM Spectrum Scale or on Elastic Storage Server (ESS). For more information, see Benefits of enabling call home.
Added a new sub-component --component to the gpfs.snap command, which allows collecting only HDFS NN/DN snap and related Hadoop information, and not all of the gpfs.snap information. For more information, see gpfs.snap command.
Added a new option --server to the mmcallhome command. For more information, see the mmcallhome command.
Added monitoring on stretch cluster with the new component STRETCHCLUSTER. For more information, see the mmhealth command.
New events added for the following:
- AFM

Commands, data types, and programming APIs

The following section lists the modifications to the documented commands, structures, and subroutines:

New commands

There are no new commands.

New structures

There are no new structures.

New subroutines

There are no new subroutines.

New user exits

There are no new user exits.

Changed commands

gpfs.snap
mmafmcosaccess
mmafmcosconfig
mmafmcosctl
mmafmctl
mmaudit
mmbackup
mmcallhome
mmchfs
mmchconfig
mmchfileset
mmchnode
mmcrcluster
mmcrfileset
mmcrfs
mmdefedquota
mmdiag
mmedquota
mmfsctl
mmhealth
mmlsfs
mmlsquota
mmnetverify
mmperfmon
mmqos
mmsetquota
mmshutdown
mmsmb
mmuserauth
mmxcp

Changed structures

There are no changed structures.

Changed subroutines

gpfs_quotactl

Deleted commands

There are no deleted commands.

Deleted structures

There are no deleted structures.

Deleted subroutines

There are no deleted subroutines.

Messages

The following are the new, changed, and deleted messages:

New messages: 6027-984, 6027-1323, 6027-1324, 6027-1325, 6027-3411, 6027-3516, 6027-3951, 6027-3952, 6027-3953, 6027-4020, 6027-4021, 6027-4022, 6027-4023, 6027-4024, 6027-4307, 6027-4308, 6027-4309, 6027-4310, and 6027-4311.
Changed messages: 6027-475 and 6027-477.
Deleted messages: None.

Documentation changes

List of documentation changes in product guides and respective IBM Documentation sections

The following is a list of documentation changes including changes in topic titles, changes in placement of topics, and deleted topics:

Table 1. List of changes in documentation
Guide	IBM Documentation section	List of changes
Problem Determination Guide	Monitoring	Added a new heading Monitoring networks by using RPC statistics for an existing content in the Network performance monitoring chapter.
Problem Determination Guide	Monitoring	Added a new topic Inspecting call home data uploads in the Understanding call home chapter.
Concepts, Planning, and Installation Guide	Installing	Changed topic title Installing and upgrading AFM-based Disaster Recovery to Installing AFM Disaster Recovery.
Concepts, Planning, and Installation Guide	Planning	Changed topic title Impact of enabling and disabling protocols on authentication to Impacts of authentication on enabling and disabling protocols.