Configuring an AFM to cloud object storage fileset with Microsoft Azure Blob

Complete this procedure to set up an AFM to cloud object storage fileset with Microsoft Azure Blob.

Ensure that the following conditions are met:

Microsoft Azure Blob service is provisioned with storage accounts and respective containers under it.
When setting up storage accounts a preferred region is selected.
Access keys and endpoint information is available for associating it with an AFM to cloud object storage fileset.
An IBM Storage Scale cluster is set up and provisioned with gateway nodes that have connectivity to Microsoft Azure Blob endpoints.
gpfs.afm.cos*.rpm is present on gateway nodes.

Note: The mmafmcoskeys command needs bucket name, access key, and secret key parameters. For the Microsoft Azure Blob support, a bucket name is the container name, an access key (akey) is name of a storage account, and a secret key is the key obtained from the storage accounts security section.

Set up Microsoft Azure Blob services, storage account, and note the endpoint and respective keys.
Add the endpoint and keys into an IBM Storage Scale cluster by issuing the following command:
```
# mmafmcoskeys fileset1 set afmtest Ga3WljrCeUTROP0Ri2IAaUMb******K5husMUb01iQtd8xwDni/UMf64bhwtYHW5vv2sb0YgEK8R6+AStaYjRLw==
```
where,

fileset1

Is the name of a container that will be created on Microsoft Azure Blob.

afmtest

Is the name of a storage account under which the fileset1 container will be created.
Set up AFM to cloud object storage relationship.
```
# mmafmcosconfig fs1 fileset1 --endpoint https://afmtest.blob.core.windows.net --xattr --new-bucket fileset1 --mode iw --object-fs --azure  --acls --directory-object
```
where,

--azure

Creates the AFM to cloud object storage relationship with Microsoft Azure Blob as a backend.

fs1

A file system name where the fileset1 fileset will be created.

--directory-object

Supports the directory structure during the creation of the AFM to cloud object relationship.

Note: Microsoft Azure Blob creates the no_name file in directories for internal usage. Do not delete this file from the cloud object storage.

Verify the fileset parameters.

# mmlsfileset fs1 fileset1 --afm -L

A sample output is as follows:

Filesets in file system 'fs1':

Attributes for fileset fileset1:
=================================
Status                                  Linked
Path                                    /gpfs/fs1/fileset1
Id                                      207
Root inode                              5767171
Parent Id                               0
Created                                 Fri Oct 20 01:52:27 2023
Comment                                 
Inode space                             11      
Maximum number of inodes                100352
Allocated inodes                        100352
Permission change flag                  chmodAndSetacl
afm-associated                          Yes
Permission inherit flag                 inheritAclOnly
Target                                  https://afmtest.blob.core.windows.net:443/fileset1
Mode                                    independent-writer
File Lookup Refresh Interval            120
File Open Refresh Interval              120
Dir Lookup Refresh Interval             120
Dir Open Refresh Interval               120
Async Delay                             1 (default)
Last pSnapId                            0
Display Home Snapshots                  no
Parallel Read Chunk Size                0
Number of Gateway Flush Threads         4
Prefetch Threshold                      0 (default)
Eviction Enabled                        yes (default)
Parallel Write Chunk Size               0
IO Flags                                0x8280000 (afmObjectXattr,afmObjectDirectoryObj,afmObjectACL)
IO Flags2                               0x8280800 (afmObjectAZ)

Create some data.

Node1 0] dd if=/dev/urandom of=/gpfs/fs1/fileset1/object1 bs=256K count=4
4+0 records in
4+0 records out
1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787632 s, 133 MB/s

Node1 0] dd if=/dev/urandom of=/gpfs/fs1/fileset1/object2 bs=256K count=4
4+0 records in
4+0 records out
1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597403 s, 176 MB/s

Node1 0] dd if=/dev/urandom of=/gpfs/fs1/fileset1/object3 bs=256K count=4
4+0 records in
4+0 records out
1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683723 s, 153 MB/s

Verify that the created data is placed on the Microsoft Azure Blob storage.

# ls -lash /gpfs/fs1/fileset1/

A sample output is as follows:

total 3.3M
 512 drwxrws---     5 root root 4.0K Oct 20 01:56 .
256K drwxr-xr-x    13 root root 256K Oct 20 01:52 ..
 512 drwx------ 65535 root root 4.0K Oct 20 01:52 .afm
1.0M -rw-r--r--     1 root root 1.0M Oct 20 01:56 object1
1.0M -rw-r--r--     1 root root 1.0M Oct 20 01:56 object2
1.0M -rw-r--r--     1 root root 1.0M Oct 20 01:56 object3
 512 drwx------ 65535 root root 4.0K Oct 20 01:52 .pconflicts
 512 drwx------ 65535 root root 4.0K Oct 20 01:52 .ptrash
 512 dr-xr-xr-x     2 root root 8.0K Dec 31  1969 .snapshots

Check the data queue on the gateway node by issuing the following command:

# mmafmctl fs1 getstate

A sample output is as follows:

Fileset Name    Fileset Target                                Cache State          Gateway Node    Queue Length   Queue numExec 
------------    --------------                                -------------        ------------    ------------   ------------- 
fileset1        https://afmtest.blob.core.windows.net:443/fileset1 Active          c7f2n09         0              10

When all the data is pushed from the queue to Microsoft Azure Blob storage, it will be visible in containers. The following figure shows files present in the fileset1 container.