Complete this procedure to set up an AFM to cloud object storage fileset with Microsoft Azure Blob.
Ensure that the following conditions are met:
- Microsoft Azure Blob service is provisioned with storage accounts and
respective containers under it.
- When setting up storage accounts a preferred region is selected.
- Access keys and endpoint information is available for associating it with an AFM to cloud object storage fileset.
- An IBM Storage Scale cluster is set up and provisioned with
gateway nodes that have connectivity to Microsoft Azure Blob endpoints.
- gpfs.afm.cos*.rpm is present on gateway nodes.
Note: The mmafmcoskeys command needs bucket name, access key, and secret key
parameters. For the Microsoft Azure Blob support, a bucket name is the
container name, an access key (akey) is name of a storage account, and a secret key is the key
obtained from the storage accounts security section.
-
Set up Microsoft Azure Blob services, storage account, and note the
endpoint and respective keys.
- Add the endpoint and keys into an IBM Storage Scale
cluster by issuing the following command:
# mmafmcoskeys fileset1 set afmtest Ga3WljrCeUTROP0Ri2IAaUMb******K5husMUb01iQtd8xwDni/UMf64bhwtYHW5vv2sb0YgEK8R6+AStaYjRLw==
where,
- fileset1
- Is the name of a container that will be created on Microsoft Azure
Blob.
- afmtest
- Is the name of a storage account under which the fileset1 container will be created.
- Set up AFM to cloud object storage relationship.
# mmafmcosconfig fs1 fileset1 --endpoint https://afmtest.blob.core.windows.net --xattr --new-bucket fileset1 --mode iw --object-fs --azure --acls --directory-object
where,
- --azure
- Creates the AFM to cloud object storage relationship with Microsoft Azure Blob as a backend.
- fs1
- A file system name where the fileset1 fileset will be created.
- --directory-object
- Supports the directory structure during the creation of the AFM to cloud object
relationship.
Note: Microsoft Azure Blob creates the no_name
file in
directories for internal usage. Do not delete this file from the cloud object storage.
- Verify the fileset
parameters.
# mmlsfileset fs1 fileset1 --afm -L
A sample output is
as follows:
Filesets in file system 'fs1':
Attributes for fileset fileset1:
=================================
Status Linked
Path /gpfs/fs1/fileset1
Id 207
Root inode 5767171
Parent Id 0
Created Fri Oct 20 01:52:27 2023
Comment
Inode space 11
Maximum number of inodes 100352
Allocated inodes 100352
Permission change flag chmodAndSetacl
afm-associated Yes
Permission inherit flag inheritAclOnly
Target https://afmtest.blob.core.windows.net:443/fileset1
Mode independent-writer
File Lookup Refresh Interval 120
File Open Refresh Interval 120
Dir Lookup Refresh Interval 120
Dir Open Refresh Interval 120
Async Delay 1 (default)
Last pSnapId 0
Display Home Snapshots no
Parallel Read Chunk Size 0
Number of Gateway Flush Threads 4
Prefetch Threshold 0 (default)
Eviction Enabled yes (default)
Parallel Write Chunk Size 0
IO Flags 0x8280000 (afmObjectXattr,afmObjectDirectoryObj,afmObjectACL)
IO Flags2 0x8280800 (afmObjectAZ)
- Create some data.
Node1 0] dd if=/dev/urandom of=/gpfs/fs1/fileset1/object1 bs=256K count=4
4+0 records in
4+0 records out
1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787632 s, 133 MB/s
Node1 0] dd if=/dev/urandom of=/gpfs/fs1/fileset1/object2 bs=256K count=4
4+0 records in
4+0 records out
1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597403 s, 176 MB/s
Node1 0] dd if=/dev/urandom of=/gpfs/fs1/fileset1/object3 bs=256K count=4
4+0 records in
4+0 records out
1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683723 s, 153 MB/s
- Verify that the created data is placed on the Microsoft Azure Blob
storage.
# ls -lash /gpfs/fs1/fileset1/
A sample output is as
follows:
total 3.3M
512 drwxrws--- 5 root root 4.0K Oct 20 01:56 .
256K drwxr-xr-x 13 root root 256K Oct 20 01:52 ..
512 drwx------ 65535 root root 4.0K Oct 20 01:52 .afm
1.0M -rw-r--r-- 1 root root 1.0M Oct 20 01:56 object1
1.0M -rw-r--r-- 1 root root 1.0M Oct 20 01:56 object2
1.0M -rw-r--r-- 1 root root 1.0M Oct 20 01:56 object3
512 drwx------ 65535 root root 4.0K Oct 20 01:52 .pconflicts
512 drwx------ 65535 root root 4.0K Oct 20 01:52 .ptrash
512 dr-xr-xr-x 2 root root 8.0K Dec 31 1969 .snapshots
- Check the data queue on the gateway node by issuing the following
command:
# mmafmctl fs1 getstate
A sample output is as
follows:
Fileset Name Fileset Target Cache State Gateway Node Queue Length Queue numExec
------------ -------------- ------------- ------------ ------------ -------------
fileset1 https://afmtest.blob.core.windows.net:443/fileset1 Active c7f2n09 0 10
When all the data is pushed from the queue to Microsoft Azure Blob storage,
it will be visible in containers. The following figure shows files present in the
fileset1 container.