Monitoring S3 (NooBaa) performance with the mmperfmon query command

Using the mmperfmon query command user can monitor and collect the S3 performance statistics data, which is being processed by dasproxy and pmcollector.

There are different types of operations are executed on S3 buckets and its objects under NooBaa. The operations are categorized the following three types:
  • s3io

    Indicates the I/O operations includes read and write operation statistics.

  • s3ops

    Contains different types of S3 operations, can be performed on buckets and its objects.

  • s3fsops

    It is a file system operations, which is performed by internal processes, based on the S3 operations.

I/O operations

  • read operation
    Metrics can be fetch with the following statistics for the read operation:
    • s3op_count
    • s3op_size
  • write operation
    Metrics can be fetch with the following statistics for the write operation:
    • s3op_count
    • s3op_size

The mmperfmon command allows the system administrator to collect performance statistics from the scale-core pod.

Example to login into scale-core pod to use the mmperfmon command:
# oc -n ibm-spectrum-scale get pods -o wide
Query output:
NAME                                                   READY   STATUS    RESTARTS   AGE    IP             NODE                                     NOMINATED NODE   READINESS GATES
ibm-spectrum-scale-gui-0                               4/4     Running   0          4d5h   10.128.5.239   master2.jfmocp4.pokprv.stglabs.ibm.com   <none>           <none>
ibm-spectrum-scale-gui-1                               4/4     Running   0          4d5h   10.128.1.105   master0.jfmocp4.pokprv.stglabs.ibm.com   <none>           <none>
ibm-spectrum-scale-noobaamonitoring-69b8b94db8-b4mj4   2/2     Running   0          20h    10.128.4.91    master2.jfmocp4.pokprv.stglabs.ibm.com   <none>           <none>
ibm-spectrum-scale-pmcollector-0                       2/2     Running   0          4d5h   10.128.5.242   master2.jfmocp4.pokprv.stglabs.ibm.com   <none>           <none>
ibm-spectrum-scale-pmcollector-1                       2/2     Running   0          4d5h   10.128.1.107   master0.jfmocp4.pokprv.stglabs.ibm.com   <none>           <none>
master0                                                2/2     Running   0          4d5h   10.28.20.32    master0.jfmocp4.pokprv.stglabs.ibm.com   <none>           <none>
master1                                                2/2     Running   0          4d5h   10.28.20.33    master1.jfmocp4.pokprv.stglabs.ibm.com   <none>           <none>
master2                                                2/2     Running   0          4d4h   10.28.20.24    master2.jfmocp4.pokprv.stglabs.ibm.com   <none>           <none>
Example with the mmperfmon query command for the read operation:
# oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size --filter "operation=read" -b 60 -n 3

Output:
===================================================================================
Legend:
 1:     dasS3.monitoring|DasS3|s3io|read|s3op_count
 2:     dasS3.monitoring|DasS3|s3io|read|s3op_size

Row           Timestamp s3op_count s3op_size
  1 2023-04-11-07:24:00          0         0
  2 2023-04-11-07:25:00          3       108
  3 2023-04-11-07:26:00          0         0
Example with the mmperfmon query for the write operation:
# oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size --filter "operation=write" -b 60 -n 3
Query output:
Legend:
 1:     dasS3.monitoring|DasS3|s3io|write|s3op_count
 2:     dasS3.monitoring|DasS3|s3io|write|s3op_size

Row           Timestamp s3op_count s3op_size
  1 2023-04-11-07:24:00          0         0
  2 2023-04-11-07:25:00          3       108
  3 2023-04-11-07:26:00          0         0

S3 operations

For all S3 operations metrics statistics can be fetch with these six values:
  • s3op_count
  • s3op_size
  • s3op_error_count
  • s3op_avg_time
  • s3op_min_time
  • s3op_max_time

create_bucket

To collect the create bucket statistics using the mmperfmon query, issue the command as follows:
# oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size,s3op_error_count,s3op_avg_time,s3op_min_time,s3op_max_time --filter "operation=create_bucket" -b 60 -n 3
Query output:
Legend:
 1:     dasS3.monitoring|DasS3|s3ops|create_bucket|s3op_count
 2:     dasS3.monitoring|DasS3|s3ops|create_bucket|s3op_size
 3:     dasS3.monitoring|DasS3|s3ops|create_bucket|s3op_error_count
 4:     dasS3.monitoring|DasS3|s3ops|create_bucket|s3op_avg_time
 5:     dasS3.monitoring|DasS3|s3ops|create_bucket|s3op_min_time
 6:     dasS3.monitoring|DasS3|s3ops|create_bucket|s3op_max_time

Row           Timestamp s3op_count s3op_size s3op_error_count s3op_avg_time s3op_min_time s3op_max_time
  1 2023-04-11-07:51:00          0         0                0             0             0             0
  2 2023-04-11-07:52:00          4         0                0           172            31           589
  3 2023-04-11-07:53:00          0         0                0             0             0             0
For file system operations use the following command:
# oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size,s3op_error_count,s3op_avg_time,s3op_min_time,s3op_max_time --filter "operation=mkdir" -b 60 -n 3
Query output:
Legend:
 1:     dasS3.monitoring|DasS3|s3fsops|mkdir|s3op_count
 2:     dasS3.monitoring|DasS3|s3fsops|mkdir|s3op_size
 3:     dasS3.monitoring|DasS3|s3fsops|mkdir|s3op_error_count
 4:     dasS3.monitoring|DasS3|s3fsops|mkdir|s3op_avg_time
 5:     dasS3.monitoring|DasS3|s3fsops|mkdir|s3op_min_time
 6:     dasS3.monitoring|DasS3|s3fsops|mkdir|s3op_max_time

Row           Timestamp s3op_count s3op_size s3op_error_count s3op_avg_time s3op_min_time s3op_max_time
  1 2023-04-11-07:51:00          0         0                0             0             0             0
  2 2023-04-11-07:52:00          4         0                0          3004           442         10562
  3 2023-04-11-07:53:00          0         0                0             0             0             0

list_buckets

To collect the create bucket statistics using the mmperfmon query, issue the command as follows:
# oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size,s3op_error_count,s3op_avg_time,s3op_min_time,s3op_max_time --filter "operation=list_buckets" -b 60 -n 3
Query output:
Legend:
 1:     dasS3.monitoring|DasS3|s3ops|list_buckets|s3op_count
 2:     dasS3.monitoring|DasS3|s3ops|list_buckets|s3op_size
 3:     dasS3.monitoring|DasS3|s3ops|list_buckets|s3op_error_count
 4:     dasS3.monitoring|DasS3|s3ops|list_buckets|s3op_avg_time
 5:     dasS3.monitoring|DasS3|s3ops|list_buckets|s3op_min_time
 6:     dasS3.monitoring|DasS3|s3ops|list_buckets|s3op_max_time

Row           Timestamp s3op_count s3op_size s3op_error_count s3op_avg_time s3op_min_time s3op_max_time
  1 2023-04-11-09:49:00          0         0                0             0             0             0
  2 2023-04-11-09:50:00          1         0                0             0             0             0
  3 2023-04-11-09:51:00          0         0                0             0             0             0

delete_bucket

  • S3 operations
    To collect the delete bucket S3 operations statistics using the mmperfmon query, issue the command as follows:
    # oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size,s3op_error_count,s3op_avg_time,s3op_min_time,s3op_max_time --filter "operation=delete_bucket" -b 60 -n 3
    Query output:
    Legend:
 1:     dasS3.monitoring|DasS3|s3ops|delete_bucket|s3op_count
 2:     dasS3.monitoring|DasS3|s3ops|delete_bucket|s3op_size
 3:     dasS3.monitoring|DasS3|s3ops|delete_bucket|s3op_error_count
 4:     dasS3.monitoring|DasS3|s3ops|delete_bucket|s3op_avg_time
 5:     dasS3.monitoring|DasS3|s3ops|delete_bucket|s3op_min_time
 6:     dasS3.monitoring|DasS3|s3ops|delete_bucket|s3op_max_time

Row           Timestamp s3op_count s3op_size s3op_error_count s3op_avg_time s3op_min_time s3op_max_time
  1 2023-04-11-08:10:00          0         0                0             0             0             0
  2 2023-04-11-08:11:00          4         0                0           147            29           490
  3 2023-04-11-08:12:00          0         0                0             0             0             0
  • File system operations
    To collect the delete bucket file system operations statistics using the mmperfmon query, issue the command as follows:
    # oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size,s3op_error_count,s3op_avg_time,s3op_min_time,s3op_max_time --filter "operation=stat" -b 60 -n 3
    Query output:
    Legend:
 1:     dasS3.monitoring|DasS3|s3fsops|stat|s3op_count
 2:     dasS3.monitoring|DasS3|s3fsops|stat|s3op_size
 3:     dasS3.monitoring|DasS3|s3fsops|stat|s3op_error_count
 4:     dasS3.monitoring|DasS3|s3fsops|stat|s3op_avg_time
 5:     dasS3.monitoring|DasS3|s3fsops|stat|s3op_min_time
 6:     dasS3.monitoring|DasS3|s3fsops|stat|s3op_max_time

Row           Timestamp s3op_count s3op_size s3op_error_count s3op_avg_time s3op_min_time s3op_max_time
  1 2023-04-11-08:10:00          0         0                0             0             0             0
  2 2023-04-11-08:11:00          8         0                0            36            12            57
  3 2023-04-11-08:12:00          0         0                0             0             0             0
    Sample query:
    # oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size,s3op_error_count,s3op_avg_time,s3op_min_time,s3op_max_time --filter "operation=realpath" -b 60 -n 6
    Query output:
    Legend:
 1:     dasS3.monitoring|DasS3|s3fsops|realpath|s3op_count
 2:     dasS3.monitoring|DasS3|s3fsops|realpath|s3op_size
 3:     dasS3.monitoring|DasS3|s3fsops|realpath|s3op_error_count
 4:     dasS3.monitoring|DasS3|s3fsops|realpath|s3op_avg_time
 5:     dasS3.monitoring|DasS3|s3fsops|realpath|s3op_min_time
 6:     dasS3.monitoring|DasS3|s3fsops|realpath|s3op_max_time

Row           Timestamp s3op_count s3op_size s3op_error_count s3op_avg_time s3op_min_time s3op_max_time
  1 2023-04-11-08:10:00          0         0                0             0             0             0
  2 2023-04-11-08:11:00          4         0                0            61            23           115
  3 2023-04-11-08:12:00          0         0                0             0             0             0
    Sample query:
    # oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size,s3op_error_count,s3op_avg_time,s3op_min_time,s3op_max_time --filter "operation=checkaccess" -b 60 -n 6
    Query output:
    Legend:
 1:     dasS3.monitoring|DasS3|s3fsops|checkaccess|s3op_count
 2:     dasS3.monitoring|DasS3|s3fsops|checkaccess|s3op_size
 3:     dasS3.monitoring|DasS3|s3fsops|checkaccess|s3op_error_count
 4:     dasS3.monitoring|DasS3|s3fsops|checkaccess|s3op_avg_time
 5:     dasS3.monitoring|DasS3|s3fsops|checkaccess|s3op_min_time
 6:     dasS3.monitoring|DasS3|s3fsops|checkaccess|s3op_max_time

Row           Timestamp s3op_count s3op_size s3op_error_count s3op_avg_time s3op_min_time s3op_max_time
  1 2023-04-11-08:10:00          0         0                0             0             0             0
  2 2023-04-11-08:11:00          4         0                0           141            64           276
  3 2023-04-11-08:12:00          0         0                0             0             0             0
    Sample query:
    # oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size,s3op_error_count,s3op_avg_time,s3op_min_time,s3op_max_time --filter "operation=readdir" -b 60 -n 6
    Query output:
    Legend:
 1:     dasS3.monitoring|DasS3|s3fsops|readdir|s3op_count
 2:     dasS3.monitoring|DasS3|s3fsops|readdir|s3op_size
 3:     dasS3.monitoring|DasS3|s3fsops|readdir|s3op_error_count
 4:     dasS3.monitoring|DasS3|s3fsops|readdir|s3op_avg_time
 5:     dasS3.monitoring|DasS3|s3fsops|readdir|s3op_min_time
 6:     dasS3.monitoring|DasS3|s3fsops|readdir|s3op_max_time

Row           Timestamp s3op_count s3op_size s3op_error_count s3op_avg_time s3op_min_time s3op_max_time
  1 2023-04-11-08:10:00          0         0                0             0             0             0
  2 2023-04-11-08:11:00          8         0                0           151            34           354
  3 2023-04-11-08:12:00          0         0                0             0             0             0
    Sample query:
    # oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size,s3op_error_count,s3op_avg_time,s3op_min_time,s3op_max_time --filter "operation=rmdir" -b 60 -n 6
    Query output:
    Legend:
 1:     dasS3.monitoring|DasS3|s3fsops|rmdir|s3op_count
 2:     dasS3.monitoring|DasS3|s3fsops|rmdir|s3op_size
 3:     dasS3.monitoring|DasS3|s3fsops|rmdir|s3op_error_count
 4:     dasS3.monitoring|DasS3|s3fsops|rmdir|s3op_avg_time
 5:     dasS3.monitoring|DasS3|s3fsops|rmdir|s3op_min_time
 6:     dasS3.monitoring|DasS3|s3fsops|rmdir|s3op_max_time

Row           Timestamp s3op_count s3op_size s3op_error_count s3op_avg_time s3op_min_time s3op_max_time
  1 2023-04-11-08:10:00          0         0                0             0             0             0
  2 2023-04-11-08:11:00          4         0                0           666           318          1158
  3 2023-04-11-08:12:00          0         0                0             0             0             0

upload_object

  • I/O operations
    To collect the upload object I/O operations statistics using the mmperfmon query, issue the command as follows:
    # oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size,s3op_error_count,s3op_avg_time,s3op_min_time,s3op_max_time --filter "operation=write" -b 60 -n 3
    Query output
    Legend:
 1:     dasS3.monitoring|DasS3|s3io|write|s3op_count
 2:     dasS3.monitoring|DasS3|s3io|write|s3op_size
 3:     dasS3.monitoring|DasS3|s3io|write|s3op_error_count
 4:     dasS3.monitoring|DasS3|s3io|write|s3op_avg_time
 5:     dasS3.monitoring|DasS3|s3io|write|s3op_min_time
 6:     dasS3.monitoring|DasS3|s3io|write|s3op_max_time

Row           Timestamp s3op_count s3op_size s3op_error_count s3op_avg_time s3op_min_time s3op_max_time
  1 2023-04-11-07:24:00          0         0                0             0             0             0
  2 2023-04-11-07:25:00          3       108                0             0             0             0
  3 2023-04-11-07:26:00          0         0                0             0             0             0
  • S3 operations
    To collect the upload object S3 operations statistics using the mmperfmon query, issue the command as follows:
    # oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size,s3op_error_count,s3op_avg_time,s3op_min_time,s3op_max_time --filter "operation=upload_object" -b 60 -n 3
    Query output:
    Legend:
 1:     dasS3.monitoring|DasS3|s3ops|upload_object|s3op_count
 2:     dasS3.monitoring|DasS3|s3ops|upload_object|s3op_size
 3:     dasS3.monitoring|DasS3|s3ops|upload_object|s3op_error_count
 4:     dasS3.monitoring|DasS3|s3ops|upload_object|s3op_avg_time
 5:     dasS3.monitoring|DasS3|s3ops|upload_object|s3op_min_time
 6:     dasS3.monitoring|DasS3|s3ops|upload_object|s3op_max_time

Row           Timestamp s3op_count s3op_size s3op_error_count s3op_avg_time s3op_min_time s3op_max_time
  2 2023-04-11-07:24:00          0         0                0             0             0             0
  3 2023-04-11-07:25:00          3         0                0            29            14            41
  4 2023-04-11-07:26:00          0         0                0             0             0             0
  • File system operations
    To collect the upload object file system operations statistics using the mmperfmon query, issue the command as follows:
    # oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size,s3op_error_count,s3op_avg_time,s3op_min_time,s3op_max_time --filter "operation=stat" -b 60 -n 3
    Query output:
    Legend:
 1:     dasS3.monitoring|DasS3|s3fsops|stat|s3op_count
 2:     dasS3.monitoring|DasS3|s3fsops|stat|s3op_size
 3:     dasS3.monitoring|DasS3|s3fsops|stat|s3op_error_count
 4:     dasS3.monitoring|DasS3|s3fsops|stat|s3op_avg_time
 5:     dasS3.monitoring|DasS3|s3fsops|stat|s3op_min_time
 6:     dasS3.monitoring|DasS3|s3fsops|stat|s3op_max_time

Row           Timestamp s3op_count s3op_size s3op_error_count s3op_avg_time s3op_min_time s3op_max_time
  1 2023-04-11-07:24:00          0         0                0             0             0             0
  2 2023-04-11-07:25:00          6         0                0            35            18            52
  3 2023-04-11-07:26:00          0         0                0             0             0             0
    Sample query:
    # oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size,s3op_error_count,s3op_avg_time,s3op_min_time,s3op_max_time --filter "operation=realpath" -b 60 -n 3
    Query output:
    Legend:
 1:     dasS3.monitoring|DasS3|s3fsops|realpath|s3op_count
 2:     dasS3.monitoring|DasS3|s3fsops|realpath|s3op_size
 3:     dasS3.monitoring|DasS3|s3fsops|realpath|s3op_error_count
 4:     dasS3.monitoring|DasS3|s3fsops|realpath|s3op_avg_time
 5:     dasS3.monitoring|DasS3|s3fsops|realpath|s3op_min_time
 6:     dasS3.monitoring|DasS3|s3fsops|realpath|s3op_max_time

Row           Timestamp s3op_count s3op_size s3op_error_count s3op_avg_time s3op_min_time s3op_max_time
  1 2023-04-11-07:24:00          0         0                0             0             0             0
  2 2023-04-11-07:25:00          4         0                1            58            15           207
  3 2023-04-11-07:26:00          0         0                0             0             0             0
    Sample query:
    # oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size,s3op_error_count,s3op_avg_time,s3op_min_time,s3op_max_time --filter "operation=fsync" -b 60 -n 3
    Query output:
    Legend:
 1:     dasS3.monitoring|DasS3|s3fsops|fsync|s3op_count
 2:     dasS3.monitoring|DasS3|s3fsops|fsync|s3op_size
 3:     dasS3.monitoring|DasS3|s3fsops|fsync|s3op_error_count
 4:     dasS3.monitoring|DasS3|s3fsops|fsync|s3op_avg_time
 5:     dasS3.monitoring|DasS3|s3fsops|fsync|s3op_min_time
 6:     dasS3.monitoring|DasS3|s3fsops|fsync|s3op_max_time

Row           Timestamp s3op_count s3op_size s3op_error_count s3op_avg_time s3op_min_time s3op_max_time
  1 2023-04-11-07:24:00          0         0                0             0             0             0
  2 2023-04-11-07:25:00          9         0                0          3936            76         18290
  3 2023-04-11-07:26:00          0         0                0             0             0             0
    Sample query:
    # oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size,s3op_error_count,s3op_avg_time,s3op_min_time,s3op_max_time --filter "operation=fileopen" -b 60 -n 3
    Query output:
    Legend:
 1:     dasS3.monitoring|DasS3|s3fsops|fileopen|s3op_count
 2:     dasS3.monitoring|DasS3|s3fsops|fileopen|s3op_size
 3:     dasS3.monitoring|DasS3|s3fsops|fileopen|s3op_error_count
 4:     dasS3.monitoring|DasS3|s3fsops|fileopen|s3op_avg_time
 5:     dasS3.monitoring|DasS3|s3fsops|fileopen|s3op_min_time
 6:     dasS3.monitoring|DasS3|s3fsops|fileopen|s3op_max_time

Row           Timestamp s3op_count s3op_size s3op_error_count s3op_avg_time s3op_min_time s3op_max_time
  1 2023-04-11-07:24:00          0         0                0             0             0             0
  2 2023-04-11-07:25:00          3         0                0          3008           576          6212
  3 2023-04-11-07:26:00          0         0                0             0             0             0
    Sample query:
    # oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size,s3op_error_count,s3op_avg_time,s3op_min_time,s3op_max_time --filter "operation=filewritev" -b 60 -n 3
    Query output:
    Legend:
 1:     dasS3.monitoring|DasS3|s3fsops|filewritev|s3op_count
 2:     dasS3.monitoring|DasS3|s3fsops|filewritev|s3op_size
 3:     dasS3.monitoring|DasS3|s3fsops|filewritev|s3op_error_count
 4:     dasS3.monitoring|DasS3|s3fsops|filewritev|s3op_avg_time
 5:     dasS3.monitoring|DasS3|s3fsops|filewritev|s3op_min_time
 6:     dasS3.monitoring|DasS3|s3fsops|filewritev|s3op_max_time

Row           Timestamp s3op_count s3op_size s3op_error_count s3op_avg_time s3op_min_time s3op_max_time
  1 2023-04-11-07:24:00          0         0                0             0             0             0
  2 2023-04-11-07:25:00          3         0                0           260           153           363
  3 2023-04-11-07:26:00          0         0                0             0             0             0
    Sample query:
    # oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size,s3op_error_count,s3op_avg_time,s3op_min_time,s3op_max_time --filter "operation=filefsync" -b 60 -n 3
    Query output:
    Legend:
 1:     dasS3.monitoring|DasS3|s3fsops|filefsync|s3op_count
 2:     dasS3.monitoring|DasS3|s3fsops|filefsync|s3op_size
 3:     dasS3.monitoring|DasS3|s3fsops|filefsync|s3op_error_count
 4:     dasS3.monitoring|DasS3|s3fsops|filefsync|s3op_avg_time
 5:     dasS3.monitoring|DasS3|s3fsops|filefsync|s3op_min_time
 6:     dasS3.monitoring|DasS3|s3fsops|filefsync|s3op_max_time

Row           Timestamp s3op_count s3op_size s3op_error_count s3op_avg_time s3op_min_time s3op_max_time
  1 2023-04-11-07:24:00          0         0                0             0             0             0
  2 2023-04-11-07:25:00          3         0                0          4342          3675          5080
  3 2023-04-11-07:26:00          0         0                0             0             0             0
    Sample query:
    # oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size,s3op_error_count,s3op_avg_time,s3op_min_time,s3op_max_time --filter "operation=fileclose" -b 60 -n 3
    Query output:
    Legend:
 1:     dasS3.monitoring|DasS3|s3fsops|fileclose|s3op_count
 2:     dasS3.monitoring|DasS3|s3fsops|fileclose|s3op_size
 3:     dasS3.monitoring|DasS3|s3fsops|fileclose|s3op_error_count
 4:     dasS3.monitoring|DasS3|s3fsops|fileclose|s3op_avg_time
 5:     dasS3.monitoring|DasS3|s3fsops|fileclose|s3op_min_time
 6:     dasS3.monitoring|DasS3|s3fsops|fileclose|s3op_max_time

Row           Timestamp s3op_count s3op_size s3op_error_count s3op_avg_time s3op_min_time s3op_max_time
  1 2023-04-11-07:24:00          0         0                0             0             0             0
  2 2023-04-11-07:25:00          3         0                0            34            29            45
  3 2023-04-11-07:26:00          0         0                0             0             0             0
    Sample query:
    # oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size,s3op_error_count,s3op_avg_time,s3op_min_time,s3op_max_time --filter "operation=rename" -b 60 -n 3
    Query output:
    Legend:
 1:     dasS3.monitoring|DasS3|s3fsops|rename|s3op_count
 2:     dasS3.monitoring|DasS3|s3fsops|rename|s3op_size
 3:     dasS3.monitoring|DasS3|s3fsops|rename|s3op_error_count
 4:     dasS3.monitoring|DasS3|s3fsops|rename|s3op_avg_time
 5:     dasS3.monitoring|DasS3|s3fsops|rename|s3op_min_time
 6:     dasS3.monitoring|DasS3|s3fsops|rename|s3op_max_time

Row           Timestamp s3op_count s3op_size s3op_error_count s3op_avg_time s3op_min_time s3op_max_time
  1 2023-04-11-07:24:00          0         0                0             0             0             0
  2 2023-04-11-07:25:00          3         0                0           563           314           881
  3 2023-04-11-07:26:00          0         0                0             0             0             0
    Sample query:
    # oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size,s3op_error_count,s3op_avg_time,s3op_min_time,s3op_max_time --filter "operation=mkdir" -b 60 -n 3
    Query output:
    Legend:
 1:     dasS3.monitoring|DasS3|s3fsops|mkdir|s3op_count
 2:     dasS3.monitoring|DasS3|s3fsops|mkdir|s3op_size
 3:     dasS3.monitoring|DasS3|s3fsops|mkdir|s3op_error_count
 4:     dasS3.monitoring|DasS3|s3fsops|mkdir|s3op_avg_time
 5:     dasS3.monitoring|DasS3|s3fsops|mkdir|s3op_min_time
 6:     dasS3.monitoring|DasS3|s3fsops|mkdir|s3op_max_time

Row           Timestamp s3op_count s3op_size s3op_error_count s3op_avg_time s3op_min_time s3op_max_time
  2 2023-04-11-07:24:00          0         0                0             0             0             0
  3 2023-04-11-07:25:00         24         0               22           271            15          6083
  4 2023-04-11-07:26:00          0         0                0             0             0             0

list_objects

  • S3 operations
    To collect the list object S3 operations statistics using the mmperfmon query, issue the command as follows:
    # oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size,s3op_error_count,s3op_avg_time,s3op_min_time,s3op_max_time --filter "operation=list_objects" -b 60 -n 3
    Query output:
    Legend:
 1:     dasS3.monitoring|DasS3|s3ops|list_objects|s3op_count
 2:     dasS3.monitoring|DasS3|s3ops|list_objects|s3op_size
 3:     dasS3.monitoring|DasS3|s3ops|list_objects|s3op_error_count
 4:     dasS3.monitoring|DasS3|s3ops|list_objects|s3op_avg_time
 5:     dasS3.monitoring|DasS3|s3ops|list_objects|s3op_min_time
 6:     dasS3.monitoring|DasS3|s3ops|list_objects|s3op_max_time

Row           Timestamp s3op_count s3op_size s3op_error_count s3op_avg_time s3op_min_time s3op_max_time
  1 2023-04-11-10:00:00          0         0                0             0             0             0
  2 2023-04-11-10:01:00          1         0                0             4             4             4
  3 2023-04-11-10:02:00          0         0                0             0             0             0
  • File system operations
    To collect the list objects file system operations statistics using the mmperfmon query, issue the command as follows:
    # oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size,s3op_error_count,s3op_avg_time,s3op_min_time,s3op_max_time --filter "operation=stat" -b 60 -n 3
    Query output:
    Legend:
 1:     dasS3.monitoring|DasS3|s3fsops|stat|s3op_count
 2:     dasS3.monitoring|DasS3|s3fsops|stat|s3op_size
 3:     dasS3.monitoring|DasS3|s3fsops|stat|s3op_error_count
 4:     dasS3.monitoring|DasS3|s3fsops|stat|s3op_avg_time
 5:     dasS3.monitoring|DasS3|s3fsops|stat|s3op_min_time
 6:     dasS3.monitoring|DasS3|s3fsops|stat|s3op_max_time

Row           Timestamp s3op_count s3op_size s3op_error_count s3op_avg_time s3op_min_time s3op_max_time
  1 2023-04-11-10:00:00          0         0                0             0             0             0
  2 2023-04-11-10:01:00          9         0                0            11             2            45
  3 2023-04-11-10:02:00          0         0                0             0             0             0
    Sample query:
    # oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size,s3op_error_count,s3op_avg_time,s3op_min_time,s3op_max_time --filter "operation=realpath" -b 60 -n 3
    Query output:
    Legend:
 1:     dasS3.monitoring|DasS3|s3fsops|realpath|s3op_count
 2:     dasS3.monitoring|DasS3|s3fsops|realpath|s3op_size
 3:     dasS3.monitoring|DasS3|s3fsops|realpath|s3op_error_count
 4:     dasS3.monitoring|DasS3|s3fsops|realpath|s3op_avg_time
 5:     dasS3.monitoring|DasS3|s3fsops|realpath|s3op_min_time
 6:     dasS3.monitoring|DasS3|s3fsops|realpath|s3op_max_time

Row           Timestamp s3op_count s3op_size s3op_error_count s3op_avg_time s3op_min_time s3op_max_time
  1 2023-04-11-10:00:00          0         0                0             0             0             0
  2 2023-04-11-10:01:00          7         0                0            20            12            56
  3 2023-04-11-10:02:00          0         0                0             0             0             0
    Sample query:
    # oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size,s3op_error_count,s3op_avg_time,s3op_min_time,s3op_max_time --filter "operation=checkaccess" -b 60 -n 3
    Query output:
    Legend:
 1:     dasS3.monitoring|DasS3|s3fsops|checkaccess|s3op_count
 2:     dasS3.monitoring|DasS3|s3fsops|checkaccess|s3op_size
 3:     dasS3.monitoring|DasS3|s3fsops|checkaccess|s3op_error_count
 4:     dasS3.monitoring|DasS3|s3fsops|checkaccess|s3op_avg_time
 5:     dasS3.monitoring|DasS3|s3fsops|checkaccess|s3op_min_time
 6:     dasS3.monitoring|DasS3|s3fsops|checkaccess|s3op_max_time

Row           Timestamp s3op_count s3op_size s3op_error_count s3op_avg_time s3op_min_time s3op_max_time
  1 2023-04-11-10:00:00          0         0                0             0             0             0
  2 2023-04-11-10:01:00          1         0                0            49            49            49
  3 2023-04-11-10:02:00          0         0                0             0             0             0
    Sample query:
    # oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size,s3op_error_count,s3op_avg_time,s3op_min_time,s3op_max_time --filter "operation=readdir" -b 60 -n 3
    Query output:
    Legend:
 1:     dasS3.monitoring|DasS3|s3fsops|readdir|s3op_count
 2:     dasS3.monitoring|DasS3|s3fsops|readdir|s3op_size
 3:     dasS3.monitoring|DasS3|s3fsops|readdir|s3op_error_count
 4:     dasS3.monitoring|DasS3|s3fsops|readdir|s3op_avg_time
 5:     dasS3.monitoring|DasS3|s3fsops|readdir|s3op_min_time
 6:     dasS3.monitoring|DasS3|s3fsops|readdir|s3op_max_time

Row           Timestamp s3op_count s3op_size s3op_error_count s3op_avg_time s3op_min_time s3op_max_time
  1 2023-04-11-10:00:00          0         0                0             0             0             0
  2 2023-04-11-10:01:00          1         0                0            68            68            68
  3 2023-04-11-10:02:00          0         0                0             0             0             0

delete_objects

  • S3 operations
    To collect the delete object S3 operations statistics using the mmperfmon query, issue the command as follows:
    # oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size,s3op_error_count,s3op_avg_time,s3op_min_time,s3op_max_time --filter "operation=delete_object" -b 60 -n 3
    Query output:
    Legend:
 1:     dasS3.monitoring|DasS3|s3ops|delete_object|s3op_count
 2:     dasS3.monitoring|DasS3|s3ops|delete_object|s3op_size
 3:     dasS3.monitoring|DasS3|s3ops|delete_object|s3op_error_count
 4:     dasS3.monitoring|DasS3|s3ops|delete_object|s3op_avg_time
 5:     dasS3.monitoring|DasS3|s3ops|delete_object|s3op_min_time
 6:     dasS3.monitoring|DasS3|s3ops|delete_object|s3op_max_time

Row           Timestamp s3op_count s3op_size s3op_error_count s3op_avg_time s3op_min_time s3op_max_time
  1 2023-04-11-08:52:00          0         0                0             0             0             0
  2 2023-04-11-08:53:00          2         0                0             2             2             2
  3 2023-04-11-08:54:00          0         0                0             0             0             0
  • File system operations
    To collect the delete object file system operations statistics using the mmperfmon query, issue the command as follows:
    # oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size,s3op_error_count,s3op_avg_time,s3op_min_time,s3op_max_time --filter "operation=stat" -b 60 -n 3
    Query output:
    Legend:
 1:     dasS3.monitoring|DasS3|s3fsops|stat|s3op_count
 2:     dasS3.monitoring|DasS3|s3fsops|stat|s3op_size
 3:     dasS3.monitoring|DasS3|s3fsops|stat|s3op_error_count
 4:     dasS3.monitoring|DasS3|s3fsops|stat|s3op_avg_time
 5:     dasS3.monitoring|DasS3|s3fsops|stat|s3op_min_time
 6:     dasS3.monitoring|DasS3|s3fsops|stat|s3op_max_time

Row           Timestamp s3op_count s3op_size s3op_error_count s3op_avg_time s3op_min_time s3op_max_time
  1 2023-04-11-08:52:00          0         0                0             0             0             0
  2 2023-04-11-08:53:00          2         0                0            30            25            36
  3 2023-04-11-08:54:00          0         0                0             0             0             0
    Sample query:
    # oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size,s3op_error_count,s3op_avg_time,s3op_min_time,s3op_max_time --filter "operation=realpath" -b 60 -n 3
    Query output:
    Legend:
 1:     dasS3.monitoring|DasS3|s3fsops|realpath|s3op_count
 2:     dasS3.monitoring|DasS3|s3fsops|realpath|s3op_size
 3:     dasS3.monitoring|DasS3|s3fsops|realpath|s3op_error_count
 4:     dasS3.monitoring|DasS3|s3fsops|realpath|s3op_avg_time
 5:     dasS3.monitoring|DasS3|s3fsops|realpath|s3op_min_time
 6:     dasS3.monitoring|DasS3|s3fsops|realpath|s3op_max_time

Row           Timestamp s3op_count s3op_size s3op_error_count s3op_avg_time s3op_min_time s3op_max_time
  1 2023-04-11-08:52:00          0         0                0             0             0             0
  2 2023-04-11-08:53:00          2         0                0            81            26           136
  3 2023-04-11-08:54:00          0         0                0             0             0             0
    Sample query:
    # oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size,s3op_error_count,s3op_avg_time,s3op_min_time,s3op_max_time --filter "operation=unlink" -b 60 -n 3
    Query output:
    Legend:
 1:     dasS3.monitoring|DasS3|s3fsops|unlink|s3op_count
 2:     dasS3.monitoring|DasS3|s3fsops|unlink|s3op_size
 3:     dasS3.monitoring|DasS3|s3fsops|unlink|s3op_error_count
 4:     dasS3.monitoring|DasS3|s3fsops|unlink|s3op_avg_time
 5:     dasS3.monitoring|DasS3|s3fsops|unlink|s3op_min_time
 6:     dasS3.monitoring|DasS3|s3fsops|unlink|s3op_max_time

Row           Timestamp s3op_count s3op_size s3op_error_count s3op_avg_time s3op_min_time s3op_max_time
  1 2023-04-11-08:52:00          0         0                0             0             0             0
  2 2023-04-11-08:53:00          2         0                0           757           440          1074
  3 2023-04-11-08:54:00          0         0                0             0             0             0

head_object and read_object

  • I/O operations
    To collect the I/O operations statistics using the mmperfmon query, issue the command as follows:
    # oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size,s3op_error_count,s3op_avg_time,s3op_min_time,s3op_max_time --filter "operation=read" -b 60 -n 3
    Query output:
    Legend:
 1:     dasS3.monitoring|DasS3|s3io|read|s3op_count
 2:     dasS3.monitoring|DasS3|s3io|read|s3op_size
 3:     dasS3.monitoring|DasS3|s3io|read|s3op_error_count
 4:     dasS3.monitoring|DasS3|s3io|read|s3op_avg_time
 5:     dasS3.monitoring|DasS3|s3io|read|s3op_min_time
 6:     dasS3.monitoring|DasS3|s3io|read|s3op_max_time

Row           Timestamp s3op_count s3op_size s3op_error_count s3op_avg_time s3op_min_time s3op_max_time
  2 2023-04-11-09:31:00          0         0                0             0             0             0
  3 2023-04-11-09:32:00          8  67108864                0             0             0             0
  4 2023-04-11-09:33:00          0         0                0             0             0             0
  • S3 operations
    To collect the S3 operations statistics using the mmperfmon query, issue the command as follows:
    # oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size,s3op_error_count,s3op_avg_time,s3op_min_time,s3op_max_time --filter "operation=head_object" -b 60 -n 3
    Query output:
    Legend:
 1:     dasS3.monitoring|DasS3|s3ops|head_object|s3op_count
 2:     dasS3.monitoring|DasS3|s3ops|head_object|s3op_size
 3:     dasS3.monitoring|DasS3|s3ops|head_object|s3op_error_count
 4:     dasS3.monitoring|DasS3|s3ops|head_object|s3op_avg_time
 5:     dasS3.monitoring|DasS3|s3ops|head_object|s3op_min_time
 6:     dasS3.monitoring|DasS3|s3ops|head_object|s3op_max_time

Row           Timestamp s3op_count s3op_size s3op_error_count s3op_avg_time s3op_min_time s3op_max_time
  1 2023-04-11-09:31:00          0         0                0             0             0             0
  2 2023-04-11-09:32:00          9         0                0            11             2            25
  3 2023-04-11-09:33:00          0         0                0             0             0             0
    Sample query:
    # oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size,s3op_error_count,s3op_avg_time,s3op_min_time,s3op_max_time --filter "operation=read_object" -b 60 -n 3
    Query output:
    Legend:
 1:     dasS3.monitoring|DasS3|s3ops|read_object|s3op_count
 2:     dasS3.monitoring|DasS3|s3ops|read_object|s3op_size
 3:     dasS3.monitoring|DasS3|s3ops|read_object|s3op_error_count
 4:     dasS3.monitoring|DasS3|s3ops|read_object|s3op_avg_time
 5:     dasS3.monitoring|DasS3|s3ops|read_object|s3op_min_time
 6:     dasS3.monitoring|DasS3|s3ops|read_object|s3op_max_time

Row           Timestamp s3op_count s3op_size s3op_error_count s3op_avg_time s3op_min_time s3op_max_time
  1 2023-04-11-09:31:00          0         0                0             0             0             0
  2 2023-04-11-09:32:00          8         0                0           174           155           207
  3 2023-04-11-09:33:00          0         0                0             0             0             0
  • File system operations
    To collect the file system operations statistics using the mmperfmon query, issue the command as follows:
    # oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size,s3op_error_count,s3op_avg_time,s3op_min_time,s3op_max_time --filter "operation=realpath" -b 60 -n 3
    Query output:
    Legend:
 1:     dasS3.monitoring|DasS3|s3fsops|realpath|s3op_count
 2:     dasS3.monitoring|DasS3|s3fsops|realpath|s3op_size
 3:     dasS3.monitoring|DasS3|s3fsops|realpath|s3op_error_count
 4:     dasS3.monitoring|DasS3|s3fsops|realpath|s3op_avg_time
 5:     dasS3.monitoring|DasS3|s3fsops|realpath|s3op_min_time
 6:     dasS3.monitoring|DasS3|s3fsops|realpath|s3op_max_time

Row           Timestamp s3op_count s3op_size s3op_error_count s3op_avg_time s3op_min_time s3op_max_time
  1 2023-04-11-09:31:00          0         0                0             0             0             0
  2 2023-04-11-09:32:00         17         0                0            39            13            84
  3 2023-04-11-09:33:00          0         0                0             0             0             0
    Sample query:
    # oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size,s3op_error_count,s3op_avg_time,s3op_min_time,s3op_max_time --filter "operation=stat" -b 60 -n 3
    Query output:
    Legend:
 1:     dasS3.monitoring|DasS3|s3fsops|stat|s3op_count
 2:     dasS3.monitoring|DasS3|s3fsops|stat|s3op_size
 3:     dasS3.monitoring|DasS3|s3fsops|stat|s3op_error_count
 4:     dasS3.monitoring|DasS3|s3fsops|stat|s3op_avg_time
 5:     dasS3.monitoring|DasS3|s3fsops|stat|s3op_min_time
 6:     dasS3.monitoring|DasS3|s3fsops|stat|s3op_max_time

Row           Timestamp s3op_count s3op_size s3op_error_count s3op_avg_time s3op_min_time s3op_max_time
  1 2023-04-11-09:31:00          0         0                0             0             0             0
  2 2023-04-11-09:32:00         17         0                0            15             2            40
  3 2023-04-11-09:33:00          0         0                0             0             0             0
    Sample query:
    # oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size,s3op_error_count,s3op_avg_time,s3op_min_time,s3op_max_time --filter "operation=fileopen" -b 60 -n 3
    Query output:
    Legend:
 1:     dasS3.monitoring|DasS3|s3fsops|fileopen|s3op_count
 2:     dasS3.monitoring|DasS3|s3fsops|fileopen|s3op_size
 3:     dasS3.monitoring|DasS3|s3fsops|fileopen|s3op_error_count
 4:     dasS3.monitoring|DasS3|s3fsops|fileopen|s3op_avg_time
 5:     dasS3.monitoring|DasS3|s3fsops|fileopen|s3op_min_time
 6:     dasS3.monitoring|DasS3|s3fsops|fileopen|s3op_max_time

Row           Timestamp s3op_count s3op_size s3op_error_count s3op_avg_time s3op_min_time s3op_max_time
  1 2023-04-11-09:31:00          0         0                0             0             0             0
  2 2023-04-11-09:32:00         17         0                0            35             6            84
  3 2023-04-11-09:33:00          0         0                0             0             0             0
    Sample query:
    # oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size,s3op_error_count,s3op_avg_time,s3op_min_time,s3op_max_time --filter "operation=filestat" -b 60 -n 3
    Query output:
    Legend:
 1:     dasS3.monitoring|DasS3|s3fsops|filestat|s3op_count
 2:     dasS3.monitoring|DasS3|s3fsops|filestat|s3op_size
 3:     dasS3.monitoring|DasS3|s3fsops|filestat|s3op_error_count
 4:     dasS3.monitoring|DasS3|s3fsops|filestat|s3op_avg_time
 5:     dasS3.monitoring|DasS3|s3fsops|filestat|s3op_min_time
 6:     dasS3.monitoring|DasS3|s3fsops|filestat|s3op_max_time

Row           Timestamp s3op_count s3op_size s3op_error_count s3op_avg_time s3op_min_time s3op_max_time
  1 2023-04-11-09:31:00          0         0                0             0             0             0
  2 2023-04-11-09:32:00          9         0                0             3             0             8
  3 2023-04-11-09:33:00          0         0                0             0             0             0
    Sample query:
    # oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size,s3op_error_count,s3op_avg_time,s3op_min_time,s3op_max_time --filter "operation=filegetxattr" -b 60 -n 3
    Query output:
    Legend:
 1:     dasS3.monitoring|DasS3|s3fsops|filegetxattr|s3op_count
 2:     dasS3.monitoring|DasS3|s3fsops|filegetxattr|s3op_size
 3:     dasS3.monitoring|DasS3|s3fsops|filegetxattr|s3op_error_count
 4:     dasS3.monitoring|DasS3|s3fsops|filegetxattr|s3op_avg_time
 5:     dasS3.monitoring|DasS3|s3fsops|filegetxattr|s3op_min_time
 6:     dasS3.monitoring|DasS3|s3fsops|filegetxattr|s3op_max_time

Row           Timestamp s3op_count s3op_size s3op_error_count s3op_avg_time s3op_min_time s3op_max_time
  1 2023-04-11-09:31:00          0         0                0             0             0             0
  2 2023-04-11-09:32:00          9         0                0            13             2            55
  3 2023-04-11-09:33:00          0         0                0             0             0             0
    Sample query:
    # oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size,s3op_error_count,s3op_avg_time,s3op_min_time,s3op_max_time --filter "operation=fileclose" -b 60 -n 3
    Query output:
    Legend:
 1:     dasS3.monitoring|DasS3|s3fsops|fileclose|s3op_count
 2:     dasS3.monitoring|DasS3|s3fsops|fileclose|s3op_size
 3:     dasS3.monitoring|DasS3|s3fsops|fileclose|s3op_error_count
 4:     dasS3.monitoring|DasS3|s3fsops|fileclose|s3op_avg_time
 5:     dasS3.monitoring|DasS3|s3fsops|fileclose|s3op_min_time
 6:     dasS3.monitoring|DasS3|s3fsops|fileclose|s3op_max_time

Row           Timestamp s3op_count s3op_size s3op_error_count s3op_avg_time s3op_min_time s3op_max_time
  1 2023-04-11-09:31:00          0         0                0             0             0             0
  2 2023-04-11-09:32:00         17         0                0            30             2           130
  3 2023-04-11-09:33:00          0         0                0             0             0             0
    Sample query:
    # oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size,s3op_error_count,s3op_avg_time,s3op_min_time,s3op_max_time --filter "operation=fileread" -b 60 -n 3
    Query output:
    Legend:
 1:     dasS3.monitoring|DasS3|s3fsops|fileread|s3op_count
 2:     dasS3.monitoring|DasS3|s3fsops|fileread|s3op_size
 3:     dasS3.monitoring|DasS3|s3fsops|fileread|s3op_error_count
 4:     dasS3.monitoring|DasS3|s3fsops|fileread|s3op_avg_time
 5:     dasS3.monitoring|DasS3|s3fsops|fileread|s3op_min_time
 6:     dasS3.monitoring|DasS3|s3fsops|fileread|s3op_max_time

Row           Timestamp s3op_count s3op_size s3op_error_count s3op_avg_time s3op_min_time s3op_max_time
  1 2023-04-11-09:31:00          0         0                0             0             0             0
  2 2023-04-11-09:32:00          8         0                0          1866          1385          2263
  3 2023-04-11-09:33:00          0         0                0             0             0             0

initiate_multipart, upload_part, and complete_object_upload

  • I/O operations
    To collect the I/O operations statistics using the mmperfmon query, issue the command as follows:
    # oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size,s3op_error_count,s3op_avg_time,s3op_min_time,s3op_max_time --filter "operation=write" -b 60 -n 3
    Query output:
    Legend:
 1:     dasS3.monitoring|DasS3|s3io|write|s3op_count
 2:     dasS3.monitoring|DasS3|s3io|write|s3op_size
 3:     dasS3.monitoring|DasS3|s3io|write|s3op_error_count
 4:     dasS3.monitoring|DasS3|s3io|write|s3op_avg_time
 5:     dasS3.monitoring|DasS3|s3io|write|s3op_min_time
 6:     dasS3.monitoring|DasS3|s3io|write|s3op_max_time

Row           Timestamp s3op_count s3op_size s3op_error_count s3op_avg_time s3op_min_time s3op_max_time
  1 2023-04-11-09:01:00          0         0                0             0             0             0
  2 2023-04-11-09:02:00          8  67108864                0             0             0             0
  3 2023-04-11-09:03:00          0         0                0             0             0             0
  • S3 operations
    To collect the S3 operations statistics using the mmperfmon query, issue the command as follows:
    # oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size,s3op_error_count,s3op_avg_time,s3op_min_time,s3op_max_time --filter "operation=initiate_multipart" -b 60 -n 3
    Query output:
    Legend:
 1:     dasS3.monitoring|DasS3|s3ops|initiate_multipart|s3op_count
 2:     dasS3.monitoring|DasS3|s3ops|initiate_multipart|s3op_size
 3:     dasS3.monitoring|DasS3|s3ops|initiate_multipart|s3op_error_count
 4:     dasS3.monitoring|DasS3|s3ops|initiate_multipart|s3op_avg_time
 5:     dasS3.monitoring|DasS3|s3ops|initiate_multipart|s3op_min_time
 6:     dasS3.monitoring|DasS3|s3ops|initiate_multipart|s3op_max_time

Row           Timestamp s3op_count s3op_size s3op_error_count s3op_avg_time s3op_min_time s3op_max_time
  1 2023-04-11-09:01:00          0         0                0             0             0             0
  2 2023-04-11-09:02:00          1         0                0            14            14            14
  3 2023-04-11-09:03:00          0         0                0             0             0             0
    Sample query:
    # oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size,s3op_error_count,s3op_avg_time,s3op_min_time,s3op_max_time --filter "operation=upload_part" -b 60 -n 3
    Query output:
    Legend:
 1:     dasS3.monitoring|DasS3|s3ops|upload_part|s3op_count
 2:     dasS3.monitoring|DasS3|s3ops|upload_part|s3op_size
 3:     dasS3.monitoring|DasS3|s3ops|upload_part|s3op_error_count
 4:     dasS3.monitoring|DasS3|s3ops|upload_part|s3op_avg_time
 5:     dasS3.monitoring|DasS3|s3ops|upload_part|s3op_min_time
 6:     dasS3.monitoring|DasS3|s3ops|upload_part|s3op_max_time

Row           Timestamp s3op_count s3op_size s3op_error_count s3op_avg_time s3op_min_time s3op_max_time
  1 2023-04-11-09:01:00          0         0                0             0             0             0
  2 2023-04-11-09:02:00          8         0                0             0             0             1
  3 2023-04-11-09:03:00          0         0                0             0             0             0
    Sample query:
    # oc -n ibm-spectrum-scale rsh -c gpfs master2 mmperfmon query s3op_count,s3op_size,s3op_error_count,s3op_avg_time,s3op_min_time,s3op_max_time --filter "operation=complete_object_upload" -b 60 -n 3
    Query output:
    Legend:
 1:     dasS3.monitoring|DasS3|s3ops|complete_object_upload|s3op_count
 2:     dasS3.monitoring|DasS3|s3ops|complete_object_upload|s3op_size
 3:     dasS3.monitoring|DasS3|s3ops|complete_object_upload|s3op_error_count
 4:     dasS3.monitoring|DasS3|s3ops|complete_object_upload|s3op_avg_time
 5:     dasS3.monitoring|DasS3|s3ops|complete_object_upload|s3op_min_time
 6:     dasS3.monitoring|DasS3|s3ops|complete_object_upload|s3op_max_time

Row           Timestamp s3op_count s3op_size s3op_error_count s3op_avg_time s3op_min_time s3op_max_time
  1 2023-04-11-09:01:00          0         0                0             0             0             0
  2 2023-04-11-09:02:00          1         0                0            92            92            92
  3 2023-04-11-09:03:00          0         0                0             0             0             0