IT39834: IBM BLOCK STORAGE AND SPECTRUM PROTECT SCALE STORAGE CAN NOT BE USED AS A MINIO OBJECT STORE

APAR status

• Closed as program error.

Error description

• IBM block storage and Spectrum Protect Scale storage type can
  not be used as a MinIO Object Store in Velero to protect cluster
  resources. Affected versions: IBM Spectrum Protect Plus 10.1.8
  and later.
  MinIO pod can not start because of the following errors:
  $ oc logs baas-minio-0 -n baas
  mkdir: cannot create directory '/storage/velero': Permission
  denied
  Certdir exist exiting loop
  cp: cannot create regular file '/config/certs/public.crt':
  Permission denied
  cp: cannot create regular file '/config/certs/private.key':
  Permission denied
  ERROR Unable to initialize backend: mkdir /storage/.minio.sys:
  permission denied
  
  Warning FailedMount 91s kubelet Unable to attach or mount
  volumes: unmounted volumes=[storage], unattached
  volumes=[tls-key-crt baas-minio-token-qf8q2 storage config]:
  timed out waiting for the condition
  Warning FailedAttachVolume 28s (x15 over 14m)
  attachdetach-controller AttachVolume.Attach failed for volume
  "pvc-ed2dc86e-659e-4775-84e9-c28572b5992b" : CSINode XXX does
  not contain driver spectrumscale.csi.ibm.com
  

Local fix

•  Use Ceph file system storage or Ceph RBD (RADOS Block Device)
  block storage type for Velero MinIO storage.
  

Problem summary

• ****************************************************************
  * USERS AFFECTED:                                              *
  * IBM Spectrum Protect Plus Container Backup Support level     *
  * 10.1.9 on OpenShift and Kubernetes environments              *
  ****************************************************************
  * PROBLEM DESCRIPTION:                                         *
  *  see ERROR Description                                       *
  ****************************************************************
  * RECOMMENDATION:                                              *
  * Apply fixing level when available. This problem is currently *
  * projected to be fixed in Spectrum Protect Plus Container     *
  * Backup Support level 10.1.9 ifix3 and 10.1.10. Note that     *
  * this is subject to change at the discretion of IBM           *
  ****************************************************************
  

Problem conclusion

• Code changed were made to add supplemental group 1000 to minio
  pod security context.
  Code changes were made to add node selector field to baas-minio
  statefulset to schedule minio pod to run on the worker node that
  has IBM Spectrum Scale CSI driver.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Modules/Macros

• Apps     k8s      ocp
  

Fix information

• Fixed component name

  SP PLUS

• Fixed component ID

  5737SPLUS

Applicable component levels