Configuring IBM Cloud Pak foundational services by using the CommonService custom resource

The IBM Cloud Pak foundational services operator creates the CommonService custom resource (CR) in the ibm-common-services namespace.

The custom resource has the services, cluster configurations, and hardware profile that you can set before or after you install foundational services.

You can access the CommonService custom resource by using the Red Hat® OpenShift® Container Platform console or by using the command-line interface (CLI).

• To use the console, complete these steps:

  1. From the navigation pane, click Operators > Installed Operators.
  2. From the Project drop-down list, select ibm-common-services.
  3. Click IBM Cloud Pak foundational services.
  4. Select the CommonService tab. You can see the common-service custom resource instance.
  5. Click the common-service resource.
  6. Select the YAML tab.

• To use the CLI, run the following command:

  oc edit CommonService common-service -n ibm-common-services
  

Updating the custom resource

Set the following parameters by adding them to the CommonService custom resource. You can update these parameters before you create an OperandRequest instance or after you install the services. Add or modify the parameters and values in the spec section.

• Approval strategy
• Namespace permissions
• Bring your own CA Certificate
• License
• Hardware profile
• Storage class
• Crossplane service
• Services configuration
  • General settings
    • Federal Information Processing Standard (FIPS)
  • MongoDB settings
    • Storage class (ibm-mongodb-operator)
    • Changing the default values of the MongoDB username and password (ibm-mongodb-operator)
    • Configuring resources for MongoDB (ibm-mongodb-operator)
  • Authentication settings
    • Changing the default admin username
    • Delegating authentication to OpenShift (ibm-iam-operator)
      • Assigning the Cloud Pak administrator privileges to an OpenShift user
    • Adding custom OIDC claims
    • Disabling nonce
    • Adding description of the login options (ibm-commonui-operator)
    • Setting the preferred login options (ibm-iam-operator)
    • Changing the OIDC issuer URL (ibm-iam-operator)
    • Changing the provider issuer URL (ibm-iam-operator)
    • Changing the cluster name (ibm-iam-operator)
    • HA configuration
    • Sample ibm-iam-operator operator section in the custom resource
    • Configuring resources for IAM (ibm-iam-operator)
  • Logging service settings Note: The deprecated Logging service is removed in IBM Cloud Pak® foundational services version 3.7.x.
    • General
    • Filebeat
    • Logstash
    • Kibana
    • Elasticsearch
    • Security
    • Curator
      • Sample CR
  • Log storage
  • Audit logging settings
  • License Service settings
    • Enable additional information in logs for troubleshooting purposes
    • Configure resources for License Service (ibm-licensing-operator)
  • Platform API settings
  • Catalog UI settings
  • Helm API settings
  • Helm Repo settings
  • Monitoring settings
    • PrometheusExt settings
      • Configure resources for Monitoring PrometheusExt (ibm-monitoring-prometheusext-operator)
    • Exporters settings
      • Configure resources for Monitoring Exporters (ibm-monitoring-exporters-operator)
    • Grafana settings
      • Configure resources for Monitoring Grafana (ibm-monitoring-grafana-operator)
  • Certificate management service settings
    • Configuring resources for Certificate manager (ibm-cert-manager-operator)
    • certManagerCAInjector
    • certManagerController
    • certManagerWebhook
    • Leaf certificate refresh parameters
  • Common Web UI settings
  • Serviceability settings
    • MustGather configuration settings
    • MustGather job settings
    • MustGather service settings
    • System Healthcheck service settings
    • System Healthcheck resources settings (ibm-healthcheck-operator)
  • Management Ingress settings
  • Nginx Ingress settings
  • Metering settings Note: The deprecated Metering service is removed in IBM Cloud Pak® foundational services version 3.7.x.
  • Common UI settings

Approval strategy

The approval strategy defines whether the approval is needed to install, or upgrade IBM Cloud Pak foundational services. By default, the approval strategy is set to Automatic, however, you can change this setting during installation. You can also set the approval strategy after the installation by adding or changing the installPlanApproval parameter in the custom resource. If the approval strategy is set to Automatic, the operator is automatically installed or upgraded when a new version is available. If you set the installPlanApproval parameter to Manual, the operator is not automatically installed or upgraded. Instead, you get an an install plan that needs to be manually approved before an upgrade.

Notes:

• When you set the installPlanApproval for the IBM Cloud Pak foundational services operator, it applies to all foundational services installed with this operator.
• If you install the IBM Cloud Pak foundational services operator in a namespace where another operator is installed that already has the installPlanApproval: Manual set in its subscription, then the IBM Cloud Pak foundational services operator inherits this setting. The approval plan is set to Manual and IBM Cloud Pak foundational services operator cannot be automatically installed or upgraded.

Changing approval strategy from Automatic to Manual

To change the approval strategy from Automatic to Manual, change the installPlanApproval parameter value to Manual in the spec section of the CommonService CR.

Note: If the installPlanApproval parameter is not in the CR, add the following line to the CR in the spec section: installPlanApproval: Manual.

apiVersion: operator.ibm.com/v3
kind: CommonService
metadata:
  name: common-service
  namespace: ibm-common-services
spec:
  installPlanApproval: Manual

Changing approval strategy from Manual to Automatic

Note: Before you begin, make sure that the operators that are not a part of IBM Cloud Pak foundational services in the namespace have the installPlanApproval parameter set to Automatic in the subscription. Otherwise, even if you change the approval strategy from Manual to Automatic for the IBM Cloud Pak foundational services operator, this setting might be overwritten.

To change the approval strategy from Manual to Automatic, complete the following steps:

1. Change the installPlanApproval parameter value to Automatic in the spec section of the CommonService CR.

   Note: If the installPlanApproval parameter is not in the CR, add the following line to the CR in the spec section: installPlanApproval: Automatic.

   apiVersion: operator.ibm.com/v3
   kind: CommonService
   metadata:
    name: common-service
    namespace: ibm-common-services
   spec:
    installPlanApproval: Automatic
   

2. Change the installPlanApproval parameter value to Automatic in the IBM Cloud Pak foundational services subscription. If you have more than one IBM Cloud Pak foundational services subscription, change the value of the installPlanApproval parameter in all subscriptions.

Namespace permissions

Manually authorize foundational services to watch a namespace by adding manualManagement: true in the spec section of the CommonService CR. For more information about namespace permissions, see IBM NamespaceScope Operator.

apiVersion: operator.ibm.com/v3
kind: CommonService
metadata:
  name: common-service
  namespace: ibm-common-services
spec:
  size: medium
  manualManagement: true

Bring your own CA Certificate

You can replace the foundational services self-signed certificate authority (CA) certificate with your own CA certificate. To do so, first update BYOCACertificate: true in the spec section of the CommonService CR.

Note: If BYOCACertificate specification is not in the CR, you can add it in the spec section as shown in the following example:

apiVersion: operator.ibm.com/v3
kind: CommonService
metadata:
  name: common-service
  namespace: ibm-common-services
spec:
  BYOCACertificate: true

After you update the CommonService CR, complete the following steps to replace the foundational services self-signed CA certificate with your own CA certificate:

1. Prepare and have your Transport Layer Security (TLS) certificate, TLS private key, and CA certificate ready.

2. Create a backup of the foundational services self-signed CA certificate. By default, the certificate is created in the ibm-common-services namespace.

   oc get certificate cs-ca-certificate -n ibm-common-services -o yaml > cs-ca-certificate.yaml
   

3. Delete the foundational services self-signed CA certificate resource so that the cert-manager service does not re-create the updated secret.

   oc delete certificate cs-ca-certificate -n ibm-common-services
   

4. Delete the foundational services self-signed CA certificate secret.

   oc delete secret cs-ca-certificate-secret -n ibm-common-services
   

5. Re-create the cs-ca-certificate-secret secret with your CA certificate (ca.crt), TLS certificate (tls.crt), and private key (tls.key).

   oc -n ibm-common-services create secret generic cs-ca-certificate-secret --from-file=ca.crt=<your path>/ca.crt  --from-file=tls.crt=<your path>/tls.crt  --from-file=tls.key=<your path>/tls.key
   

6. Refresh the leaf certificates that are created by individual foundational services. For more information and steps to refresh the leaf certificates, see Refreshing foundational services internal certificates.

License

Accept the license to use foundational services.

To do so, add spec.license.accept: true in the spec section. See the following sample code:

apiVersion: operator.ibm.com/v3
kind: CommonService
metadata:
  name: common-service
  namespace: ibm-common-services
spec:
  license:
    accept: true
  size: as-is

Hardware profile

Set the hardware requirements profile based on the workloads in your cluster. For more information about the profiles, see Hardware requirements and recommendations for foundational services.

You can use the templates to update the hardware requirements of the services that you are installing. You can also use the templates to set the configuration parameters of the services.

The default profile is starterset. You can change the profile to small, medium, or large, if required. If you are upgrading your cluster from a previous release, the default profile setting is as-is, which means that the hardware requirements setting from the previous release is retained.

apiVersion: operator.ibm.com/v3
kind: CommonService
metadata:
  name: common-service
  namespace: ibm-common-services
spec:
  size: as-is

Storage class

Instead of specifying the storage class for each service in the spec.<service-name> section, you can specify a storage class for all services in the spec section. Any service that needs to use a storage class will then use this specified storage class.

apiVersion: operator.ibm.com/v3
kind: CommonService
metadata:
  name: common-service
  namespace: ibm-common-services
spec:
  size: as-is
  storageClass: <storage-class-name>

When you specify a storage class in the spec section, the storage class of a service is updated to reflect this value only if the service is not yet deployed. For example, MongoDB and the Monitoring service both use a storage class. If IBM Cloud Pak A specifies rook-cephfs as the storage class in the spec section and requests only for MongoDB, then MongoDB is deployed with rook-cephfs as the storage class. Though IBM Cloud Pak A did not request for the Monitoring service, internally the storage class for the Monitoring service is set as rook-cephfs. In the same cluster, if IBM Cloud Pak B then specifies rook-ceph-block as the storage class in the spec section and requests for the Monitoring service, then the service is deployed with rook-ceph-block as the storage class. The storage class for MongoDB is not changed and it continues to use rook-cephfs.

Crossplane service

Crossplane is an open source Kubernetes add-on that extends any cluster with the ability to provision and manage cloud infrastructure, services, and applications by using kubectl, GitOps, or any tool that works with the Kubernetes API.

The IBM Crossplane operators are automatically installed during foundational services installation. However, you must manually enable the service if you want to use it in your cluster.

Enable the service by adding the following piece of code in the CommonService CR.

spec:
  features:
    bedrockshim:
      enabled: true

If you want to uninstall the IBM Crossplane Provider operators, you can add the crossplaneProviderRemoval: true configuration to the CommonService CR. This configuration uninstalls only the IBM Crossplane Provider service, and leaves the Crossplane service untouched.

See the following sample.

spec:
  features:
    bedrockshim:
      enabled: true
      crossplaneProviderRemoval: true

When you add crossplaneProviderRemoval: true, foundational services deletes the CSV and subscription of the following operators:

• ibm-crossplane-provider-ibm-cloud-operator
• ibm-crossplane-provider-kubernetes-operator

Services configuration

Add any service-related configuration in the spec.services section. First, add services: under spec:. You need to add services: section only once. You can add all services-related configuration under the same section.

apiVersion: operator.ibm.com/v3
kind: CommonService
metadata:
  name: common-service
  namespace: ibm-common-services
spec:
  services:

Then, add the service configuration. For example, if you are adding a storage class for MongoDB, the configuration would be as shown in the following example:

apiVersion: operator.ibm.com/v3
kind: CommonService
metadata:
  name: common-service
  namespace: ibm-common-services
spec:
  services:
  - name: ibm-mongodb-operator
    spec:
      mongoDB:
        storageClass: cephfs

General settings

Federal Information Processing Standard (FIPS)

For information on configuring foundational services to enable FIPS, see Enabling FIPS.

MongoDB settings

Storage class (ibm-mongodb-operator)

Specify a storage class name for the persistent volume. If you do not add the storage class configuration, the default storage class that is available in the cluster is used for MongoDB.

- name: ibm-mongodb-operator
  spec:
    mongoDB:
      storageClass: <storage_class_name>

Note: If you change the resource settings post upgrade, the default settings get restored.

If you are installing your IBM Cloud Pak on IBM Cloud®, use the ibmc-file-gold-gid or ibmc-block-gold storage class.

Changing the default values of the MongoDB username and password (ibm-mongodb-operator)

The ibm-mongodb-operator generates default username and password, which are random strings. You can change these parameter values before you install the IBM Cloud Pak foundational services.

1. Get the base64-encoded values of the new username and password.

   echo <new-username or new-password> | base64
   

2. Define a YAML file of kind Secret. Use the ibm-common-services namespace, and the base64-encoded username and password values in this YAML file. Following is a sample file:

   apiVersion: v1
   kind: Secret
   metadata:
     name: icp-mongodb-admin
     namespace: ibm-common-services
   type: Opaque
   data:
     password: SFV6a2NYMkdKa2tBZA==
     user: dGpOcDR5Unc=
   

3. Create the secret by running the following command:

   kubectl apply -f <YAML-file-name>.yaml
   

Configuring resources for MongoDB (ibm-mongodb-operator)

Specify resource settings. If you do not specify a setting, the default resource settings that are available in the cluster are used for MongoDB.

      replicas:
      resources:
        requests:
          cpu: <cpu-request>
          memory: <memory-request>
        limits:
          cpu: <cpu-limit>
          memory: <memory-limit>

Authentication settings

• Changing the default admin username
• Delegating authentication to OpenShift
  • Assigning the cloud pak administrator privileges to an OpenShift user
• Adding custom OIDC claims
• Disabling nonce
• Adding description of the login options
• Setting the preferred login options (ibm-iam-operator)
• Changing the OIDC issuer URL
• Changing the provider issuer URL
• Changing the cluster name
• HA configuration
• Sample ibm-iam-operator operator section in the custom resource
• Configuring resources for IAM (ibm-iam-operator)

Changing the default admin username

The IBM Cloud Pak foundational services installation creates a default admin user, who is a cluster administrator. You can customize the username by adding the defaultAdminUser parameter.
Note: If you already have a user by the name admin in your cluster, you must set the defaultAdminUser parameter with a custom name that is not admin before you install foundational services. This is to avoid your admin user from being removed if you uninstall foundational services later.

- name: ibm-iam-operator
  spec:
    authentication:
      config:
        defaultAdminUser: <custom-username>

Delegating authentication to OpenShift (ibm-iam-operator)

Authentication with Red Hat OpenShift is enabled by default. The ibm-iam-operator has the following default configuration. If you do not update these parameters, authentication with OpenShift is enabled with no prefix.

- name: ibm-iam-operator
  spec:
    authentication:
      config:
        roksEnabled: true
        roksURL: <your-endpoint-URL>
        roksUserPrefix: ""

You can disable the authentication, or update the parameters as required. Following are the parameter descriptions:

• roksEnabled: Set to false: to disable authentication with Red Hat OpenShift.
• roksURL: The public service endpoint URL of your public cloud cluster. For more information about how to get the URL, see Updating OpenShift authentication.
• roksUserPrefix: Prefix to be used with the username. When you access your cluster console or CLI, you use the prefix along with the username to authenticate with OpenShift. If you are using IAM with Red Hat® OpenShift® Kubernetes Service in the IBM Cloud, you must set the prefix to IAM#.

Assigning the cloud pak administrator privileges to an OpenShift user

If you are delegating authentication to OpenShift, you can assign the cloud pak administrator privileges to an existing OpenShift user. The user can be added to the bootstrapUserId parameter.

To add the bootstrapUserId parameter before IAM service installation, see the following configuration:

- name: ibm-iam-operator
  spec:
    authentication:
      config:
        bootstrapUserId: "<custom-username>"

To add the bootstrapUserId parameter after IAM service installation, see Changing the default cluster administrator.

Adding custom OIDC claims

The IAM service uses the default scopes and claims that WebSphere® Application Server Liberty provides. Based on your OIDC authentication requirements, you can customize the OIDC claims that are returned by the UserInfo endpoint. For more information about custom claims, see Adding custom OIDC claims.

To add the custom claims before you install the IAM service, add the following configuration:

- name: ibm-iam-operator
  spec:
    authentication:
      config:
        claimsSupported: "<list-of-claims>"
        claimsMap: "<list-of-claims-map>"
        scopeClaim: "profile=<list-of-claims-in-the-profile>"

See the following example:

apiVersion: operator.ibm.com/v3
kind: CommonService
metadata:
  name: common-service
  namespace: ibm-common-services
spec:
  size: medium
  services:
  - name: ibm-iam-operator
    spec:
      authentication:
        config:
          claimsSupported: "name,family_name,display_name,given_name,preferred_username"
          claimsMap: "name=\"givenName\" family_name=\"givenName\" given_name=\"givenName\" preferred_username=\"givenName\" display_name=\"displayName\""
          scopeClaim: "profile=\"name,family_name,display_name,given_name,preferred_username\""

Disabling nonce

To improve security, nonce is enabled by default. Nonce associates a client session with an ID token and is used in authentication to ensure that attackers do not use old sessions.

To disable nonce, add the following configuration:

- name: ibm-iam-operator
  spec:
    authentication:
      config:
        nonceEnabled: false

Adding description of the login options (ibm-commonui-operator)

You can configure multiple authentication types in your cluster. For more information, see Authentication types.

The authentication types that you configure in your cluster are displayed on the console login page. You can provide a short description of the authentication types in the ibm-commonui-operator by configuring these parameters. The description is then displayed on the console login page.

- name: ibm-commonui-operator
  spec:
    globalUIConfig:
      enterpriseLDAP: <Provide a short description about the enterprise LDAP authentication type.>
      defaultAuth: <Provide a short description about the default authentication type.>
      osAuth: <Provide a short description about the OpenShift authentication type.>
      enterpriseSAML: <Provide a short description about the enterprise SAML authentication type.>

Setting the preferred login options (ibm-iam-operator)

If you configured multiple authentication types in your cluster, but you want the console login page to display any one or a selected set of the configured login options, you can set the preferredLogin: parameter. For example, preferredLogin:SAML,LDAP.

Use the following parameter values:

• For default authentication, use DEFAULT
• For enterprise LDAP, use LDAP
• For enterprise SAML, use SAML
• For OpenShift authentication, use ROKS

- name: ibm-iam-operator
  spec:
    authentication:
      config:
        preferredLogin: <option1>,<option2>,<optionN>

Changing the OIDC issuer URL (ibm-iam-operator)

Default configuration of OpenID Connect (OIDC) in your product uses <cluster_address>:443 in the authentication endpoints, which are used to authenticate users to Kubernetes. However, in the oidcIssuerURL, local host IP address is used, the default value of which is https://127.0.0.1:443/idauth/oidc/endpoint/OP. If required, you can change the oidcIssuerURL endpoint to use a hostname for Kubernetes authentication. If you want to use the .well-known/openid-configuration endpoint, you must update the oidcIssuerURL with the hostname that you want to use.

- name: ibm-iam-operator
  spec:
    authentication:
      config:
        oidcIssuerURL: <https://<hostname>:443/oidc/endpoint/OP>

Changing the provider issuer URL (ibm-iam-operator)

If you want to use the .well-known/openid-configuration endpoint, you must update the providerIssuerURL with the hostname that you want to use. You can use the endpoint to get the OIDC configuration information of the provider. The default setting is providerIssuerURL: ''.

- name: ibm-iam-operator
  spec:
    authentication:
      config:
        providerIssuerURL: <https://<hostname>:443/idprovider/v1/auth/.well-known/openid-configuration

Changing the cluster name (ibm-iam-operator)

The default cluster name is mycluster. You can change the cluster name by using the clusterName parameter. The cluster name is used in Cloud Resource Names (CRNs).

- name: ibm-iam-operator
  spec:
    authentication:
      config:
        clusterName: <custom-name>

HA configuration

If you are configuring high availability (HA) in your cluster, add the following configuration for the IAM operator:

- name: ibm-iam-operator
   spec:
     authentication:
       replicas: 3
     pap:
       replicas: 3
     policycontroller: {}
     policydecision:
       replicas: 3
     secretwatcher: {}
     securityonboarding: {}

Sample ibm-iam-operator operator section in the custom resource

Following is a sample configuration of the ibm-iam-operator operator in the custom resource. The sample includes the authentication with Red Hat OpenShift and OIDC URL parameters.

- name: ibm-iam-operator
    spec:
      authentication:
        config:
          bootStrapUserId: clusteradmin
          roksEnabled: true
          roksURL: 'https://c100-e.eu-de.containers.cloud.ibm.com:32301'
          roksUserPrefix: 'IAM#'
          oidcIssuerURL: 'https://c100-e.eu-de.containers.cloud.ibm.com:443/oidc/endpoint/OP'
          preferredLogin: ROKS
      oidcclientwatcher: {}
      pap: {}
      policycontroller: {}
      policydecision: {}
      secretwatcher: {}
      securityonboarding: {}

Configuring resources for IAM (ibm-iam-operator)

Specify resource settings. If you do not specify a setting, the default resource settings that are available in the cluster are used for IAM.

  - name: ibm-iam-operator
    spec:
      authentication:
        replicas: 1
        auditService:
          resources:
            limits:
              cpu:
              memory:
            requests:
              cpu:
              memory:
        authService:
          resources:
            limits:
              cpu:
              memory:
            requests:
              cpu:
              memory:
        clientRegistration:
          resources:
            limits:
              cpu:
              memory:
            requests:
              cpu:
              memory:
        identityManager:
          resources:
            limits:
              cpu:
              memory:
            requests:
              cpu:
              memory:
        identityProvider:
          resources:
            limits:
              cpu:
              memory:
            requests:
              cpu:
              memory:
      oidcclientwatcher:
        replicas: 1
        resources:
          limits:
            cpu:
            memory:
          requests:
            cpu:
            memory:
      pap:
        auditService:
          resources:
            limits:
              cpu:
              memory:
            requests:
              cpu:
              memory:
        papService:
          resources:
            limits:
              cpu:
              memory:
            requests:
              cpu:
              memory:
        replicas: 1
      policycontroller:
        replicas: 1
        resources:
          limits:
            cpu:
            memory:
          requests:
            cpu:
            memory:
      policydecision:
        auditService:
          resources:
            limits:
              cpu:
              memory:
            requests:
              cpu:
              memory:
        resources:
          limits:
            cpu:
            memory:
          requests:
            cpu:
            memory:
        replicas: 1
      secretwatcher:
        resources:
          limits:
            cpu:
            memory:
          requests:
            cpu:
            memory:
        replicas: 1
      securityonboarding:
        replicas: 1
        resources:
          limits:
            cpu:
            memory:
          requests:
            cpu:
            memory:
        iamOnboarding:
          resources:
            limits:
              cpu:
              memory:
            requests:
              cpu:
              memory:

Logging service settings

Note: The deprecated Logging service is removed in IBM Cloud Pak® foundational services version 3.7.x.

• Table 1. General Logging service settings
• Table 2. Filebeat settings
• Table 3. Logstash settings
• Table 4. Kibana settings
• Table 5. Elasticsearch settings
• Table 6. Security settings
• Table 7. Curator settings

General

The following parameters can be specified in the custom resource to override default values. Adding parameters to custom resource is not necessary if you want to use the default values.

Filebeat

Logstash

Kibana

Elasticsearch

Security

Curator

The curator is a tool to clean out old log indices from Elasticsearch. For more information, see Elastic's official documentation.

Sample CR

Following is a sample CR:

apiVersion: elasticstack.ibm.com/v1alpha1
kind: ElasticStack
metadata:
  name: logging
spec:
  nameOverride: elk
  general:
    environment: OpenShift
    ingressPort: "443"
  security:
    ca:
      # set to `external` to use existing CA stored in Kubernetes secret to generate certs
      origin: external
      external:
        # the secret need to be in the same namespace as the chart release
        secretName: cs-ca-certificate-secret
        # the Kubernetes field name (key) within the specified secret that stores CA cert
        certFieldName: tls.crt
        # the Kubernetes field name (key) within the specified secret that stores CA private key
        keyFieldName: tls.key
    tls:
      version: TLSv1.2

  logstash:
    replicas: 3
    port: 5044
    nodeSelector:
      management: "true"
    tolerations:
      - key: "dedicated"
        operator: "Exists"
        effect: "NoSchedule"
  kibana:
    replicas: 3
    nodeSelector:
      management: "true"
    tolerations:
      - key: "dedicated"
        operator: "Exists"
        effect: "NoSchedule"
    # accepted values:
    # ingress or loadBalancer, defaults to loadBalancer
    access: ingress
    ingress:
      # "/kibana" for managed service logging instance
      # sample value for custom ingress: "/tenantA/kibana"
      # no trailing /
      path: "/kibana"
      # additional labels to facilitate link rendering in icp console
      labels:
        # Nav link currently explicitly created by platform UI when Logging is present. If this is changed, inmenu should be set to "true".
        inmenu: "false"
        # if provided, the link will open in a new tab with the target value in the <a> tag
        target: "platform-logging"
    service:
      # "/kibana" for managed service logging instance
      # sample value for custom ingress: "/tenantA/kibana"
      # no trailing /
      path: "/kibana"
      # additional labels to facilitate link rendering in icp console
      labels:
        # Nav link currently explicitly created by platform UI when Logging is present. If this is changed, inmenu should be set to "true".
        inmenu: "false"
        # if provided, the link will open in a new tab with the target value in the <a> tag
        target: "platform-logging"
      # additional annotations to facilitate link rendering in icp console
      annotations:
        # display name that will show in the menu
        name: "Logging"
        # provided by icp console
        id: "add-ons"
        # list of roles to be able to view TA in the menu
        # for managed logging instance deployed
        # without other multi-tenant logging
        # instances, use the following:
        roles: "ClusterAdministrator,Administrator,Operator,Viewer,Auditor,Editor"
        # for managed logging instance deployed
        # with other multi-tenant logging
        # instances, use the following:
        # roles: "ClusterAdministrator"
        # show link if user is in any of the teams
        # ui.icp.ibm.com/tenant: "tenantA,tenantB"
    security:
      authc:
        enabled: true
        # accepted values: icp
        # what it does: redirects to icp login page first
        provider: icp
      authz:
        enabled: false
        # accepted values: icp
        # what it does: only allow request to pass if user
        # have access to the required namespaces
        # that the current user has access to
        # requires authc.enabled = true and authc.provider = icp
        provider: icp
        icp:
          # 1. user is allowed to access the kibana ingress
          #    if namespaces granted to user are in the following list
          # 2. when the following list is empty, only cluster admin
          #    can access this kibana ingress
          authorizedNamespaces:
            # - tenantadev
            # - tenantatest
            # - tenantaprod

  filebeat:
    scope:
      nodes:
        # for managed logging instance deployed
        # with other multi-tenant logging
        # instances, it might be a good
        # idea to have managed logging instance
        # only collect log from limited workloads
        # (rather than workloads from all tenants).
        # set desired node selector here
      namespaces:
    tolerations:
      - key: "dedicated"
        operator: "Exists"
        effect: "NoSchedule"

  elasticsearch:
    security:
      authz:
        enabled: true
        # accepted values: icp
        # what it does: filter log content by the namespace
        # that the current user has access to
        provider: icp
    client:
      tolerations:
        - key: "dedicated"
          operator: "Exists"
          effect: "NoSchedule"
      nodeSelector:
        management: "true"

    master:
      tolerations:
        - key: "dedicated"
          operator: "Exists"
          effect: "NoSchedule"
      nodeSelector:
        management: "true"

    data:
      replicas: 1
      tolerations:
        - key: "dedicated"
          operator: "Exists"
          effect: "NoSchedule"
      nodeSelector:
        management: "true"
      storage:
        size: "20Gi"
        storageClass: "gp2"
        useDynamicProvisioning: true

  curator:
    nodeSelector:
      management: "true"
    tolerations:
      - key: "dedicated"
        operator: "Exists"
        effect: "NoSchedule"

Log Storage

Static Persistent Volumes

For each Elasticsearch pod, a persistent volume (PV) is required if no dynamic provisioning is set up. For more information, see official Kubernetes documentation.

Create a PV for logging.

1. Create a logging service data directory and change the permissions. On the node, grant at least 0755 permission to the logging service data directory. The default location of this customizable path is /var/lib/icp/logging/elk-data.

2. Create the PV.

   a. Create PV YAML file, pv-logging-datanode-aaa.bbb.ccc.ddd.yaml. Modify the following sample YAML file to replace aaa.bbb.ccc.ddd with the IP address of the node on which logging runs.

   apiVersion: v1
   kind: PersistentVolume
   metadata:
   # replace `aaa.bbb.ccc.ddd` with the IP address of the node on which logging runs
   name: logging-datanode-aaa.bbb.ccc.ddd
   spec:
   accessModes:
   - ReadWriteOnce
   capacity:
     storage: 30Gi # adjust to your need
   local:
     path: /var/lib/icp/logging/elk-data # adjust to your need
   nodeAffinity:
     required:
       nodeSelectorTerms:
       - matchExpressions:
         - key: kubernetes.io/hostname
           operator: In
           values:
           - <worker_node_name> # add node name to this list, which can be found using kubectl get nodes
   persistentVolumeReclaimPolicy: Retain
   storageClassName: logging-storage-datanode  # adjust to your need
   

   b. Run the following command to create a PV.

   kubectl apply -f pv-logging-datanode-aaa.bbb.ccc.ddd.yaml
   

   c. Repeat steps a. and b. for each node on which logging runs.

Dynamically provisioned persistent volumes

See documentation for the underlying container platform, for example, such as Red Hat OpenShift. If elasticsearch.data.storage.storageClass is not specified, the default storage provisioner that is configured for the cluster is used.

Audit logging settings

Add policy controller settings for Audit logging 3.8.0.

To configure the policy controller for Audit logging version 3.7.0, see Configuring the policy controller for Audit logging version 3.7.0.

Note: Fluentd runs as a DaemonSet so resource values are per node.

You can update these parameters by adding them in the custom resource:

Note: Default values are used when fields are empty.

- name: ibm-auditlogging-operator
  spec:
    fluentd:
      enabled: <true or false>
      imageRegistry: <fluentd image registry>
      pullPolicy: <fluentd pull policy>
      journalPath: <path-to-the-journal>
      issuer: <certificate-authority-name>
      resources:
        requests:
          cpu: <cpu-request>
          memory: <memory-request>
        limits:
          cpu: <cpu-limit>
          memory: <memory-limit>

Following is an example configuration with the default values:

- name: ibm-auditlogging-operator
  spec:
    fluentd:
      enabled: false
      imageRegistry: quay.io/opencloudio/
      pullPolicy: IfNotPresent
      journalPath: /run/log/journal
      issuer: cs-ca-issuer
      resources:
        requests:
          cpu: 25m
          memory: 100Mi
        limits:
          cpu: 300m
          memory: 400Mi

You can update these parameters by adding them in the custom resource:

Note: Default values are used when fields are empty.

- name: ibm-auditlogging-operator
  spec:
    fluentd:
      enabled: <true or false>
      imageRegistry: <fluentd image registry>
      pullPolicy: <fluentd pull policy>
      journalPath: <path-to-the-journal>
      issuer: <certificate-authority-name>
    policyController:
      imageRegistry: <policy controller image registry>
      pullPolicy: <policy controller pull policy>
      verbosity: <"log-output-level">
      frequency: <"frequency-of-status-update">

Following is an example configuration with the default values:

- name: ibm-auditlogging-operator
  spec:
    fluentd:
      enabled: false
      imageRegistry: quay.io/opencloudio/
      pullPolicy: IfNotPresent
      journalPath: /run/log/journal
      issuer: cs-ca-issuer
    policyController:
      imageRegistry: quay.io/opencloudio/
      pullPolicy: IfNotPresent
      verbosity: "0"
      frequency: "10"

License Service settings

Enable additional information in logs for troubleshooting purposes

By default, the License Service instance pod logs contain only the basic information about the service. You can enable the additional information in logs for troubleshooting purposes by adding the logLevel parameter to the spec section.

The available logLevel options:

• DEBUG - This option enables all debug information in logs.
• VERBOSE - This option extends the logs with information about license calculation and API calls.

Following is an example configuration:

- name: ibm-licensing-operator
  spec:
    IBMLicensing:
      logLevel: DEBUG

Configure resources for License Service (ibm-licensing-operator)

Specify resource settings. If you do not specify a setting, the default resource settings that are available in the cluster are used for License Service.

  - name: ibm-licensing-operator
    spec:
      IBMLicensing:
        resources:
          requests:
            cpu:
            memory:
          limits:
            cpu:
            memory:
      IBMLicenseServiceReporter:
        databaseContainer:
          resources:
            requests:
              cpu:
              memory:
            limits:
              cpu:
              memory:
        receiverContainer:
          resources:
            requests:
              cpu:
              memory:
            limits:
              cpu:
              memory:

Note: If you do not specify a setting, the default resource settings that are available in the cluster are used for License Service. The following are the default resource settings.

  - name: ibm-licensing-operator
    spec:
      IBMLicensing:
        resources:
          requests:
            cpu: 200m
            memory: 256Mi
          limits:
            cpu: 500m
            memory: 512Mi
      IBMLicenseServiceReporter:
        databaseContainer:
          resources:
            requests:
              cpu: 200m
              memory: 256Mi
            limits:
              cpu: 300m
              memory: 300Mi
        receiverContainer:
          resources:
            requests:
              cpu: 200m
              memory: 256Mi
            limits:
              cpu: 300m
              memory: 384Mi

Platform API settings

Example configuration with default values:

- name: ibm-platform-api-operator
  spec:
    platformApi:
      auditService:
        config:
          enabled: true
        resources:
          limits:
            cpu: 200m
            memory: 250Mi
          requests:
            cpu: 200m
            memory: 250Mi
      platformApi:
        resources:
          limits:
            cpu: 500m
            memory: 512Mi
          requests:
            cpu: 500m
            memory: 512Mi
      replicas: 1

Catalog UI settings

Example configuration with default values:

- name: ibm-catalog-ui-operator
  spec:
    catalogui:
      resources:
        limits:
          cpu: 300m
          memory: 300Mi
        requests:
          cpu: 300m
          memory: 300Mi
      replicaCount: 1

Helm API settings

Example configuration with default values:

- name: ibm-helm-api-operator
  spec:
  helmapi:
    env:
      HTTP_PROXY: ""
      HTTPS_PROXY: ""
      NO_PROXY: mycluster.icp,mongodb,platform-identity-provider,platform-identity-management,icp-management-ingress,iam-pap,localhost,127.0.0.1
    resources:
      limits:
        cpu: 300m
        memory: 400Mi
      requests:
        cpu: 300m
        memory: 400Mi
  replicaCount: 1
  rudder:
    resources:
      limits:
        cpu: 150m
        memory: 128Mi
      requests:
        cpu: 150m
        memory: 128Mi
  tiller:
    resources:
      limits:
        cpu: 100m
        memory: 128Mi
      requests:
        cpu: 100m
        memory: 128Mi

Helm Repo settings

Example configuration with default values:

- name: ibm-helm-repo-operator
  spec:
  helmrepo:
    resources:
      limits:
        cpu: 100m
        memory: 512Mi
      requests:
        cpu: 100m
        memory: 512Mi
  replicaCount: 1

Cluster Monitoring settings

PrometheusExt settings

The resource: sizes are controlled by the IBM Common Service Operator. The following sample configuration uses the medium resource values as an example.

- name: ibm-monitoring-prometheusExt
  spec:
    prometheusConfig:
      retention: 24h
      scrapeInterval: 1m
      logLevel: info
      pvSize: 10Gi
      nodeSelector:
        key1: value1
        key2: value2
      routerResource:
            requests:
              cpu: 10m
              memory: 50Mi
            limits:
              cpu: 75m
              memory: 50Mi
          resource:
            requests:
              cpu: 150m
              memory: 10200Mi
            limits:
              cpu: 230m
              memory: 13500Mi

    alertmanager:
      LogLevel: info  
      pvSize: 10Gi
      resource:
            requests:
              cpu: 30m
              memory: 50Mi
            limits:
              cpu: 30m
              memory: 50Mi
    storageClassName: myStorageClass

Configure resources for Monitoring PrometheusExt (ibm-monitoring-prometheusext-operator)

Specify resource settings. If you do not specify a setting, the default resource settings that are available in the cluster are used for the service.

  - name: ibm-monitoring-prometheusext-operator
    spec:
      prometheusExt:
        prometheusConfig:
          routerResource:
            requests:
              cpu:
              memory:
            limits:
              cpu:
              memory:
          resource:
            requests:
              cpu:
              memory:
            limits:
              cpu:
              memory:
        alertManagerConfig:
          resource:
            requests:
              cpu:
              memory:
            limits:
              cpu:
              memory:

Exporter settings

Example configuration with default values:

name: ibm-monitoring-exporters-operator
    spec:
      exporter:
        collectd:
          enable: true
          resource:
            requests:
              cpu: 30m
              memory: 50Mi
            limits:
              cpu: 30m
              memory: 50Mi
          routerResource:
            limits:
              cpu: 25m
              memory: 50Mi
            requests:
              cpu: 20m
              memory: 50Mi
        nodeExporter:
          enable: true
          resource:
            requests:
              cpu: 5m
              memory: 50Mi
            limits:
              cpu: 20m
              memory: 50Mi
          routerResource:
            requests:
              cpu: 50m
              memory: 128Mi
            limits:
              cpu: 100m
              memory: 256Mi
        kubeStateMetrics:
          enable: true
          resource:
            requests:
              cpu: 500m
              memory: 180Mi
            limits:
              cpu: 540m
              memory: 220Mi
          routerResource:
            limits:
              cpu: 25m
              memory: 50Mi
            requests:
              cpu: 20m
              memory: 50Mi

Configure resources for Monitoring Exporters (ibm-monitoring-exporters-operator)

Specify resource settings. If you do not specify a setting, the default resource settings that are available in the cluster are used for the service.

  - name: ibm-monitoring-exporters-operator
    spec:
      nodeSelector:
        key1: value1
        key2: value2
      exporter:
        collectd:
          resource:
            requests:
              cpu:
              memory:
            limits:
              cpu:
              memory:
          routerResource:
            limits:
              cpu:
              memory:
            requests:
              cpu:
              memory:
        nodeExporter:
          resource:
            requests:
              cpu:
              memory:
            limits:
              cpu:
              memory:
          routerResource:
            requests:
              cpu:
              memory:
            limits:
              cpu:
              memory:
        kubeStateMetrics:
          resource:
            requests:
              cpu:
              memory:
            limits:
              cpu:
              memory:
          routerResource:
            limits:
              cpu:
              memory:
            requests:
              cpu:
              memory:

Grafana settings

Example configuration with default values:

spec:
  nodeSelector:
    key1: value1
    Key2: value2
  grafanaConfig:
    resources:
            requests:
              cpu: 25m
              memory: 200Mi
            limits:
              cpu: 150m
              memory: 230Mi
  dashboardConfig:
     resources:
            requests:
              cpu: 25m
              memory: 140Mi
            limits:
              cpu: 70m
              memory: 170Mi
  routerConfig:
    routerConfig:
          resources:
            requests:
              cpu: 25m
              memory: 65Mi
            limits:
              cpu: 70m
              memory: 80Mi
  isHub: false

Using datasourceConfig parameter with Red Hat® OpenShift® Container Platform monitoring

By default IBM Cloud Pak foundational services Prometheus is installed and configured as data source for Grafana. If you want to use Red Hat® OpenShift® Container Platform monitoring Prometheus as your data source, you must add the following parameters to your configuration.

  - name: ibm-monitoring-grafana-operator
    spec:
      grafana:
        grafanaConfig:
          datasourceConfig:
            type" "openshift"

Configure resources for Monitoring Grafana (ibm-monitoring-grafana-operator)

Specify resource settings. If you do not specify a setting, the default resource settings that are available in the cluster are used for the service.

  - name: ibm-monitoring-grafana-operator
    spec:
      grafana:
        grafanaConfig:
          resources:
            requests:
              cpu:
              memory:
            limits:
              cpu:
              memory:
        dashboardConfig:
          resources:
            requests:
              cpu:
              memory:
            limits:
              cpu:
              memory:
        routerConfig:
          resources:
            requests:
              cpu:
              memory:
            limits:
              cpu:
              memory:

Certificate management service settings

• Configuring resources for Certificate manager (ibm-cert-manager-operator)
• certManagerCAInjector
• certManagerController
• certManagerWebhook
• Leaf certificate refresh parameters

Configuring resources for Certificate manager (ibm-cert-manager-operator)

Specify resource settings. If you do not specify a setting, the default resource settings are used.

  - name: ibm-cert-manager-operator
    spec:
      certManager:
        certManagerCAInjector:
          resources:
            limits:
              cpu: <cpu-limit (cores)>
              memory: <memory-limit (GB)>
            requests:
              cpu: <cpu-limit (cores)>
              memory: <memory-limit (GB)>
        certManagerController:
          resources:
            limits:
              cpu: <cpu-limit (cores)>
              memory: <memory-limit (GB)>
            requests:
              cpu: <cpu-limit (cores)>
              memory: <memory-limit (GB)>
        certManagerWebhook:
          resources:
            limits:
              cpu: <cpu-limit (cores)>
              memory: <memory-limit (GB)>
            requests:
              cpu: <cpu-limit (cores)>
              memory: <memory-limit (GB)>

certManagerCAInjector

certManagerController

certManagerWebhook

Leaf certificate refresh parameters

Example configuration with a CA certificate and namespace added for leaf refresh. Each certName requires a namespace. Leaf certificates based on the CA certificate will be refreshed when the CA certificate is refreshed by cert-manager.

- name: ibm-cert-manager-operator
    spec:
      refreshCertsBasedOnCA:
      - certName: sample-common-ca-cert
        namespace: sample-namespace

Common Web UI settings

Example configuration with default values:

- name: ibm-commonui-operator
    spec:
      commonWebUI:
        replicas: 1
        resources:
          requests:
            memory: 256Mi
            cpu: 300m
          limits:
            memory: 256Mi
            cpu: 300m

Serviceability settings

MustGather configuration settings

Example configuration with default values:

apiVersion: operator.ibm.com/v1alpha1
kind: MustGatherConfig
metadata:
  name: default
spec:
  gatherConfig: |-
    modules="overview,system,failure,ocp,cloudpak"
    namespaces="common-service,ibm-common-services"
    labels=""

MustGather job settings

Example configuration with default values:

apiVersion: operator.ibm.com/v1alpha1
kind: MustGatherJob
metadata:
  name: example-mustgatherjob
spec:
  serviceAccountName: default
  mustgatherConfigName: default

MustGather service settings

Note: The System Healthcheck service provides the live status of the service that is regenerated every 10 minutes. No backup or recovery is needed. The MustGather support information is used for troubleshooting purposes, and no backup or restore is needed.

Example configuration with default values:

apiVersion: operator.ibm.com/v1alpha1
kind: MustGatherService
metadata:
  name: must-gather-service
spec:
  # Add fields here
  persistentVolumeClaim:
    name: must-gather-pvc
    storageClassName: ""
    resources:
      requests:
        storage: 5Gi
  mustGather:
    name: must-gather-service
    replicas: 1
    serviceAccountName: default
    nodeSelector:
      node-role.kubernetes.io/worker: ""
    tolerations:
    - effect: NoSchedule
      key: dedicated
      operator: Exists
    - key: CriticalAddonsOnly
      operator: Exists
    securityContext:
      allowPrivilegeEscalation: false
      capabilities:
        drop:
        - ALL
      readOnlyRootFilesystem: true
      runAsNonRoot: true
    resources:
      requests:
        memory: "64Mi"
        cpu: "50m"
      limits:
        memory: "512Mi"
        cpu: "500m"

System Healthcheck service settings

Following is a sample CR:

apiVersion: operator.ibm.com/v1alpha1
kind: HealthService
metadata:
  name: system-healthcheck-service
spec:
  memcached:
    name: icp-memcached
    image:
      repository: quay.io/opencloudio/icp-memcached
      tag: 3.5.0
    replicaCount: 1
    serviceAccountName: ibm-healthcheck-operator-cluster
    nodeSelector:
      node-role.kubernetes.io/worker: ""
    tolerations:
    - effect: NoSchedule
      key: dedicated
      operator: Exists
    - key: CriticalAddonsOnly
      operator: Exists
    securityContext:
      allowPrivilegeEscalation: false
      capabilities:
        drop:
        - ALL
      readOnlyRootFilesystem: true
    command:
    - memcached
    - -m 64
    - -o
    - modern
    - -v
  healthService:
    name: system-healthcheck-service
    configmapName: system-healthcheck-service-config
    image:
      repository: quay.io/opencloudio/system-healthcheck-service
      tag: 3.5.0
    replicaCount: 1
    serviceAccountName: ibm-healthcheck-operator-cluster
    nodeSelector:
      node-role.kubernetes.io/worker: ""
    tolerations:
    - effect: NoSchedule
      key: dedicated
      operator: Exists
    - key: CriticalAddonsOnly
      operator: Exists
    securityContext:
      allowPrivilegeEscalation: false
      capabilities:
        drop:
        - ALL
      readOnlyRootFilesystem: true
      runAsNonRoot: true
    hostNetwork: false
    serviceNameSetting: Annotations:productName

System Healthcheck resources settings (ibm-healthcheck-operator)

Specify resource settings. If you do not specify a setting, the default resource settings that are available in the cluster are used for the System Healthcheck service.

  - name: ibm-healthcheck-operator
    spec:
      healthService:
        memcached:
          replicas: 1
          resources:
            requests:
              memory:
              cpu:
            limits:
              memory:
              cpu:
        healthService:
          replicas: 1
          resources:
            requests:
              memory:
              cpu:
            limits:
              memory:
              cpu:

Management Ingress settings

Example configuration with default values:

- name: ibm-management-ingress-operator
  spec:
    managementIngress:
      replicas: 1
      resources:
        limits:
          cpu: 200m
          memory: 240Mi
        requests:
          cpu: 1800m
          memory: 195Mi

Nginx Ingress settings

Note: Nginx ingress is deprecated with installer version 3.10.0 and might be removed in a future release.

Example configuration with default values:

- name: ibm-ingress-nginx-operator
  spec:
    nginxIngress:
      defaultBackend:
        replicas: 1
        resources:
          limits:
            cpu: 20m
            memory: 50Mi
          requests:
            cpu: 20m
            memory: 50Mi
      ingress:
        replicas: 1
        resources:
          limits:
            cpu: 100m
            memory: 225Mi
          requests:
            cpu: 100m
            memory: 140Mi            
      kubectl:
        resources:
          limits:
            cpu: 30m
            memory: 150Mi
          requests:
            cpu: 30m
            memory: 150Mi

Metering settings

Note: The deprecated Metering service is removed in IBM Cloud Pak® foundational services version 3.7.x.

Specify resource settings. If you do not specify a setting, the default resource settings that are available in the cluster are used for the metering service.

  - name: ibm-metering-operator
    spec:
      metering:
        dataManager:
          dm:
            resources:
              limits:
                cpu:
                memory:
              requests:
                cpu:
                memory:
        reader:
          rdr:
            resources:
              limits:
                cpu:
                memory:
              requests:
                cpu:
                memory:
      meteringReportServer:
        reportServer:
          resources:
            limits:
              cpu:
              memory:
            requests:
              cpu:
              memory:
      meteringUI:
        replicas: 1
        ui:
          resources:
            limits:
              cpu:
              memory:
            requests:
              cpu:
              memory:

Common UI settings

Specify resource settings. If you do not specify a setting, the default resource settings that are available in the cluster are used for the common UI service.

  - name: ibm-commonui-operator
    spec:
      commonWebUI:
        replicas: 1
        resources:
          requests:
            memory:
            cpu:
          limits:
            memory:
            cpu: