Manually scaling resources for services
You can adjust IBM Cloud Pak® for Data services by scaling the resources that they use to support high availability or to increase processing capacity. Resources can be scaled based on predefined resource configurations.
- Permissions that you need for this task
- You must be either:
- A cluster administrator
- An administrator of the project where the software is installed, for example
cpd-instance
- When you need to complete this task
- If a service supports scaling, you can scale the service at any time after you install it.
Scaling changes the capacity of services by adjusting the memory resource settings, the number of CPUs, and the number of pods that are available. Scaling a service from small to medium increases the processing capacity of the application.
Use the scaleConfig
setting in the service custom resource (CR) to set the
scaling configuration for a service.
The following table shows the services that support manual scaling. Click a service name or scroll to the procedure for manually scaling services for detailed instructions about how to scale each service.
Service name | Uses the scaleConfig setting | Uses a different scaling method |
---|---|---|
Cloud Pak for Data control plane | ✓ | |
Common core services | ✓ | |
IBM Cloud Pak foundational services | ✓ | |
AI Factsheets | ✓ | |
Analytics Engine powered by Apache Spark | ✓ | |
Cognos® Analytics | ✓ | |
Cognos Dashboards | ✓ | |
Data Privacy | ✓ | |
Data Refinery | ✓ | |
Data Replication | ✓ | |
DataStage® | ✓ | |
Db2® | ✓ | |
Db2 Big SQL | ✓ | |
Db2 Data Gate | ||
Db2 Data Management Console | ✓ | |
Db2 Warehouse | ✓ | |
Decision Optimization | ✓ | |
EDB Postgres | ||
Execution Engine for Apache Hadoop | ||
IBM Match 360 with Watson™ | ✓ | |
Informix® | ✓ | |
MANTA Automated Data Lineage | ✓ | |
OpenPages® | ✓ | |
Planning Analytics | ✓ | |
Product Master | ✓ | |
RStudio® Server Runtimes | ||
Scheduling service | ✓ | |
SPSS® Modeler | ✓ | |
Voice Gateway | ✓ | |
Watson Assistant | ✓ | |
Watson Discovery | ✓ | |
Watson Knowledge Catalog | ✓ | |
Watson Knowledge Studio | ✓ | |
Watson Machine Learning | ✓ | |
Watson Machine Learning Accelerator | ✓ | |
Watson OpenScale | ✓ | |
Watson Pipelines | ✓ | |
Watson Query | ✓ | |
Watson Speech services | ✓ | |
Watson Studio | ✓ | |
Watson Studio Runtimes | ||
watsonx.data | ✓ |
Before you begin
In most cases, you use the cpd-cli manage update-cr command to set the scaleConfig setting to scale services.
Some services support scaling but do not support the use of the cpd-cli manage update-cr, or do not support the use of the scaleConfig parameter.
Before you scale up a service, ensure that your cluster can support the additional workload. If necessary, contact your IBM Support representative.When you change the scaling configuration of a service, you also must scale the related services individually. For more information about service dependencies, see Service software requirements, and then scale the services as necessary.
Ensure that you source the environment variables before you run the commands in this task.
Procedure
Cloud Pak for Data control plane
- Set the
SIZE
environment variable to the appropriate value for your environment.export SIZE=<size>
The following sizes are supported:- small_mincpureq (Use this setting when you want to use minimal reserved CPU resources without removing the CPU pod setting. For example, you can use this setting with Express Parts installations.)
- small (default)
- medium
- large
- Run the following command to scale the component by updating the custom
resource::
oc patch zenservice lite-cr \ --namespace ${PROJECT_CPD_INST_OPERANDS} \ --type="json" \ --patch '[{"op": "replace", "path":"/spec/scaleConfig", "value":'${SIZE}' }]'
Common core services
- Set the
SIZE
environment variable to the appropriate value for your environment.export SIZE=<size>
The following sizes are supported:- small_mincpureq (Use this setting when you want to use minimal reserved CPU resources without removing the CPU pod setting. For example, you can use this setting with Express Parts installations.)
- small (default)
- medium
- large
- Run the following command to scale the common core services:
oc patch ccs ccs-cr \ --namespace ${PROJECT_CPD_INST_OPERANDS} \ --type="json" \ --patch '[{"op": "replace", "path":"/spec/scaleConfig", "value":'${SIZE}' }]'
You can optionally scale the Flight service when you scale the common core services. To scale the Flight service, you must adjust the number of replicas and the amount of vCPU that is allocated to Flight service pods:
- Small
-
oc patch ccs ccs-cr -n ${PROJECT_CPD_INST_OPERANDS} --type merge --patch ' { "spec":{ "wdp_connect_flight_min_replicas": 1, "wdp_connect_flight_resources":{"requests":{"cpu": "200m", "memory": "650Mi"}, "limits":{"cpu": "2", "memory": "4Gi"}} } } '
- Medium
-
oc patch ccs ccs-cr -n ${PROJECT_CPD_INST_OPERANDS} --type merge --patch ' { "spec":{ "wdp_connect_flight_min_replicas": 4, "wdp_connect_flight_resources":{"requests":{"cpu": "1", "memory": "650Mi"}, "limits":{"cpu": "2", "memory": "4Gi"}} } } '
- Large
-
oc patch ccs ccs-cr -n ${PROJECT_CPD_INST_OPERANDS} --type merge --patch ' { "spec":{ "wdp_connect_flight_min_replicas": 8, "wdp_connect_flight_resources":{"requests":{"cpu": "2", "memory": "650Mi"}, "limits":{"cpu": "4", "memory": "4Gi"}} } } '
IBM Cloud Pak foundational services
For information about how to scale IBM Cloud Pak foundational services, see Configuring IBM Cloud Pak foundational services by using the CommonService custom resource.AI Factsheets
- Set the
SIZE
environment variable to the appropriate value for your environment.export SIZE=<size>
The following sizes are supported:- small (default)
- medium
- large
- Run the following command to scale the component by updating the custom
resource:
cpd-cli manage update-cr \ --component=factsheet \ --cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \ --patch="{\"scaleConfig\":\"${SIZE}\"}"
Analytics Engine powered by Apache Spark
- Set the
SIZE
environment variable to the appropriate value for your environment.export SIZE=<size>
The following sizes are supported:- small (default)
- medium
- large
- Run the following command to scale the component by updating the custom
resource:
cpd-cli manage update-cr \ --component=analyticsengine \ --cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \ --patch="{\"scaleConfig\":\"${SIZE}\"}"
Cognos Analytics
fixed minimum
(no scale)small
medium
large
For detailed information about scaling a service instance, see Provisioning the Cognos Analytics service.
You can also change the size of the instance after it is already provisioned. For more information, see Changing the plan size of the Cognos Analytics service instance.
Cognos Dashboards
- Set the
SIZE
environment variable to the appropriate value for your environment.export SIZE=<size>
The following sizes are supported:- small (default)
- medium
- large
- Run the following command to scale the component by updating the custom
resource:
cpd-cli manage update-cr \ --component=dashboard \ --cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \ --patch="{\"scaleConfig\":\"${SIZE}\"}"
Data Privacy
- Set the
SIZE
environment variable to the appropriate value for your environment.export SIZE=<size>
The following sizes are supported:- small (default)
- medium
- large
- Run the following command to scale the component by updating the custom
resource:
cpd-cli manage update-cr \ --component=dp \ --cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \ --patch="{\"scaleConfig\":\"${SIZE}\"}"
Data Refinery
Data Refinery does not support the
cpd-cli
manage
update-cr
command to set value of the
scaleConfig setting.
- Set the
SIZE
environment variable to the appropriate value for your environment.export SIZE=<size>
The following sizes are supported:- small (default)
- medium
- large
- Run the following command to scale the component by updating the custom
resource:
oc patch DataRefinery datarefinery-sample \ -n ${PROJECT_CPD_INST_OPERANDS} \ --type=merge -p '{"spec":{"scaleConfig":"size"}}'
If you changed the default name of the Data Refinery custom resource, replace datarefinery-sample in the command.
Data Replication
- Set the
SIZE
environment variable to the appropriate value for your environment.export SIZE=<size>
The following sizes are supported:- small (default)
- medium
- large
- Run the following command to scale the component by updating the custom
resource:
oc patch replicationservice replicationservice-cr \ --namespace ${PROJECT_CPD_INST_OPERANDS} \ --patch '{"spec":{"scaleConfig":"${SIZE}"}}' \ --type=merge
If you changed the default name of the Data Replication custom resource, replace replicationservice-cr in the command.
DataStage
- Set the
SIZE
environment variable to the appropriate value for your environment.export SIZE=<size>
The following sizes are supported:- small (default)
- medium
- large
- Run the following command to scale the component by updating the custom
resource:
cpd-cli manage update-cr \ --component=datastage_ent \ --cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \ --patch="{\"scaleConfig\":\"${SIZE}\"}"
Db2
To scale Db2, work with your IBM Sales representative to size your cluster.
For more information about scaling Db2, see Scaling up Db2.
Db2 Big SQL
You can scale a Db2 Big SQL service instance from the instance details page in the Cloud Pak for Data web client.
For information about scaling Db2 Big SQL, see Updating instance configuration.
Db2 Data Management Console
- Set the
SIZE
environment variable to the appropriate value for your environment.export SIZE=<size>
The following sizes are supported:- small (default)
- medium
- large
- Run the following command to scale the component by updating the custom
resource:
oc patch dmc <dmcs_cr_name> \ --namespace ${PROJECT_CPD_INST_OPERANDS} \ --patch '{"spec":{"scaleConfig":"${SIZE}"}}' \ --type=merge
Replace <dmcs_cr_name> with the name of the Db2 Data Management Console custom resource.
Db2 Warehouse
To scale Db2 Warehouse, work with your IBM Sales representative to size your cluster.
For more information about scaling Db2 Warehouse, see Scaling Db2 Warehouse.
Decision Optimization
- Set the
SIZE
environment variable to the appropriate value for your environment.export SIZE=<size>
The following sizes are supported:- small (default)
- medium
- large
- Run the following command to scale the component by updating the custom
resource:
cpd-cli manage update-cr \ --component=dods \ --cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \ --patch="{\"scaleConfig\":\"${SIZE}\"}"
IBM Match 360 with Watson
Work with your IBM Sales representative to get accurate sizing based on your expected workload.
- Set the
SIZE
environment variable to the appropriate value for your environment.export SIZE=<size>
The following sizes are supported:
- x-small (demonstration use only)
- small_mincpureq (Use this setting when you want to use minimal reserved CPU resources without removing the CPU pod setting.)
- small (default)
- medium
- large
- Run the following command to scale the component by updating the custom
resource:
oc patch mdm mdm-cr \ --namespace ${PROJECT_CPD_INST_OPERANDS} \ --patch '{"spec":{"scaleConfig":"${SIZE}"}}' \ --type=merge
Informix
Informix does not support the cpd-cli
manage
update-cr
command to scale the service.
You can run the following commands to adjust the amount of CPU and memory that are allocated to an Informix service instance:
- From the Informix instance details page
in the Cloud Pak for Data web client.
- Get the deployment ID of the instance.
- Review the current configuration to determine the number of nodes, the number of cores, and the amount of memory allocated to the instance.
- Set the
INSTANCE_ID
environment variable:export INSTANCE_ID=<instance-id>
- To scale the number of Informix servers
in the service instance,
run:
oc patch Informixes informix-${INSTANCE_ID} \ --namespace ${PROJECT_CPD_INST_OPERANDS} --patch '{"spec":{"replicas":2}}' \ --type=merge
- To increase the CPU or memory that the service instance can use, run the appropriate command:
- Increase memory limits
-
The following example sets the memory limits to 4 GiB.
oc patch Informixes informix-${INSTANCE_ID}-server \ --namespace ${PROJECT_CPD_INST_OPERANDS} \ --patch '{"spec":{"resources":{"limits":{"memory":"4Gi"}}}}' \ --type=merge
- Increase memory
-
The following example sets the memory requests to 2 GiB.
oc patch Informixes informix-${INSTANCE_ID}-server \ --namespace ${PROJECT_CPD_INST_OPERANDS} \ --patch '{"spec":{"resources":{"requests":{"memory":"2Gi"}}}}' \ --type=merge
- Increase CPU limits
-
The following example sets the CPU limits to 4 CPU.
oc patch Informixes informix-${INSTANCE_ID}-server \ --namespace ${PROJECT_CPD_INST_OPERANDS} \ --patch '{"spec":{"resources":{"limits":{"cpu":"4"}}}}' \ --type=merge
- Increase CPU
-
The following example sets the CPU requests to 2 CPU.
oc patch Informixes informix-${INSTANCE_ID}-server \ --namespace ${PROJECT_CPD_INST_OPERANDS} \ --patch '{"spec":{"resources":{"requests":{"cpu":"2"}}}}' \ --type=merge
- Delete the
StatefulSet
for the changes to take effect:oc delete sts informix-${INSTANCE_ID}-server
MANTA Automated Data Lineage
- Set the
SIZE
environment variable to the appropriate value for your environment.export SIZE=<size>
The following sizes are supported:- small (default)
- medium
- large
- Run the following command to scale the component by updating the custom
resource:
cpd-cli manage update-cr \ --component=mantaflow \ --cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \ --patch="{\"scaleConfig\":\"${SIZE}\"}"
OpenPages
- Extra Small (XS)
- Small
- Medium
- Large
For more information about scaling a service instance, see Scaling OpenPages.
Planning Analytics
When you provision a Planning Analytics service instance, you choose a size for the instance.
- Set the
SIZE
environment variable to the appropriate value for your environment.export SIZE=<size>
The following sizes are supported:- small_mincpureq (Use this setting when you want to use minimal reserved CPU resources without removing the CPU pod setting. For example, you can use this setting with Express Parts installations.)
- small (default)
- medium
- large
- Set the
INSTANCE_NAME
environment variable to the appropriate value for your environment.export INSTANCE_NAME=<instance-name>
- Run the following command to scale the component by updating the
custom
resource:
oc patch PAServiceInstance ${INSTANCE_NAME} \ --namespace ${PROJECT_CPD_INST_OPERANDS} \ --patch '{"spec":{"scaleConfig":"'${SIZE}'"}}' \ --type=merge
For more information about scaling a service instance, see Provisioning the Planning Analytics service.
Product Master
- Small
- Medium
- Large
- Extra large (XLarge)
For more information about scaling a service instance, see Scaling Product Master instances.
Scheduling service
- Set the
SIZE
environment variable to the appropriate value for your environment.export SIZE=<size>
The following sizes are supported:
- small: Up to 50 worker nodes.
- medium: Up to 100 worker nodes. One standby pod is created.
- large: More than 100 worker nodes. One standby pod is created.
- Run the following command to scale the component by updating the custom
resource:
- Workstations with
olm-utils-v2:4.7.2
or later -
cpd-cli manage update-cr \ --component=scheduler \ --cluster_component_ns=${PROJECT_SCHEDULING_SERVICE } \ --patch="{\"scaleConfig\":\"${SIZE}\"}"
- Workstations with
olm-utils-v2:4.7.0
orolm-utils-v2:4.7.1
-
oc patch scheduling ibm-cpd-scheduler \ -n ${PROJECT_SCHEDULING_SERVICE} \ --type merge \ -p "{\"spec\":{\"scaleConfig\": \"${SIZE}\"}}"
- Workstations with
SPSS Modeler
- Set the
SIZE
environment variable to the appropriate value for your environment.export SIZE=<size>
The following sizes are supported:- small (default)
- medium
- large
- Run the following command to scale the component by updating the custom
resource:
cpd-cli manage update-cr \ --component=spss \ --cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \ --patch="{\"scaleConfig\":\"${SIZE}\"}"
Voice Gateway
You can scale Voice Gateway by changing the number of replicas.
- Get the list of Voice Gateway instances
in the
${PROJECT_CPD_INST_OPERANDS}
project:oc get voicegateway --namespace ${PROJECT_CPD_INST_OPERANDS}
- Set the
INSTANCE_NAME
environment variable to the instance of Voice Gateway that you want to scale:export INSTANCE_NAME=<instance-name>
- Get the current number of replicas for the
instance:
oc get voicegateway ${INSTANCE_NAME} \ --namespace ${PROJECT_CPD_INST_OPERANDS} \ -o=jsonpath='{..status.replicas}'
Use the output of this command to determine the appropriate number of replicas for the instance.
- Set the
REPLICAS
environment variable to the number of replicas to use for the instance:export REPLICAS=<number-of-replicas>
- Scale the
instance:
oc scale voicegateway ${INSTANCE_NAME} \ --namespace ${PROJECT_CPD_INST_OPERANDS} \ --replicas=${REPLICAS}
Watson Assistant
- Set the
SIZE
environment variable to the appropriate value for your environment.export SIZE=<size>
The following sizes are supported:small
You can specify either
small
orStarter
medium
(default)You can specify either
medium
orProduction
large
- Run one of the following commands to scale the component by updating the custom
resource:
-
cpd-cli manage update-cr \ --component=watson_assistant \ --cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \ --patch="{\"size\":\"${SIZE}\"}"
-
oc project ${PROJECT_CPD_INST_OPERANDS} export INSTANCE=`oc get wa -n ${PROJECT_CPD_INST_OPERANDS} |grep -v NAME| awk '{print $1}'` oc patch wa ${INSTANCE} --namespace ${PROJECT_CPD_INST_OPERANDS} --patch "{\"spec\":{\"size\":\"${SIZE}\"}}" --type=merge
-
Watson Discovery
- Set the
SIZE
environment variable to the appropriate value for your environment.export SIZE=<size>
The following sizes are supported:- xsmall
- _default
- medium
- largeImportant: Scaling Watson Discovery from medium to large significantly increases the number of replicas. Be sure that you want to make this change because it is not recommended that you scale backwards.
- Run the following command to scale the component by updating the custom
resource:
cpd-cli manage update-cr \ --component=watson_discovery \ --cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \ --patch="{\"scaleConfig\":\"${SIZE}\"}"
Watson Knowledge Catalog
- Set the
SIZE
environment variable to the appropriate value for your environment.export SIZE=<size>
The following sizes are supported:- small (default)
- medium
- large
- Run the following command to scale the component by updating the custom
resource:
cpd-cli manage update-cr \ --component=wkc \ --cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \ --patch="{\"scaleConfig\":\"${SIZE}\"}"
For more information on scaling the Db2 database that is associated with Watson Knowledge Catalog, see Scaling up Db2 for Watson Knowledge Catalog.
Watson Knowledge Studio
- Set the
SIZE
environment variable to the appropriate value for your environment.export SIZE=<size>
The following sizes are supported:- small
- medium
- Run the following command to scale the component by updating the custom
resource:
oc -n ${PROJECT_CPD_INST_OPERANDS} patch --type=merge KnowledgeStudio \ wks -p "{\"spec\":{\"global\":{\"size\":\"${SIZE}\"}}}"
Watson Machine Learning
- Set the
SIZE
environment variable to the appropriate value for your environment.export SIZE=<size>
The following sizes are supported:- small_mincpureq (Use this setting when you want to use minimal reserved CPU resources without removing the CPU pod setting. For example, you can use this setting with Express Parts installations.)
- small (default)
- medium
- Run the following command to scale the component by updating the custom
resource:
oc patch wmlbase wml-cr \ --namespace ${PROJECT_CPD_INST_OPERANDS} \ --patch "{\"spec\":{\"scaleConfig\":\"${SIZE}\"}}" \ --type=merge
If you changed the default name of the Watson Machine Learning custom resource, replace wmlbase in the command.
Watson Machine Learning Accelerator
- Set the
SIZE
environment variable to the appropriate value for your environment.export SIZE=<size>
The following sizes are supported:- small (default)
- medium
- large
- Run the following command to scale the component by updating the custom
resource:
cpd-cli manage update-cr \ --component=wml_accelerator \ --cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \ --patch="{\"scaleConfig\":\"${SIZE}\"}"
Watson OpenScale
- Set the
SIZE
environment variable to the appropriate value for your environment.export SIZE=<size>
The following sizes are supported:- small
- medium
- Run the following command to scale the component by updating the custom
resource:
cpd-cli manage update-cr \ --component=openscale \ --cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \ --patch="{\"scaleConfig\":\"${SIZE}\"}"
Watson Pipelines
- Set the
SIZE
environment variable to the appropriate value for your environment.export SIZE=<size>
The following sizes are supported:- small (default)
- medium
- large
- Run the following command to scale the component by updating the custom
resource:
cpd-cli manage update-cr \ --component=ws_pipelines \ --cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \ --patch="{\"scaleConfig\":\"${SIZE}\"}"
Watson Query
Watson Query does not use thespec.scaleConfig
value in the custom resource for scaling. Work with your IBM Sales
representative to get an accurate sizing based on your expected workload. IBM Sales can help you
estimate the total demand for Watson Query.For more information about the process for scaling Watson Query, see Scaling Watson Query.
Watson Speech services
- Set the
SIZE
environment variable to the appropriate value for your environment.export SIZE=<size>
The following sizes are supported:- xsmall
- small
- medium
- large
- Run the following command to scale the component by updating the custom
resource:
For Watson Speech to Text:
cpd-cli manage update-cr \ --component=watson_speech \ --cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \ --patch="{\"scaleConfig\":{\"stt\":{\"size\":\"${SIZE}\"}}}"
For Watson Text to Speech:
cpd-cli manage update-cr \ --component=watson_speech \ --cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \ --patch="{\"scaleConfig\":{\"tts\":{\"size\":\"${SIZE}\"}}}"
Watson Studio
- Set the
SIZE
environment variable to the appropriate value for your environment.export SIZE=<size>
The following sizes are supported:- small (default)
- medium
- large
- Run the following command to scale the component by updating the custom
resource:
cpd-cli manage update-cr \ --component=ws \ --cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \ --patch="{\"scaleConfig\":\"${SIZE}\"}"
Watson Studio Runtimes
Watson Studio Runtimes are started with the requested resources. You cannot scale up the started runtime. Every runtime is user and project bound.
watsonx.data
- Set the
SIZE
environment variable to the appropriate value for your environment.export SIZE=<size>
The following sizes are supported:- small (default)
- medium
- large
- xlarge
- Run the following command to scale the addon components UI, API, and
Nodeclient:
oc patch wxdAddon wxdaddon \ --type=merge \ -n ${PROJECT_CPD_INST_OPERANDS} \ -p '{ "spec": { "scaleConfig": "$SIZE" } }'
- Run the following command to scale the instance components Postgres, Hive-metastore, and
MinIO:
oc patch wxd lakehouse \ --type=merge \ -n ${PROJECT_CPD_INST_OPERANDS} \ -p '{ "spec": { "scaleConfig": "$SIZE" } }'
- Follow the steps to scale Presto engine components Presto, Presto-coordinator, and Presto-worker:
- Run the following command to get the Presto engine
name:
oc get wxdengine
oc get wxdengine NAME AGE lakehouse-presto-01 75m
- Run the following command to scale Presto components in the
pods:
oc patch wxdengine/lakehouse-presto-01 \ --type=merge \ -n ${PROJECT_CPD_INST_OPERANDS} \ -p '{ "spec": { "scaleConfig": "small", "lakehouseDeploymentType": "multinode" } }'
- Run the following command to get the Presto engine
name: