Option 1a: Mirroring catalogs to a private registry with a bastion server
You can use a bastion host to mirror the catalogs to a private registry for an airgap deployment.
About this task
The bastion host must be connected to both the private registry and the Red Hat OpenShift
Container Platform cluster. When you mirror the images, you must run the oc mirror
commands on the bastion host. The IBM Catalog Management plug-in or ibm-pak
uses
File-based catalogs (FBC) to mirror all the catalogs. The FBC is in JSON or YAML format that
contains information that is needed to install an operator into the OpenShift Container Platform
.
release.major.minor
standard. For example, the first interim fix
for 23.0.2 is
packaged in the archive ibm-cp-automation-5.1.1.tgz. Procedure
- Set the environment variables and authenticate the registries.
- Set the environment variable $TARGET_REGISTRY by running the following
command.
export TARGET_REGISTRY=<target-registry>
The <target-registry> refers to the registry (fully qualified hostname and port) where the images are mirrored to and accessed by the OpenShift Container Platform cluster. For example:
localhost.localdomain:5000
or172.16.0.10:5000
. - Set the following environment variables with the installer image name and the version.
export CASE_NAME=ibm-cp-automation export CASE_VERSION=5.1.0
- Run the following command to set the preferred tool parameter in the YAML file to
oc-mirror
.oc ibm-pak config mirror-tools --enabled oc-mirror
- Authenticate the registries.
You must store authentication credentials for all the Cloud Pak for Business Automation source Docker registries. The following registries require authentication:
cp.icr.io
registry.redhat.io
registry.access.redhat.com
If you plan to run Podman as the non-root user, see Basic Setup and Use of Podman in a Rootless environment.
Set the environment variable XDG_RUNTIME_DIR by running the following commands:
whereexport XDG_RUNTIME_DIR=/run/user/${UID} export REGISTRY_AUTH_FILE=$XDG_RUNTIME_DIR/containers/auth.json
${UID}
is the user id of the current user. Make sure that the directory /run/user/${UID} exists and the current user must have write permission.Note: Run the following command as the non-root user if you do not want to usesudo podman
:mkdir -p /run/user/<ID of non-root> && chown <non-root-user>:<non-root-group> /run/user/<ID of non-root>
If you are using Podman, run the following command to configure credentials for all target registries that require authentication. You must run the command separately for each registry.
export REGISTRY_AUTH_FILE=<path to the file that has the auth credentials generated on podman login> podman login cp.icr.io podman login <TARGET_REGISTRY>
If you are using docker, run the following command to configure credentials for all target registries that require authentication. You must run the command separately for each registry.export REGISTRY_AUTH_FILE=<path to the file that has the auth credentials generated on docker login> docker login cp.icr.io docker login <TARGET_REGISTRY>
Important: When you log in tocp.icr.io
, you must specify the user ascp
and the IBM entitlement key as the password. For example:podman login cp.icr.io Username: cp Password: xxxxxxxxxxxxxxxxxxxxx Login Succeeded!
The password can be copied from the IBM container library. You can add
--tls-verify=false
to the command, if you see "cert error" messages.If you export
REGISTRY_AUTH_FILE=~/.ibm-pak/auth.json
, and then run thepodman login
command, you can see that the file is populated with registry credentials.If you usedocker login
, the authentication file is typically located in$HOME/.docker/config.json
on Linux or%USERPROFILE%/.docker/config.json
on Windows. After you run thedocker login
command, you can exportREGISTRY_AUTH_FILE
to point to that location. For example, on Linux you can run the following command:export REGISTRY_AUTH_FILE=$HOME/.docker/config.json
- Set the environment variable $TARGET_REGISTRY by running the following
command.
- Run the following command to generate mirror manifests to be used when mirroring the
catalog to the target registry. The $TARGET_REGISTRY refers to the registry where
the catalogs are mirrored to and accessed by the OpenShift Container Platform cluster.
oc ibm-pak generate mirror-manifests $CASE_NAME $TARGET_REGISTRY \ --version $CASE_VERSION
Important: Thegenerate mirror-manifests
command provides an output that lists the command for mirroring. Save the command for later use in step 3.The following example shows sample output.oc ibm-pak generate mirror-manifests ibm-cp-automation --version 5.1.0 docker-na-public.artifactory.swg-devops.com/ecm-containerization-docker-local/cp4ba-2302 ibm-cloud-native-postgresql-4.14.0+20230614.093000 done ibm-licensing-4.2.0 done ibm-cp-common-services-4.2.0 done ibm-cs-flink-1.3.14 done ibm-cs-mongodb-4.2.0 done ibm-cs-elastic-1.3.14 done ibm-cp-automation-5.1.0 ●∙∙ 13s ibm-events-operator-4.8.0 done ibm-cert-manager-4.2.0 done ibm-cs-iam-4.2.0 done ibm-zen-3.0.2+20230922.110438 done ibm-cs-commonui-4.2.0 done ibm-bts-bundle-1.0.20 done ibm-cp-fncm-case-1.8.0 done Generating mirror manifests of CASE: ibm-cp-automation, version: 5.1.0 is complete Next steps - To mirror the catalog: oc mirror --config /root/airgap-3.0/.ibm-pak/data/mirror/ibm-cp-automation/5.1.0/image-set-config.yaml docker://docker-na-public.artifactory.swg-devops.com/ecm-containerization-docker-local/cp4ba-2302 --dest-skip-tls --max-per-registry=6
A new directory $IBMPAK_HOME/.ibm-pak/mirror is created when you issue the
oc ibm-pak generate mirror-manifests
command. The mirror directory stores the catalog-sources.yaml, image-content-source-policy.yaml, and image-set-config.yaml files.Tip: If you are using a Red Hat® Quay.io registry and need to mirror the images to a specific organization in the registry, you can set the target to that organization. Specify the organization name in thegenerate mirror-manifests
command:export ORGANIZATION=<your-organization> \ oc ibm-pak generate mirror-manifests $CASE_NAME $TARGET_REGISTRY/$ORGANIZATION \ --version $CASE_VERSION
Restriction: Currently, you cannot select the images to mirror by their target architecture because image registries do not support sparse manifests (manifests that reference image digests outside of the package).However, you can use image groups for the
mirror-images
action to mirror a subset of the images. You can use the--filter
option to limit what images are mirrored for Cloud Pak for Business Automation capability specific images. By selecting any specified filter all the dependency foundational services images and other dependencies are also mirrored.The following table provides the image groups for each capability.
Table 1. Groups for mirroring a subset of images Target capabilities Image groups, including operator-related images Automation Decision Services ibmcp4baProd,ibmcp4baADSImages,ibmcp4baBANImages,ibmcp4baBASImages,ibmcp4baAAEImages,ibmEdbStandard Automation Document Processing ibmcp4baProd,ibmcp4baADPImages,ibmcp4baFNCMImages,ibmcp4baBANImages,ibmcp4baBASImages,ibmcp4baAAEImages,ibmEdbStandard Automation Workstream Services ibmcp4baProd,ibmcp4baBAWImages,ibmcp4baPFSImages,ibmcp4baFNCMImages,ibmcp4baBANImages,ibmcp4baBASImages,ibmcp4baAAEImages,ibmEdbStandard Business Automation Application ibmcp4baProd,ibmcp4baBASImages,ibmcp4baAAEImages,ibmEdbStandard Business Automation Workflow ibmcp4baProd,ibmcp4baBAWImages,ibmcp4baFNCMImages,ibmcp4baBANImages,ibmcp4baBASImages,ibmcp4baAAEImages,ibmEdbStandard,ibmcp4baUMSImages Note: To install Process Federation Server with Business Automation Workflow, add ibmcp4baPFSImages to the list of groups.Process Federation Server ibmcp4baProd,ibmcp4baPFSImages,ibmcp4baAAEImages Note: To enable Workplace in the Common UI console, add ibmcp4baBANImages,ibmEdbStandard to the list of groups.IBM FileNet® Content Manager ibmcp4baProd,ibmcp4baFNCMImages,ibmcp4baBANImages,ibmcp4baAAEImages,ibmEdbStandard Operational Decision Manager ibmcp4baProd,ibmcp4baODMImages,ibmcp4baBASImages,ibmcp4baAAEImages,ibmEdbStandard Workflow Process Service ibmcp4baProd,ibmcp4baAAEImages,ibmcp4baBASImages,ibmcp4baWFPSImages,ibmEdbStandard Note: To install Process Federation Server with Workflow Process Service Runtime, add ibmcp4baPFSImages to the list of groups. To enable Workplace in the Common UI console, add ibmcp4baBANImages to the list of groups.Business Automation Insights standalone ibmcp4baProd,ibmcp4baBAIImages,ibmEdbStandard Note:- If Business Automation Insights (BAI) is needed for FileNet Content Manager, Operational Decision Manager, Business Automation Workflow, or Workflow Process Service, then add ibmcp4baBAIImages and ibmEdbStandard to the list of groups.
- All the images in the groups are pulled irrespective of architecture.
For example, to download FNCM images from CP4BA, and all the dependent product images, run the following command.
oc ibm-pak generate mirror-manifests $CASE_NAME $TARGET_REGISTRY \ --version $CASE_VERSION \ --filter ibmcp4baProd,ibmcp4baFNCMImages,ibmcp4baBANImages,ibmcp4baAAEImages,ibmEdbStandard
The
$IBMPAK_HOME/.ibm-pak
directory structure is built over time as you save CASEs and mirror. The following tree shows an example of the$IBMPAK_HOME/.ibm-pak
directory structure:tree ~/.ibm-pak /root/.ibm-pak ├── config │ └── config.yaml ├── data │ ├── cases │ │ └── ibm-cp-automation │ │ └── 5.1.x │ │ ├── XXXXX │ │ ├── XXXXX │ └── mirror │ └── ibm-cp-automation │ └── 5.1.x │ ├── catalog-sources.yaml │ ├── image-content-source-policy.yaml │ └── image-set-config.yaml └── logs └── oc-ibm_pak.log
-
Run the command generated in step 2 to mirror the catalogs. Note: By default the
oc mirror
command mirrors all the images from all channels in the CP4BA multi-pattern catalog and CP4BA FNCM catalog. You can edit the$IBMPAK_HOME/.ibm-pak/data/mirror/ibm-cp-automation/5.1.0/image-set-config.yaml
file to remove the OLM channels and keep only the v23.2 channel. This is only applicable for new installations.The following is a sample of
image-set-config.yaml
:- name: ibm-cp4a-operator channels: - name: v21.3 - name: v22.1 - name: v22.2 - name: v23.1 - name: v23.2
The following is an example command.oc mirror --config /root/cp4ba/.ibm-pak/data/mirror/ibm-cp-automation/5.1.0/image-set-config.yaml docker://$TARGET_REGISTRY/cp4ba --dest-skip-tls --max-per-registry=6
You must run the command on the host that is connected to both the local Docker registry and the OpenShift Container Platform cluster.
The command does not produce any console logs for about 6 - 8 minutes as it prepares the list from the CASE package. If you want, you can add verbose (-v) to the command with possible values of 1 to 9.
The following command can be used to see all the available options.oc mirror --help
Based on the number and size of the images to mirror, theoc mirror
command can take a considerable amount of time. If you are running the command on a remote system, run the command in the background with thenohup
POSIX command so that it does not stop if the user logs out. The following command starts the mirroring process in the background and writes the log to a cp4ba-510.txt file.nohup oc mirror --config /root/cp4ba/.ibm-pak/data/mirror/$CASE_NAME/$CASE_VERSION/image-set-config.yaml docker://$TARGET_REGISTRY/cp4ba-2301 --dest-skip-tls --max-per-registry=6 > /opt/cp4ba-510.txt 2>&1 &
The following example shows sample output.
nohup: ignoring input Logging to .oc-mirror.log Checking push permissions for docker-na-public.artifactory.swg-devops.com Creating directory: oc-mirror-workspace/src/publish Creating directory: oc-mirror-workspace/src/v2 Creating directory: oc-mirror-workspace/src/charts Creating directory: oc-mirror-workspace/src/release-signatures No metadata detected, creating new workspace 1145 related images processed in 1m2.231903162s Writing image mapping to oc-mirror-workspace/operators.1697865931/manifests-ibm-cp-automation-catalog/mapping.txt 57 related images processed in 6.853202235s Writing image mapping to oc-mirror-workspace/operators.1697865931/manifests-ibm-fncm-catalog/mapping.txt -- -- -- info: Mirroring completed in 1h11m19.09s (4.185MB/s) Rendering catalog image "docker-na-public.artifactory.swg-devops.com/ecm-containerization-docker-local/cp4ba510-1025/root/ibm-cp-automation-catalog:9c5d8c" with file-based catalog Rendering catalog image "docker-na-public.artifactory.swg-devops.com/ecm-containerization-docker-local/cp4ba510-1025/root/ibm-fncm-catalog:c54f63" with file-based catalog Writing image mapping to oc-mirror-workspace/results-1698276172/mapping.txt Writing CatalogSource manifests to oc-mirror-workspace/results-1698276172 Writing ICSP manifests to oc-mirror-workspace/results-1698276172
- Update the global image pull secret for your Red Hat OpenShift cluster to have authentication credentials in place to pull images from your $TARGET_REGISTRY as specified in the image-content-source-policy.yaml file. For more information, see Updating the global cluster pull secret.
- Run the following command to create
ImageContentsourcePolicy
.oc apply -f $IBMPAK_HOME/.ibm-pak/data/mirror/$CASE_NAME/$CASE_VERSION/image-content-source-policy.yaml
- Verify that the
ImageContentsourcePolicy
resource is created.oc get imageContentSourcePolicy
- Verify your cluster node status.
oc get MachineConfigPool -w
After the
ImageContentsourcePolicy
and global image pull secret are applied, the configuration of your nodes is updated sequentially. Wait until all theMachineConfigPools
are updated before you move on to the next step. - Create a project for the CASE commands (
cp4ba
is an example) by running the following commands. Before you run the command in this step, you must be logged in to your OpenShift cluster.export NAMESPACE=cp4ba oc new-project $NAMESPACE
- Optional: If you use an insecure registry, you must add the target registry to the
cluster
insecureRegistries
list.oc patch image.config.openshift.io/cluster \ --type=merge -p '{"spec":{"registrySources":{"insecureRegistries":["'${TARGET_REGISTRY}'"]}}}'
What to do next
You can now go ahead and install the Cloud Pak operator. For more information, see Option 2: Installing the Cloud Pak catalog and an operator instance.