Setting up a mirror image registry

A local image registry must be used to store all images in your local environment. Create a registry and make sure that it meets the requirements. It must be available and accessible from the OCP cluster nodes.

About this task

Make sure that your image registry meets the following requirements:

Supports Docker Manifest V2, Schema 2.
Supports multi-architecture images.
Note: Do not use the OpenShift image registry as your local registry. The OpenShift registry does not support multi-architecture images.
Is accessible from both the bastion server and your OpenShift Container Platform cluster nodes.
Has the username and password of a user who can write to the target registry from the bastion host.
Has the username and password of a user who can read from the target registry that is on the OpenShift cluster nodes.
Allows path separators in the image name.

Important: If you want to set up multiple air gap environments for development, test, and production deployments, for example, you do not need to run the mirror-images command for each environment. On the bastion host, follow all 10 steps for your first (development) environment, but you can skip sub steps 10.i and 10.j for any subsequent environment.

Procedure

Create a cp namespace to store the images in the IBM Entitled Registry cp.icr.io/cp.

The cp namespace needs an entitlement key and credentials to pull the images. The namespace must have a user who can write and create repositories, and read all repositories.
Create a ibmcom namespace to store all images from the docker.io.

The ibmcom namespace is for all IBM images that do not require credentials to pull.
Create a cpopen namespace to store all images from the icr.io/cpopen repository.

The cpopen namespace is for publicly available images that are hosted by IBM that do not require credentials to pull.
Create a opencloudio namespace to store the images from quay.io/opencloudio.

The opencloudio namespace is for select IBM open source component images that are available on quay.io. The IBM Cloud® Platform Common Services images are hosted on opencloudio.
On the bastion host, create a directory that serves as the offline store.
The following example directory is used in the subsequent steps.
```
mkdir $HOME/offline
```
Note: The offline store must be persistent to avoid transferring data multiple times. The persistence also helps to run the mirroring process multiple times or on a schedule.
Create environment variables for the installer and image inventory.
On the bastion host, create the following environment variables with the installer image name and the image inventory to be able to connect to the internet and download the corresponding CASE file.
```
export CASE_ARCHIVE=ibm-cp-automation-3.1.0.tgz
export CASE_INVENTORY_SETUP=cp4aOperatorSetup
export OFFLINEDIR=${HOME}/offline
```
For 21.0.1-IF001 use the following commands.
```
export CASE_ARCHIVE=ibm-cp-automation-3.0.1.tgz
export CASE_INVENTORY_SETUP=cp4aOperatorSetup
export OFFLINEDIR=${HOME}/offline
```
Note: Releases with interim fixes are packaged in archives with a new minor version. The version numbers follow the release.major.minor standard. For example, the first interim fix for 21.0.2 is packaged in the archive ibm-cp-automation-3.1.1.tgz.
Download the Cloud Pak archive and image inventory, put them in the offline store, and unpack the case package.
```
cloudctl case save \
  --case https://github.com/IBM/cloud-pak/raw/master/repo/case/${CASE_ARCHIVE} \
  --outputdir ${OFFLINEDIR}
```
Unpack the case file:
```
cd ${OFFLINEDIR}
tar -xvzf ${CASE_ARCHIVE}
cd cert-kubernetes
```
Tip: If you are unable to download the case package from the bastion host, try downloading the archive to another workstation and then upload it to the bastion host. Use the following command to locate the archive and output it to the bastion ${HOME}/offline/ directory:
```
cloudctl case save \
  --case ${CASE_ARCHIVE} \
  --outputdir ${OFFLINEDIR}
```
The host is now configured and you are ready to mirror your images.

oc login <cluster host:port> --username=<cluster admin user> --password=<cluster admin password>

Create the namespace where you want to install the Cloud Pak operator.
Set the environment variable:
```
export NAMESPACE=<Cloud Pak operator namespace>
```
To create the namespace:
```
oc new-project ${NAMESPACE}
```
To go to the namespace:
```
oc project ${NAMESPACE}
```
Mirror the images and configure the cluster.
Complete these steps on your bastion host that is connected to both the image registry and the OpenShift Container Platform cluster. The following registries need authentication:
- cp.icr.io
- registry.redhat.io
- registry.access.redhat.com
Note: Do not use the tilde within double quotation marks in any command. For example, do not use args "--registry <registry> --user <registry userid> --pass {registry password} --inputDir ~/offline". The tilde does not expand and your commands might fail.
1. Store authentication credentials for the image registry.
  Run the following commands to set the environment variables for the IBM Entitled Registry.
```
export SOURCE_REGISTRY=cp.icr.io
export SOURCE_REGISTRY_USER=cp
export SOURCE_REGISTRY_PASS=<entitlement key>
```
2. Store and cache the registry credentials in a file on your file system.
  The following command stores and caches the registry credentials in a file on your file system in the $HOME/.airgap/secrets folder.
```
cloudctl case launch \
  --case ${OFFLINEDIR}/${CASE_ARCHIVE} \
  --inventory ${CASE_INVENTORY_SETUP} \
  --action configure-creds-airgap \
  --namespace ${NAMESPACE} \
  --args "--registry ${SOURCE_REGISTRY} --user ${SOURCE_REGISTRY_USER} --pass ${SOURCE_REGISTRY_PASS}"
```
  To check the registry credentials, run the following command:
```
cat $HOME/.airgap/secrets/${SOURCE_REGISTRY}.json
```
3. Create environment variables for the registry connection information.
```
export LOCAL_REGISTRY_HOST=<registry hostname>
export LOCAL_REGISTRY_PORT=':443'
export LOCAL_REGISTRY=$LOCAL_REGISTRY_HOST$LOCAL_REGISTRY_PORT
export LOCAL_REGISTRY_USER=<registry user>
export LOCAL_REGISTRY_PASS=<registry password>  
```
  Note: If your registry uses a non-standard port, specify the port by using the syntax <host>:<port>. For example, export LOCAL_REGISTRY=myregistry.local:5000. The username and password must be a user who can write to the local registry from the bastion host. For more information, see Setting up a mirror image registry.
4. Configure an authentication secret for the image registry.
  The following command stores and caches the registry credentials in a file on your file system in the $HOME/.airgap/secrets folder.
```
cloudctl case launch \
  --case ${OFFLINEDIR}/${CASE_ARCHIVE} \
  --inventory ${CASE_INVENTORY_SETUP} \
  --action configure-creds-airgap \
  --namespace ${NAMESPACE} \
  --args "--registry ${LOCAL_REGISTRY} --user ${LOCAL_REGISTRY_USER} --pass ${LOCAL_REGISTRY_PASS}" 
```
  To check the registry credentials, run the following command:
```
cat $HOME/.airgap/secrets/${LOCAL_REGISTRY}.json
```
5. Optional: Test whether you can create the ImageContentSourcePolicy resource by doing a dry run.
```
cloudctl case launch \
 --case ${OFFLINEDIR}/${CASE_ARCHIVE} \
 --inventory ${CASE_INVENTORY_SETUP} \
 --action configure-cluster-airgap \
 --namespace ${NAMESPACE} \
 --args "--registry ${LOCAL_REGISTRY} --user ${LOCAL_REGISTRY_USER} --pass ${LOCAL_REGISTRY_PASS} --inputDir ${OFFLINEDIR} --dryRun" 
```
  The ImageContentSourcePolicy provides the mapping from the public image registries to your private registry. For more information, see Configuring image registry repository mirroring.
6. Configure a global image pull secret and the ImageContentSourcePolicy resource.
  
  The following command restarts all of the OCP cluster nodes. Depending on the applications that are running on the cluster, the nodes might take some time to be ready.
```
cloudctl case launch \
 --case ${OFFLINEDIR}/${CASE_ARCHIVE} \
 --inventory ${CASE_INVENTORY_SETUP} \
 --action configure-cluster-airgap \
 --namespace ${NAMESPACE} \
 --args "--registry ${LOCAL_REGISTRY} --user ${LOCAL_REGISTRY_USER} --pass ${LOCAL_REGISTRY_PASS} --inputDir ${OFFLINEDIR}" 
```
  Note: You can run the following command to verify the status before you move on to the next step.
```
oc get nodes -w
```
7. Verify that the ImageContentSourcePolicy resource is created.
```
oc get imageContentSourcePolicy
```
8. Optional: If you are using an insecure registry, you must add the image registry to the cluster insecureRegistries list.
```
oc patch image.config.openshift.io/cluster --type=merge -p '{"spec":{"registrySources":{"insecureRegistries":["'${LOCAL_REGISTRY}'"]}}}'
```
9. Verify your cluster node status.
```
oc get nodes
```
  After the imageContentsourcePolicy and global image pull secret are applied, you might see the node status as Ready, Scheduling, or Disabled. Wait until all the nodes show a Ready status.
10. Optional: Test whether you can pull the images by doing a dry run.
```
cloudctl case launch --case ${OFFLINEDIR}/${CASE_ARCHIVE} \
  --inventory ${CASE_INVENTORY_SETUP}   \
  --action mirror-images \
  --namespace ${NAMESPACE} \
  --args "--registry ${LOCAL_REGISTRY} --user ${LOCAL_REGISTRY_USER} --pass ${LOCAL_REGISTRY_PASS} --inputDir  ${OFFLINEDIR} --dryRun"
```
  Note: If the mirror-images action appears to hang, you can enter CTRL+C and run the same command. The mirror-images action then starts from where it stopped.
11. Mirror the images to the image registry.
  The cloudctl case launch command is used to preserve manifest digests when an image is moved from one registry to another.
```
cloudctl case launch \
  --case ${OFFLINEDIR}/${CASE_ARCHIVE} \
  --inventory ${CASE_INVENTORY_SETUP} \
  --action mirror-images \
  --namespace ${NAMESPACE} \
  --args "--registry ${LOCAL_REGISTRY} --user ${LOCAL_REGISTRY_USER} --pass ${LOCAL_REGISTRY_PASS} --inputDir ${OFFLINEDIR}" 
```

What to do next

Important: Verify that each namespace meets the following requirements:

Supports auto-repository creation.
Has credentials of a user who can write and create repositories. The bastion host uses these credentials.
Has credentials of a user who can read all repositories. The OpenShift Container Platform cluster uses these credentials.