Previewing install command outputs and files to be mirrored

The air-gapped install scripts have been updated so that you can preview the output of the install commands and list of files to be mirrored, to a file, before the commands are run. These supported commands do not require cluster access.

In order to see the list of commands and images that you will run and mirror prior to completing the installation steps, you can specify the --dryRunOutput <location> and --dryRun arguments to the actions that support these arguments. Once the action is run with the dry run arguments, the action writes the output to a location specified by --dryRunOutput. If no location is specified, then the output is sent to the stdout or the console.

Notes:

The CASE save step saves the files to a cache. All subsequent steps after a CASE save uses this cache. However, if a CASE save is run again on a newer version of the CASE, then the —mirror-images action will generate a list of only the new images that need to be mirrored.
The --dryRun arguments for actions like —configure-cluster-airgap can be run without requiring cluster access. The output of the action is saved to a file.
If you're using Skopeo to work with your images (USE_SKOPEO=true is set), the --dry-run parameter yields no results.

The following diagram details the steps run for an air-gapped installation:

Air-gapped installation steps

Use cases

Use case 1

As a container image administrator, I want to identify all the images that would be moved into my internal container registry before allowing an install of a product.

Run cloudctl case save. This command saves the product to its own offline directory
Run cloudctl case launch mirror-images with --dryRun and --dryRunOutput options. This command generates a list of the actual mirror commands that would run, in addition to a full recursive list of all the images that would be mirrored. The output is the full list of images and can be run through an image scanner. If the image list contains all the images that you will mirror, you can proceed with mirroring the images.
Set up container registry credentials using cloudctl case launch configure-creds-airgap. This action also supports --dryRun and --dryRunOutput options.
Rerun the cloudctl case launch mirror-images command without the --dryRun options to actually mirror those images. Once these images have been successfully mirrored, a “cache” of images exists. This offline directory can now be used for follow-on use cases for determining the “delta” set of images that a product upgrade may introduce to the cluster.

Use case 2

As a container image administrator, I want to identify the list of delta images that would be moved into my internal container registry before allowing an upgrade of this product.

Run cloudctl case save. Save this product to the offline directory.
Run cloudctl case launch mirror-images again with --dryRun and --dryRunOutput options. This command will generate a list of only the new, delta images that this new version of the product introduces.

Notes:

The actual mirroring of images does not require any access to an OCP cluster.
If you place the offline folder under version control, you now have a pinned version of this product, including all of its dependencies.
The offline folder also contains the list of OLM catalog images that would need to be installed on a cluster.
If the images are actually mirrored to a local registry, an OCP cluster admin can now proceed to configure one or more clusters with the same content (a repeatable cache)

Use case 3

As an OCP Cluster administrator, I want to configure an air-gapped OCP cluster for this new product to understand its potential impacts to a cluster.

Run cloudctl case launch configure-cluster with --dryRun and --dryRunOutput options. This command with the --dryRun options does not require any access to an OCP cluster. The command shows the steps it would take to retrieve and patch an OCP cluster’s global image pull secret. The command also shows the imagecontentsourcepolicy.yaml file that would be required based on where the original container images are sourced from.

Sample outputs

The following section outlines sample outputs of aforementioned commands:

cloudctl case save

JohnDoe@ibm.com@JohnDoe-MBP ~ %
cloudctl case save \
  --repo $CASE_REPO_PATH \
  --case $CASE_NAME \
  --version $CASE_VERSION \
  --outputdir $OFFLINEDIR

configure-creds-airgap

cloudctl case launch \
  --case $HOME/offline/$CASE_REPO_PATH \
  --inventory $CASE_INVENTORY_SETUP \
  --action configure-creds-airgap \
  --args "--dryRunOutput $HOME/offline --registry $LOCAL_DOCKER_REGISTRY --user cp --pass xxxx --dryRun" --tolerance 1

[INFO] [DRY RUN] Saving commands that would be executed with dry run disabled to /Users/JohnDoe@ibm.com/offline/1-configure-creds-airgap/create
[✓] CASE launch script completed successfully

This saves the creds to a file in the $OFFLINEDIR folder

JohnDoe@ibm.com@JohnDoe-MBP offline % tree
.
├── 1-configure-creds-airgap
│   └── create
│       └── commands.txt


JohnDoe@ibm.com@JohnDoe-MBP ~ % cat  /JohnDoe/offline/1-configure-creds-airgap/create/commands.txt

# All dry run commands will be printed, creating registry secret for cp.icr.io and saving to the following file: /JohnDoe/.airgap/secrets/cp.icr.io.json
JohnDoe@ibm.com@JohnDoe-MBP ~ % cat /User/.airgap/secrets/cp.icr.io.json
{"auths":{"cp.icr.io":{"email":"unused","auth":"xxxx"}}}%

# Note: cp.icr.io is just for illustration purposes.

mirror-images

JohnDoe@ibm.com@JohnDoe-MBP ~ % cloudctl case launch  --case $HOME/offline/$CASE_REPO_PATH --inventory $CASE_INVENTORY_SETUP --action mirror-images --tolerance 1  --args "--registry $LOCAL_DOCKER_REGISTRY --inputDir $OFFLINEDIR --dryRunOutput $HOME/offline --dryRun"

Welcome to the CASE launcher
Attempting to retrieve and extract the CASE from the specified location
[✓] CASE has been retrieved and extracted
Attempting to validate the CASE
Skipping CASE validation...
Attempting to locate the launch inventory item, script, and action in the specified CASE
[✓] Found the specified launch inventory item, action, and script for the CASE
Attempting to check the cluster and machine for required prerequisites for launching the item
Checking for required prereqs...

Prerequisite                                                                                      Result
Client oc CLI must meet the following regex: Client Version: 4.[3456789].d*                       true
Client skopeo CLI must meet the following regex: skopeo version (0.1.[4-9][0-9]|1.[0-9]+.[0-9]+)  true

Required prereqs result: OK
Checking user permissions...
No user rules specified.
[✓] Cluster and Client Prerequisites have been met for the CASE
Running the CASE ibmCommonServiceOperatorSetup launch script with the following action context: mirrorImages
skopeo version 1.4.1
Executing inventory item ibmCommonServiceOperatorSetup, action mirrorImages : launch.sh
-------------Mirroring images-------------
……
……
[INFO] Found 238 images
[INFO] [DRY RUN] Saving mirroring information to /Users/JohnDoe@ibm.com/offline/2-mirror-images/docker-local.artifactory.swg-devops.com:443
[INFO] [DRY RUN] Saving commands that would be executed with dry run disabled to /Users/JohnDoe@ibm.com/offline/2-mirror-images/docker-local.artifactory.swg-devops.com:443
[INFO] Deleting mirrored image csv files created during this mirror attempt
[✓] CASE launch script completed successfully
OK
JohnDoe@ibm.com@JohnDoe-MBP ~ %

This saves the list of images to mapping files:

JohnDoe@ibm.com@JohnDoe-MBP offline % tree
.
├── 1-configure-creds-airgap
│   └── create
│       └── commands.txt
├── 2-mirror-images
│   └── docker-local.artifactory.swg-devops.com:443
│       ├── commands.txt
│       ├── mapping_aa.txt
│       ├── mapping_ab.txt
│       └── mapping_ac.txt

configure-cluster-airgap

JohnDoe@ibm.com@JohnDoe-MBP ~ % cloudctl case launch  --case $HOME/offline/$CASE_REPO_PATH --inventory $CASE_INVENTORY_SETUP --action configure-cluster-airgap --namespace $NAMESPACE --args "--registry $LOCAL_DOCKER_REGISTRY --user $LOCAL_DOCKER_USER --pass $LOCAL_DOCKER_PASSWORD --inputDir $OFFLINEDIR --dryRunOutput $HOME/offline --dryRun " --tolerance 1

Welcome to the CASE launcher
Attempting to retrieve and extract the CASE from the specified location
[✓] CASE has been retrieved and extracted
Attempting to validate the CASE
Skipping CASE validation...
Attempting to locate the launch inventory item, script, and action in the specified CASE
[✓] Found the specified launch inventory item, action, and script for the CASE
Attempting to check the cluster and machine for required prerequisites for launching the item
Checking for required prereqs...
No requires section specified.
Required prereqs result: OK
Checking user permissions...
No user rules specified.
[✓] Cluster and Client Prerequisites have been met for the CASE
Running the CASE ibmCommonServiceOperatorSetup launch script with the following action context: configureClusterAirgap
oc: Client Version: 4.8.3
skopeo version 1.4.1
Executing inventory item ibmCommonServiceOperatorSetup, action configureClusterAirgap : launch.sh
-------------Configuring cluster for airgap-------------
-------------Configuring cluster pullsecret-------------
docker-local.artifactory.swg-devops.com:443
[INFO] [DRY RUN] Saving commands that would be executed with dry run disabled to /Users/JohnDoe@ibm.com/offline/3-configure-cluster-airgap/update-pull-secret
-------------Configuring imagecontentsourcepolicy-------------
name is ibm-cp-common-services
dir is /Users/JohnDoe@ibm.com/offline
[INFO] [DRY RUN] Saving commands that would be executed with dry run disabled to /Users/JohnDoe@ibm.com/offline/3-configure-cluster-airgap/apply-image-policy
[✓] CASE launch script completed successfully
OK

This saves the cluster configuration (pull secret) to a file that can be applied later, and also creates the image content source policy file.

JohnDoe@ibm.com@JohnDoe-MBP offline % tree
.
├── 1-configure-creds-airgap
│   └── create
│       └── commands.txt
├── 2-mirror-images
│   └── docker-local.artifactory.swg-devops.com:443
│       ├── commands.txt
│       ├── mapping_aa.txt
│       ├── mapping_ab.txt
│       └── mapping_ac.txt
├── 3-configure-cluster-airgap
│   ├── apply-image-policy
│   │   ├── commands.txt
│   │   └── icsp.yaml
│   └── update-pull-secret
│       └── commands.txt


JohnDoe@ibm.com@JohnDoe-MBP ~ % cat /Users/JohnDoe@ibm.com/offline/3-configure-cluster-airgap/apply-image-policy/commands.txt
# All dry run commands will be printed.
# Generate and apply image content source policy
oc apply -f /Users/JohnDoe@ibm.com/offline/3-configure-cluster-airgap/apply-image-policy/icsp.yaml


JohnDoe@ibm.com@JohnDoe-MBP ~ % cat /Users/JohnDoe@ibm.com/offline/3-configure-cluster-airgap/apply-image-policy/icsp.yaml
apiVersion: operator.openshift.io/v1alpha1
kind: ImageContentSourcePolicy
metadata:
  name: ibm-cp-common-services
spec:
  repositoryDigestMirrors:
  - mirrors:
    - docker-local.artifactory.swg-devops.com:443/cp
    source: cp.icr.io/cp
  - mirrors:
    - docker-local.artifactory.swg-devops.com:443/cpopen
    source: icr.io/cpopen
  - mirrors:
    - docker-local.artifactory.swg-devops.com:443/opencloudio
    source: quay.io/opencloudio
JohnDoe@ibm.com@JohnDoe-MBP ~ %

Installation methods

The following sections illustrate the required steps for each approach to an air-gapped installation. Pick the appropriate methodology and follow the steps within that section. Step ordering might vary depending on your methodology: