Transformation Advisor Prerequisites
Prerequisites
IBM Operator Catalog
IMPORTANT NOTE ON CATALOGS: Transformation Advisor 2.3.x was delivered in the Red Hat Certified catalog and has now been removed from there. Transformation Advisor 2.4+ is only be available in the IBM Operator Catalog.
Upgrading from 2.3 to 2.4 is not automatic. See Upgrading for more on how to upgrade from 2.3 to 2.4. When you have the IBM Operator Catalog installed and you search for "Transformation Advisor" the OperatorHub
UI, you will see two operators appear - one from the IBM Operator Catalog, and one from the Red Hat Certified Catalog. Please select the Provider Type
checkbox called IBM Operator Catalog in the OperatorHub UI to
ensure that you are installing the latest version of Transformation Advisor from the IBM Operator Catalog.
Transformation Advisor is available in the IBM Operator Catalog. See IBM Operator Catalog documentation for more details.
When installing Transformation Advisor using the CASE actions, there is an option --installIbmCatalog
which will automatically install the IBM operator catalog if it is not already installed on your cluster.
When installing Transformation Advisor from the UI, you will need to follow the instructions IBM Operator Catalog documentation to install the catalog.
Running on a FIPS enabled environment
If you are installing Transformation Advisor on an environment where FIPS is enabled you will need to set the FIPS configuration when installing the Transformation Advisor instance. For a UI installation, there is a FIPS Mode Enable toggle on the create instance UI form. For a CASE installation, you need to pass the --fips true
argument to the install command.
Cluster Admin Permissions
To install Transformation Advisor, you need to perform some tasks as a user with cluster admin permissions.
Please refer to the Installing documentation for more details.
Access Transformation Advisor images from the entitled registry
Transformation Advisor can be installed with a time limited evaluation license (fully featured), or as an entitled product. Please read the license for more details:
-
If installing as an evaluation, you can ignore the following instructions that describe how to get access to the entitled registry. In this case, the required images will be pulled from a public registry and no access details are required.
-
If installing Transformation as an entitled product, you will need to have an entitled registry key.
Entitlement to Transformation Advisor is obtained by purchasing one of a number of other IBM products. To find your entitlement key you must first ensure that you have entitlement to any one of the IBM products listed here. Then log in to MyIBM Container Software Library with the IBMid and password that are associated with the entitled software. Your Entitlement Key will be displayed there.
Secret needs to be created in the same namespace where the product instance is installed or globally.
For a namespaced install create an image pull secret called ibm-entitlement-key
using your entitlement key as the password, cp
as the username, and cp.icr.io
as the docker server for use with this deployment.
See https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/#create-a-secret-by-providing-credentials-on-the-command-line for how to create a pull secret.
Alternatively you can update the cluster's global pull secret using your entitlement key as the password, cp
as the username, and cp.icr.io
as the docker server for use with this deployment. See https://docs.openshift.com/container-platform/4.9/openshift_images/managing_images/using-image-pull-secrets.html#images-update-global-pull-secret_using-image-pull-secrets for how to update the global pull secret.
Notice: Migration to IBM Cloud Container Registry
As of Transformation Advisor 2.4.3, some of the Transformation Advisor images (the operator, bundle and catalog) were migrated from Docker Hub (docker.io/ibmcom
) to the IBM Cloud Container Registry (icr.io/cpopen
).
This change affects all of your IBM Operator Catalog provided software, not just Transformation Advisor. Please refer to the following documentation for full details of this change: Migrating from Docker to the IBM Container Registry
For Transformation Advisor, the following are the changes that are required:
If you operate in an internet-connected Red Hat® OpenShift® Container Platform cluster:
- Update the IBM Operator Catalog to point to
icr.io/cpopen
, instead ofdocker.io/ibmcom
. The change is outlined in the document Migrating from Docker to the IBM Container Registry - If you are upgrading Transformation Advisor from a version LESS THAN 2.4.3, apply an
ImageContentSourcePolicy
. This change is outlined in the document Migrating from Docker to the IBM Container Registry
If you operate in an air-gapped Red Hat OpenShift Container Platform cluster:
- When mirroring images to the mirror or portable registry, you need to redirect from
docker.io/ibmcom
toicr.io/cpopen
. This is documented in the air-gap instructions here: Mirror images to portable registry.
Persistent Storage
NOTE: The following section is a guide to how you can configure persistence for Transformation Advisor. It is not a thorough guide to storage on a Kubernetes in general. Please refer to the Kubernetes documentation if you are not familiar with concepts mentioned in this section. Refer to Red Hat OpenShift documentation and your storage administrator to understand the persistence options available on your specific version of OpenShift.
Transformation Advisor requires persistence to be configured in order to persist the database data. When Transformation Advisor starts for the first time, it will create a PersistentVolumeClaim
(unless you have set the option to use
an existing claim). The persistence options provided by Transformation Advisor, allow you to control the attributes of the PersistentVolumeClaim
that Transformation Advisor creates. The PersistentVolumeClaim
that Transformation
Advisor creates, must bind to a PersistentVolume
on the cluster. The properties set for the PersistentVolumeClaim
must be compatible with the PersistentVolumes
available on your cluster.
From Transformation Advisor 3.0.x onwards, two PersistentVolumeClaims
are required.
The persistence may be configured from the UI form view or the YAML view when installing the Transformation Advisor instance.
It is NOT recommended to use local volumes as backing for persistence for Transformation Advisor.
Transformation Advisor Persistence Options
The following properties can be configured to control the persistence that Transformation Advisor uses. See Example Persistence Configurations to see how to use these configuration options.
Enable / Disable
Configuration setting: persistence.enabled: <true|false>
Although it is NOT recommended for a production install, you can also turn the persistence off. Transformation Advisor will have full functionality with persistence disabled except importantly, if persistence is off, you will lose all of your Transformation Advisor data when the database container restarts.
Access modes
The access mode determines the PersistentVolume
that the Transformation Advisor PersistentVolumeClaim
can bind to. The value for access mode must be supported by the type of storage that is available on your cluster. For
example, if you are installing Transformation Advisor on public cloud, the accessMode is ReadWriteOnce (RWO)
. Consult your storage administrator for more details.
Configuration setting: persistence.accessMode: <"ReadWriteOnce">
Transformation Advisor supports ReadWriteOnce (RWO)
access mode. ReadWriteMany (RWX)
and ReadOnlyMany (ROX)
access modes are not supported by Transformation Advisor.
Storage class
Configuration setting: persistence.storageClassName: <"storage class name">
A StorageClass
provides a way for administrators to describe the "classes" of storage they offer. If the PersistentVolumes
configured for your cluster have a storage class, then you can set the storageClassName
property in Transformation Advisor, so that PersistentVolumeClaim
that Transformation Advisor creates will bind to those PersistentVolumes
.
If you are configuring persistence in the UI form view, the Storage Class widget will be pre-populated with the StorageClasses available on your cluster. Alternatively, you can navigate to storage classes in the navigation in OpenShift UI to see what storage class are available on your cluster.
Alternatively, you can use the following command to see the storage classes available.
oc get storageclass
Storage type and permissions
If you are installing on an environment that uses block storage - (specified using a storageClass
), Transformation Advisor can bind to the storage and set the correct permissions for that storage. If you are installing on an environment
that uses file system storage such as NFS, you must ensure that Transformation Advisor has read/write permissions to the storage on the host. For details, see Setting permissions for NFS.
If there are not sufficient permissions for the storage that Transformation Advisor uses, the CouchDB pod fails to start and a permission denied message is added to the couch db pod logs:
[error] 2019-08-23T21:08:33.974674Z nonode@nohost <0.256.0>
-------- Could not open file ./data/_users.couch: permission denied
Setting permissions for NFS
Note: From Transformation Advisor 3.0.x onwards, two PersistentVolumeClaims
are required, and you need to repeat the following procedures for both PersistentVolmueClaims
accordingly. When shared storage
is mounted into a container, it is mounted with the same ownership and permissions that are found on the exported NFS directory. The container that uses the storage is not run with that owner and might not have permission to read or write to the
storage. To enable read/write permission to the storage, consider the following options:
Option A: Control permissions with the supplementalGroups setting
Configuration setting: persistence.supplementalGroups: <[id1,id2...]>
Supplemental groups are regular Linux groups and are typically used for controlling access to shared storage, such as NFS. When a process runs in Linux, it has a UID, a GID, and one or more supplemental groups. The supplemental groups can be set for a container’s main process. The supplemental group gives the container process group ownership on the storage and consequently the group level permissions to allow the container process to read/write to the storage.
Consider the following example where the underlying host storage is located at /netstore/transadv/
. The nfsnobody group might have full permissions on the shared directory when NFS is shared with the root_squash option:
> ls -ld /netstore/transadv/
drwxrwx---. 7 nfsnobody nfsnobody 182 Aug 31 22:30 /netstore/transadv/
To control permissions with the supplementalGroups
setting, and give the container the ability read/write to the storage, add the nfsnobody
GID to the supplemental groups as follows:
-
Get the nfsnobody GID using the
id
command.> id nfsnobody uid=65534(nfsnobody) gid=65534(nfsnobody) groups=65534(nfsnobody)
-
Add the GID (65534 in this case) using the
supplementalGroups
setting, e.g. configuration.persistence: ... <couchdb | neo4j> supplementalGroups: [65534]
Option B: Create a PersistentVolume specifically for use with Transformation Advisor
Configuration setting: persistence.existingClaim: <"claim name">
Creating a PersistentVolume
specifically for Transformation Advisor allows you to set the access permissions for storage usedby that PersistentVolume
.
This option describes how to:
- Create a NFS
PersistentVolume
specifically for Transformation Advisor - Create a
PersistentVolumeClaim
to bind to thatPersistentVolume
- Use that claim (with appropriate permissions) for Transformation Advisor
We will manually create a PersistentVolumeClaim
and ensure that the storage backing its PersistentVolume
has the correct (open) permissions. You can ensure a PersistentVolumeClaim
binds to a specific PersistentVolume
with selector and label attributes.
-
Define an NFS PersistentVolume. The following sample is an NFS
PersistentVolume
definition. Substitute the path and server values for your own environment.apiVersion: v1 kind: PersistentVolume metadata: name: tanfspv labels: pvc_for_app: "ta" spec: capacity: storage: 20Gi accessModes: - ReadWriteOnce nfs: path: /nfs/ta_storage server: xxx.xxx.xxx.xxx persistentVolumeReclaimPolicy: Recycle
Make sure that the directory at the path has open permissions so that the container can read/write there:
chmod -R 777 /netstore/transadvopen/
-
Define a PersistentVolumeClaim. The following sample is a PersistentVolumeClaim definition that binds to the
PersistentVolume
described in the previous sample. Ensure that you create thePersistentVolumeClaim
in the Transformation Advisor namespace.apiVersion: v1 kind: PersistentVolumeClaim metadata: name: tapvc spec: accessModes: - ReadWriteOnce resources: requests: storage: 20Gi selector: matchLabels: pvc_for_app: "ta"
If you want Transformation Advisor to use a specific PersistentVolumeClaim, add the name of the claim using the existingClaim
property.
Option C: Update permissions for the PersistenceVolume after Transformation Advisor has tried and failed to start
If you are using NFS storage, and you do not complete option A or B, when Transformation Advisor attempts to start - you will see that the Couch DB pod will not start. If you look at the logs for the Couch DB pod you will see a permissions issue:
[error] 2019-08-23T21:08:33.974674Z nonode@nohost <0.256.0>
-------- Could not open file ./data/_users.couch: permission denied
At this point, you can find the PersistentVolumeClaim
that Transformation Advisor has created:
oc get pvc
That will reveal the PersistentVolume
that it has bound to (under the VOLUME
header).
You can describe that PersistentVolume
to discover the path for the storage:
oc describe pv tanfspv | grep Path:
You can then open the permissions on that path chmod -R 777 <path>
The Couch DB should then restart sucessfully within a few minutes. Or, you can delete the Couch DB pod and it will be automatically restarted.
Storage Classes Red Hat OpenShift Container Platform on IBM Cloud
IBM Cloud provides a number of storage classes for OpenShift clusters. Some of the storage classes use block storage and can be identified by the "-block-" in their name. Other storage classes use file storage and can be identified by the "-file-" in their name. The storage class with the name "default" uses file storage also. Please note that the storage with the name "default" may be different to the storage class that is marked as the default storage class for the cluster.
Transformation Advisor supports the use of any of the block based storage classes. Transformation advisor does not support the use of the file based storage (including the storage class called "default"), except for the storage classes that have a "-gid" suffix, i.e.
- ibmc-file-bronze-gid
- ibmc-file-silver-gid
- ibmc-file-gold-gid
These storage classes allow you to specify a supplemental group when installing so that the Transformation Advisor non-root containers can write to the storage. When using these *-gid
storage classes, please specify the supplementalGroups
of 65531 at install time.
Please refer to the following document for more information on storage classes on Red Hat OpenShift on IBM Cloud: Storage class reference.
Example Persistence Configurations
Please refer to the Installing document on how to apply these example configurations.
EXAMPLE 1: Using storageClass
Use the storage class to create a PersistentVolumeClaim
that will bind to a PersistentVolumeClaim
of that class. In this case, we are using "ibmc-block-silver" on public cloud.
For a UI install, the custom resource YAML would look like this:
...
persistence:
enabled: true
<couchdb | neo4j>
accessMode: "ReadWriteOnce"
size: 20Gi
useDynamicProvisioning: true
existingClaim: ""
storageClassName: "ibmc-block-silver"
supplementalGroups: []
...
For a CASE install the arguments would look like this:
--args "... --storageClass ibmc-block-silver --accessMode ReadWriteOnce ..."
EXAMPLE 2: Using supplementalGroups
In the following example, we are using the supplementalGoups
setting to ensure the database pods have sufficient permissions on NFS storage we are using. The group ownership of the shared file system has a GID of 65534
For a UI install, the custom resource YAML would look like this:
...
persistence:
enabled: true
<couchdb | neo4j>
accessMode: "ReadWriteOnce"
size: 20Gi
useDynamicProvisioning: true
existingClaim: ""
storageClassName: ""
supplementalGroups: [65534]
...
For a CASE install the arguments would look like this:
--args "... --supplementalGroups [65534] ..."
EXAMPLE 3: Using existingClaim
In the following example, a persistence volume claim called my-ta-pvc
has been already set up by the user. The persistence volume claim has open permissions to allow the database container to write to it.
For a UI install, the custom resource YAML would look like this:
...
persistence:
enabled: true
<couchdb | neo4j>
accessMode: "ReadWriteOnce"
size: 20Gi
useDynamicProvisioning: true
existingClaim: "my-ta-pvc"
storageClassName: ""
supplementalGroups: []
...
For a CASE install the arguments would look like this:
--args "... --persistenceClaimCouchDB my-ta-pvc --persistenceClaimNeo4j my-ta-pvc2..."
Deleting a Persistence Volume Claim:
NOTE: Do not delete PersistentVolumes
or PersistentVolumeClaims
unless you are entirely satisfied that they are not being used by containers. Deleting PersistentVolumes
or PersistentVolumeClaims
may interfere with running applications and/or delete data that cannot be recovered.
In a normal uninstall of Transformation Advisor, the PersistentVolumeClaim
that Transformation Advisor has created will be deleted, and the PersistentVolume
that the claim binds to will be released for use.
If you need to manually delete PersistentVolumes
or PersistentVolumeClaims
please consult the relevant Kubernetes documentation
Image Registry Access
The images required for Transformation Advisor are hosted in the following locations:
- icr.io
- cp.icr.io
Your cluster will need access to these registries. If you operate block or allow lists for registries in your cluster, then you may need to update the image.config.openshift.io/cluster
resource accordingly to allow access to icr.io
and cp.icr.io
Accessing the Installed Transformation Advisor Product
Transformation Advisor leverages the OpenShift APIs to perform authentication, and as such, will authenticate against the configured identity provider on the cluster. Any user wishing to access Transformation Advisor, must have valid credentials. Anonymous access is not supported.
Resources Required
This section describes the resources required to install and run Transformation Advisor. Choose a configuration profile that aligns with your usage, and allocate resources to Transformation Advisor accordingly. Please note that Transformation Advisor does not support a High Availability configuration.
Configuration Profile Definitions
Configuration Profile Name | Usage |
---|---|
Starter | minimum size for functioning product, ie demo use case |
Production | recommended size for production environment |
Starter Profile Configuration
Subsystem | CPU | Memory (GB) | Disk Space (GB) |
---|---|---|---|
CouchDB | 0.5 | 1 | 20 |
GraphDB | 0.5 | 1 | 5 |
Server | 0.5 | 1 | N/A |
UI | 0.5 | 1 | N/A |
Operator | 0.4 | 0.5 | N/A |
Production Profile Configuration
Subsystem | CPU | Memory (GB) | Disk Space (GB) |
---|---|---|---|
CouchDB | 1.5 | 3 | 20 |
GraphDB | 1.5 | 3 | 5 |
Server | 1.5 | 3 | N/A |
UI | 1.5 | 3 | N/A |
Operator | 0.5 | 0.5 | N/A |
These values can be modified in the custom resource yaml prior to install. See Configuring for more details.