Helm CLI returns either connection errors or errors about incompatible Helm versions

After installing the Helm CLI, the Helm CLI commands fail with connection errors or errors about incompatible versions.

Symptoms

The Helm CLI commands fail with connection errors or errors about incompatible versions.

Causes

The Helm Tiller version that is installed is a different version than version 2.9.1+icp, which is installed with IBM Cloud Private.

This probably happened when you configured your command line client by using the kubectl commands and ran the helm init command with the --upgrade flag. When you run the helm init command without specifying the --client-only or the -c flag, the Helm CLI displays a warning message. The message reads that Tiller was already deployed on the cluster, and that the --upgrade flag is needed if you wanted to upgrade Tiller. If you continue with the helm init command using the --upgrade flag it overwrites the Tiller version.

You can verify that this is the problem by completing either of the following procedures:

Log in to the the IBM Cloud Private console with an ID that has the ClusterAdministrator role.
Navigate to Workloads > Deployments.
Locate the tiller-deploy deployment.
Select Edit to view its deployment information.

Locate the string that shows the path of the Tiller image. It should end with tiller:v2.9.1-icp. If it shows a different version, then the Tiller image has been changed.

You can also complete this procedure:

Run one of the following commands from the command line:
```
helm version --tls
```
or
```
helm version
```
If it is able to communicate with Tiller, the information about the client version of the Helm CLI and the server version of Tiller is displayed.
Ensure that the server version is 2.9.1+icp, and that the client version is 2.9.1.

If the versions are correct, the information that is returned is similar to the following example:
```
$ helm version --tls
Client: &version.Version{SemVer:"v2.9.1", GitCommit:"8478fb4fc723885b155c924d1c8c410b7a9444e6", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.9.1+icp", GitCommit:"27442e4cfd324d8f82f935fe0b7b492994d4c289", GitTreeState:"clean"}
```
If this command returns a different server version, the Tiller image has been changed.

Note: A Helm client that is newer than the required version is not compatible with the 2.9.1+icp version that runs on the cluster.

Resolving the problem

To resolve the problem, you must restore the Tiller image version in IBM Cloud Private 3.1.0 and IBM Cloud Private 3.1.0 Fix Pack. To restore the version to 2.9.1, complete the following steps:

Configure the client with a user that has ClusterAdministrator role access.
Run the following kubectl command to list the deployment history of the Tiller-Deploy deployment in the kube-system namespace:
```
kubectl rollout history deployment tiller-deploy -n kube-system
```
A list that contains two or more revisions of that deployment is displayed. Each revision is identified by a sequential revision number.
Run the following kubectl command to display the deployment information of each revision, beginning with the latest. For example:
```
kubectl rollout history deployment tiller-deploy -n kube-system --revision=4
```
Look through the deployment information that is returned for the reference to the Tiller image. The image name should be a string that ends with "tiller:v2.9.1-icp". The information for the highest revision number is the current state of the deployment with the incorrect Tiller image version. Display the deployment history for prior revisions until you find the first revision with the correct Tiller image. Note the revision number to use in step 4.
Use the revision number found in step 3 to undo the current deployment and revert to the correct revision by using the kubectl rollout undo command. You can use the following example as a guide:
```
kubectl rollout undo deployment tiller-deploy -n kube-system --to-revision=3
```
Verify that the deployment has been updated correctly by completing the following steps:
1. Navigate to Workload > Deployments, and find the Tiller-Deploy deployment.
2. View the deployment information by selecting Edit.
3. Find the Tiller image reference in the deployment information and ensure that it is the correct version.
Run the following command to log in to the cluster and configure the Helm client:
```
bx pr login --skip-ssl-validation
```
Initialize the Helm CLI again using the --client-only flag on the helm init command, as shown in the following examples:
```
helm init --client-only
```
or
```
helm init -c
```
Ensure that you are running the correct versions by running the following command:
```
helm version --tls
```

Remember that ICP 3.1.0 does not support Helm client versions newer than v2.9.1