Configure your on-premises Jenkins server to deploy and update an application running on virtual servers in IBM Cloud Virtual Private Cloud (VPC).

The “Use a VPC/VPN gateway for secure and private on-premises access to cloud resources” tutorial covers in great detail how to connect on-premises resources to virtual private cloud (VPC) resources via a Virtual Private Network (VPN). 

In the process of walking through the tutorial, you are instructed to manually copy a small application to the virtual server instance running in the VPC. You then perform the application build steps on the virtual server before starting the application. But, what if you have an existing continuous integration and continuous delivery (CI/CD) tool—such as Jenkins—deployed on-premises? Instead of following the aforementioned manual steps, you can use it to perform not only the build and test steps, but you can also add a deploy step to copy the build artifacts to the virtual server instance running in the VPC. 

This post will cover the steps to follow to add a CI/CD tool to the mix.

Deployment scenario

Let’s recap the environment described in the original tutorial along with an update to include Jenkins. The diagram shows the virtual private cloud containing a virtual server that hosts a microservice interfacing with two distinct IBM Cloud services. A (simulated) on-premises network and the virtual cloud environment are connected via VPN gateways.

1. The infrastructure (VPC, Subnets, Security Groups with rules, Network ACL and VSIs) are set up using a provided shell script that calls to the IBM Cloud CLI. 
2. The microservice interfaces with IBM Cloud Object Storage and Databases for PostgreSQL through private endpoints.
3. A VPC/VPN Gateway is provisioned to expose the virtual private cloud environment to the on-premises network.
4. The strongSwan open source IPsec gateway software is used on-premises to establish a VPN connection with the cloud environment.
5. The VPC/VPN allows access to service endpoints from on-premises access service endpoints using VPN.
6. A Jenkins (or other CI/CD tool) is used to build, test, and deploy the application to the VPC virtual server instance by leveraging the on-premises strongSwan to the VPC VPN tunnel.

Deploying and configuring the infrastructure components

At minimum, you should perform the following steps from the tutorial to create the resources in the VPC:   

• Get the code
• Create Virtual Private Cloud baseline resources
• Create an on-premises virtual server
• Create the Virtual Private Network gateway and connection
• Create the on-premises Virtual Private Network gateway

If you plan on using our example application and the Jenkinsfiles provided below, you should also follow all the steps under “Create Services.” 

As mentioned in the “Create Virtual Private Cloud baseline resources” section, the script includes the creation of a bastion host. We will not be making use of the bastion host, so feel free to modify the script to prevent its creation or delete it after it is created. 

Configure routing from the Jenkins server to the VPC instance(s)

Confirm there is no connectivity from the Jenkins server to the virtual server in VPC by issuing a simple ping command to the virtual server running in the VPC.  

Next, add an IP route on the Jenkins server that sends all requests to the CIDR of the VPC through the server that is running the strongSwan gateway. In the tutorial, we used instances that run on Ubuntu 18.0.4. If your Jenkins instance also runs on the Ubuntu 18.0.4 (or higher) server, you can add the following IP route to the /etc/netplan/*.yaml file.  

• to is the CIDR for your VPC, (i.e. in the tutorial, it is called CLOUD_CIDR).
• via is the internal IP address of the instance running strongSwan. Please note, in the tutorial, the external IP is used to configure the VPN connectivity, however, your on-premises Jenkins server will likely connect to the strongSwan server also running on-premises over its internal IP. You can obtain the IP address by issuing an ifconfig command on the strongSwan server.

Once you have completed the edit and saved the file, run the following command to enforce the new route: netplan apply

If running on a different operating system, the steps to update the IP route will vary. For example, on CentOS/Red Hat, the file to update is /etc/sysconfig/network-scripts/route-eth0 and can also vary based on your network card assignments.

You can now confirm there is connectivity from the Jenkins server to the virtual server in VPC by issuing a ping command and seeing responses come in.  

Configure a Jenkins Pipeline

With the routing all set and the services created, you can now create a Jenkins Pipeline using the following minimalist Jenkinsfile:

pipeline {
    agent none 
    parameters {
      string(name: 'VSI_CLOUD_IP', defaultValue: '', description: 'IP address to the application server.')
    }
    stages {
        stage('App Build') {
            agent { docker 'node:12.13.0' } 
            steps {
                sh "npm install"
                sh "npm run build"
                sh "cp config/config.template.json config/config.json"
            }
        }
        stage('VPC Deploy') {
            agent { docker 'node:12.13.0' }
            steps {
                withCredentials([file(credentialsId: 'jenkins-cos-credentials', variable: 'cos_credentials'), \
                file(credentialsId: 'jenkins-pg-credentials', variable: 'pg_credentials'), \
                file(credentialsId: 'jenkins-pg-certificate', variable: 'pg_certificate'), \
                sshUserPrivateKey(credentialsId: 'jenkins-ssh-private-key', keyFileVariable: 'ssh_private_key')]) {
                    sh "cp $cos_credentials config/; cp $pg_credentials config/; cp $pg_certificate config/"
                    sh "mkdir -p ~/.ssh; cp $ssh_private_key ~/.ssh/id_rsa; chmod 400 ~/.ssh/id_rsa"
                    sh "ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null root@$VSI_CLOUD_IP \"cd /; rm -rf nodejs-graphql; mkdir -p nodejs-graphql\""
                    sh "scp -r -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null \"build\" root@$VSI_CLOUD_IP:/nodejs-graphql/build/"
                    sh "scp -r -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null \"config\" root@$VSI_CLOUD_IP:/nodejs-graphql/config/"
                    sh "scp -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null \"package.json\" root@$VSI_CLOUD_IP:/nodejs-graphql/package.json"
                    sh "ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null root@$VSI_CLOUD_IP \"apt-get update; apt-get install nodejs npm -y; npm install pm2@latest -g; cd /nodejs-graphql;  npm install; pm2 start build/index.js; pm2 startup systemd; pm2 save\""
                }
            }
        }
    }
}

To run this pipeline, you will need to create the following credentials in Jenkins based on the files generated during the “Create Services” step: 

• id: jenkins-cos-credentials. Import the cos_credentials.json file.
• id: jenkins-pg-credentials. Import the pg_credentials.json file.
• id: jenkins-pg-certificate. Import the certificate file for the Databases for PostgreSQL.

Add the SSH private key for the SSH public key:

• id: jenkins-ssh-private-key. Paste the content of your SSH private key.

Supply the values for the following parameter: 

• name: VSI_CLOUD_IP. IP address of the virtual server running in the VPC.

Wrapping up

Try out the tutorial and add the network routing configuration to your Jenkins server. Take note that in the associated repository, a Terraform template is provided as an alternative to the CLI-based shell script that is used to create the VPC resources. A Terraform template is particularly ideal to use as part of a CI/CD Pipeline to build/update the infrastructure.  

An enhancement you can make to the above Jenkinsfile is to change the Docker image used during the VPC Deploy stage to an image that includes Terraform and the IBM Cloud Provider for Terraform. You can use the included Terraform template to build/update the environment as part of the pipeline instead of creating them via the shell script. An excellent example of doing that can be found in another blog entry: “Application Deployment to a Virtual Private Cloud with a DevOps Toolchain.” 

Questions and feedback

The GitHub repository for this scenario—and all our other VPC-related scenarios—has an Issues tab where you can comment on the content and code. If you have suggestions or issues, please submit your feedback.