Many enterprises today have multiple clouds, creating new challenges in operating a business and managing even more complex hybrid and multicloud environments. 

Multicloud and hybrid cloud bring a need for a set of management tools and processes to ensure team productivity, efficiency in DevOps, and application portability. Containers that provide lightweight packaging that includes the software and all its dependency are getting widespread adoption. Additionally, containers are easily portable across on-prem and public cloud environments. As the adoption of containers and the Kubernetes platform is growing, so is the need for standardization and automation at scale for defining, installing, and upgrading Kubernetes applications.  

Using Helm to manage charts at scale

At IBM Multicloud Manager (MCM), we have hundreds of workloads across IBM and third parties that we needed to make available in the catalog—some being very complex workloads. We adopted Helm Charts for packaging workloads as you can leverage Helm to install and upgrade even the most complex Kubernetes application. 

The challenge we faced was adding all these charts to the Multicloud Manager Catalog while keeping quality and consistent user experience at the forefront. The Helm community publishes Helm best practices around charts but these were not sufficient to achieve the consistent experience and quality we needed to establish when handling a large number of charts. Therefore, as we progressed in our journey, we developed standards and conventions to enable the desired user experience and manage quality when dealing with a large number of charts. 

In this article, we will discuss some of the key conventions and standards that we developed toward the following: 

• Organization of the charts 
• Install/upgrade experience of the charts 
• Operational experience
• Integration into the CI/CD pipeline

Organization of the charts 

We recognized the importance of chart naming conventions when managing charts at scale. Regardless of the size of the organization, we highly recommend that you decide on chart naming conventions to address consistency across a large number of charts. For example, the chart name should always start with the organization name, and it should follow with an extension—such as dev, prod, staging, etc.—based on topology.  

We also recognized the importance of categorizing charts in several different categories to make it easy for end users to search. We researched the well-recognized keywords/tags that all the chart authors publishing toward the IBM Multicloud Manager Catalog adhered to. Chart authors defined keyword attributes in charts.yaml. 

The MCM Catalog Interface provided categorization and filtering based on these well-recognized keywords for ease of filter and search. 

Some examples of categorization include the following: 

Organization by Helm repositories

Different organizations can host their Helm repositories and publish them in the catalog. In the Multicloud Manager Catalog, customers can assign repositories to different teams and leverage role-based access control capabilities. 

Organization by supported architecture

Charts may support different architectures, such as x86, IBM Power, etc. Chart developers can use keywords from charts.yaml to specify supported architectures. 

Organization by maturity

Chart authors can attach a maturity level of the workload packaged by the chart:

• Beta: Early release announcement. Keyword used in charts.yaml for filtering is “Beta.”
• Tech Preview: Prior to the announcement. The extension used in naming the chart can be “-dev.” Keyword used in charts.yaml for filtering is “Tech.”
• Limited Use: The license used by the product is limited in use. Keyword used in charts.yaml for filtering is “Limited.”
• Commercial Use: Ready for production use. The extension used in naming the chart can be “-prod.” Keyword used in charts.yaml for filtering is “Commercial.”
• Open Source: The product installed by the chart is an open source product. It is intended/supported for usage in development and/or production. Keyword used in charts.yaml for filtering is “OpenSource.”

Enhancements to improve the install/upgrade experience of charts 

Every workload packaged as Helm has some of the unique input parameters for successful deployment. Additionally, end users are always interested in evaluating known issues, key fixes, or anything new with each new version of charts before deploying new or upgrading from an existing install. 

To address user experience for chart deployment and upgrade, we developed two specifications:

Values-metadata specification

The values-metadata specification is used to define metadata for values in values.yaml. Chart authors can optionally package an additional values-metadata.yaml file in the chart.tgz file, which is used by the IBM Multicloud Manager Catalog to enhance the presentation of the parameters during deployment. 

For example, instead of a generic text box for a string parameter, the metadata file could indicate that the parameter is limited to a set of strings. This set of strings can be presented as a drop-down selection list to the user.

For more info on the values-metadata specification, please refer to the IBM Multicloud Manager Knowledge Center values-metadata.yaml file.

Release notes specification

We developed the release notes specification to include release notes, known issues, fixes, prereqs, etc. Chart developers can optionally package the RELEASENOTES.md file with charts.tgz using the following specifications:

The IBM Multicloud Manager Catalog enables end-users to evaluate release notes before deploying or upgrading a chart.

Integration into platform operational services 

Operational services in Multicloud Manager provide extension points for any workloads to integrate with the operational services (e.g., monitoring, logging, metering).

For example, Multicloud Manager leverages Prometheus and Grafana for monitoring solutions. Chart authors can use chart annotations to describe monitoring endpoints, and the platform monitoring service automatically collects the metrics. See below for the annotation: 

Similar annotations are defined for other services, including logging and metering. 

Chart linting and test automation integrated into the CI/CD pipeline

We have standardized the process of creating and publishing Helm charts by adding a CV (continuous verification) linter on top of the basic Helm linter and by defining a CI (continuous integration) pipeline to continuously develop, lint, test, verify, and publish Helm charts.

The Helm chart continuous integration and development process that we adopted is summarized below:

1. The product team develops product-specific Helm chart and pushes to business unit-specific GitHub repo.
2. The build team builds, lints, packages, and tests Helm charts developed by product team and pushes it to business unit-specific Helm repo. A CV linter—a content verification linter developed by the IBM Content team—is incorporated in the build pipeline to lint the charts. The CV linter is used in addition to the standard Helm linter to enforce rules for naming, keywords, Readme format, liveness and readiness probe, pod security policy, Helm and CV tests, etc. Charts that do not meet CV lint standards are not promoted through the pipeline. Sample CV linter output is provided below:

3. The product team verifies and tests the chart, making updates to the Helm chart if necessary, and tags the final Helm chart for release.

4. The content verification team further verifies the final version of the business unit product chart, approves it, and pushes it to the IBM consolidated GitHub repository.

5. The verified and certified contents are then be published to IBM Multicloud Manager Helm repositories, such as IBM Public Cloud Helm Repo, IBM Private Cloud Helm Repo, etc.

Learn more at Helm Summit 2019

We hope you have found this topic interesting and have checked off the best practices of creating, organizing, and maintaining Helm charts in your organization.

If you’d like to learn more, check out the talk that we—Shikha Srivastava (@shikhasthoughts) and Kirti Apte (@kir_tweet)—are giving at Helm Summit 2019 about “More charts, More problems—Let’s Bring Some Sanity.”