IBM Cloud Configuration Discovery allows you to reverse-engineer your perfect application hosting environment and codify it so you can easily re-build or replicate as you need.

Do you have an IBM Cloud infrastructure that was incrementally and manually built that is now set up exactly how you want it and perfectly running your business-critical applications? Up until now, you may have wanted to replicate the environment for a number of different reasons, but it would take a lot of steps to create from scratch using the industry automation Infrastructure as Code (IaC) methodology. Perhaps you wish you could “clone” this environment:

With the growing complexity and increased adoption of cloud infrastructure resources in a business-critical application infrastructure, Infrastructure as Code (IaC) has never been more important. The promise of IaC is simple: You can use code (and the associated development practices) to seamlessly manage your cloud service infrastructure and eliminate the error-prone manual interventions. 

If you already used IaC practices to set up your perfect existing environment, you’re all set — you’ve already reached the end goal. But if you didn’t, don’t you now want to take that next step to get all the IaC benefits for your future operations? You really want to “clone” your environment through a discovery process and save that for future use. Thus is born the IBM Cloud Configuration Discovery tool.

Infrastructure to Code

“Infrastructure to Code” is the mechanism to transform an existing cloud “infrastructure” into the declarative code that can be used to create it again. It is, of course, based on the industry-leading Terraform automation, and it is the opposite to the well-known declarative process known as Infrastructure as Code (IaC).

IaC code in Terraform consists of two main components:

• The “.tf” file that is written in HashiCorp Configuration Language (HCL)
• The “state file” that the Terraform creates based on the HCL definitions provided in the “.tf” files

Importing an existing cloud infrastructure into IaC means you must generate the HCL files and the state-file, and IBM Cloud Configuration Discovery will do that for you.

IBM Cloud Configuration Discovery tool

IBM Cloud uses IaC code with Terraform, so that is where we will create by migrating your existing infrastructure into Terraform code. Let’s walk through how to do that:

Terraformer is an open-source project built by Google with a CLI that is suitable for a Terraform developer.  With Terraformer CLI, a developer can easily build Terraform configuration files for pre-existing resources in a cloud environment. It performs the reverse of what Terraform is designed to do, and in that way, this tool can be thought of as “Infrastructure to Code.” Terraformer will “import” your current environment state to a Terraform configuration from a provider. When you “import” that current environment configuration into those files, you can then use that to create a new environment with those files and Terraform. You can refer to the Terraformer materials for more information about installation and usage.

Configuration Discovery capabilities

Although Terraformer is a powerful and robust tool for importing an existing cloud infrastructure, it has a few shortcomings that need to be improved, as well as specific IBM Cloud information that must be handled. Fortunately, the Configuration Discovery tool has a number of capabilities that deliver function above and beyond Terraformer:

• Adds a brown-field scenario, which is very often the real scenario needed in mature production environments.
• Upgrades the Terraform configuration to the latest Terraform versions.
• Supports integration of the Configuration Discovery tool into the VS Code extension for IBM Cloud Schematics.
• It can rewrite your resources into Terraform modules.

Configuration Discovery import scenarios

Let’s see about some possible scenarios you might have and how you can use Configuration Discovery to permanently save your infrastructure into a repeatable and usable code format in these different scenarios.

This tool supports two general import cases:

• Green-field scenario: Import the existing infrastructure on a clean state.
• Brown-field scenario: Import the existing infrastructure and merge the generated Terraform files into the presence of already existing Terraform files.

Green-field scenario

In software engineering, green-field refers to importing existing infrastructure on a clean or blank state.

Let’s assume that the following infrastructures were created through IBM Cloud console:

• Virtual Private Cloud resource
• IBM Cloud Object Storage resource
• IAM access group and policy resource 

Going forward, if you want to adopt the Terraform tool to manage your existing cloud infrastructure, there are many benefits. This is a perfect green-field scenario — you can import the existing infrastructure where it generates both Terraform state file and configuration files to a folder. Once it generates the Terraform code, you can manage all your existing cloud infrastructure lifecycle operations through Terraform.

In this case, the configuration discovery tool will import the state from the existing infrastructure, and next time a terraform apply command is run, the Terraform will consider the resources in its state. This means any changes made to configuration will be picked up as modifications, rather than as an addition of new infrastructure

Brown-field scenario

Brown-field refers to importing an existing infrastructure in the presence of existing Terraform code. 

Let’s describe a brown-field scenario:

• You have written configuration or tf file and provisioned a IBM Cloud Virtual Private Cloud using Terraform.
• Now, you have provisioned another set of Virtual Private Cloud and COS-related resources in the IBM Cloud using the console or CLI.
• Your cloud infrastructure has been changed and now you want to synchronize and manage all the resources through Terraform.
• Configuration Discovery will import all the Virtual Private Cloud and COS-related resources and merge the TF and state files into a folder:
  • It will attempt to merge with generated .tf/state file with existing .tf/state file.
  • Once the merge completes, you can manage all your existing infrastructure through Terraform.

Using these scenarios, you can handle any situation you need to handle with various types and layouts of cloud infrastructures.

Note: This feature is not yet completed in the tool, it is coming in the future.

Using with IBM Cloud Schematics

The output files from the Configuring Discovery tool can be used directly with Terraform CLI commands to create and manage resources in the IBM Cloud. But an easier method would be to use IBM Cloud Schematics, the automation tool that runs Terraform deployment operations, among a number of other automation capabilities. The files created from using the Configuration Discovery tool can be used in Schematics, just like any other Terraform files, to recreate an environment you’d like to use in IBM Cloud.

Next steps

So that’s the story of what IBM Cloud Configuration Discovery is, why it was created, what it can do and when and why you would want to use it. There is a full Configuration Discovery Tutorial that walks you through step-by-step a detailed usage scenario that will assist you with getting started.

IBM Cloud Configuration Discovery tool is being released to open-source project in Github as an initial Beta-level code. This tool is licensed under Apache 2.0 and built using the Go programming language. Users are allowed to enhance the tool and are free to ask for feature enhancements or report something that they think is not working properly.

We look forward to engaging with you in Github!