Setting up CloudWatch Memory Metrics on Linux Instances on AWS

Steps

Prerequisites:

• AWS SSM Agent Installed on EC2
  • Linux image already has the SSM Agent installed. Linux SSM Agent is now installed by default on Amazon Linux AMIs dated 2017.09 and later. You would only need to manually install SSM Agent on other versions of Linux, as described in this article .
• IAM Role with the following permissions: AmazonSSMFullAccess 

      Note: The least privilege permissions needed for the IAM role are AmazonEC2RoleforSSM (a custom policy to allow       the action "ssm:GetParameter" to the resource "arn:aws:ssm:*:*:parameter/*", see JSON  example)

Attach IAM Role

To configure each Linux EC2 instance through System Manager, you need to attach an IAM role with the above permissions to each EC2 instance.  

• Select the EC2 instance and click Actions>Instance Settings>Attach/Replace IAM role. Then, select the IAM role with the required permissions and click apply (see example below).
  

Now you should see the role attached to the EC2 instance (see Turbo_CW below)

Installing CloudWatch Agent on your Linux Instances

• Navigate to the 'System Manager' service under the account and region you'd like to configure  
• In the navigation pane, choose Run Command and then click Run a Command:
  
• In the Command document list, choose AWS-ConfigureAWSPackage
  
• In the Targets area, choose the instance or multiple instances on which to install the CloudWatch agent. If you do not see a specific instance, it might not be configured for Run Command.
• In the Action list, choose Install.
• In the Name field, type AmazonCloudWatchAgent
• Leave Version set to latest to install the latest version of the agent.
• Choose Run. In the next screen, you should see a confirmation message that the command was sent successfully and the status should be Success after few seconds
  

Create the Agent Configuration

The CloudWatch agent is configured to define which metrics are being sent to a CloudWatch Namespace, and other data required.  The following JSON represents the minimum requirements and cannot be changed.  By default, this configuration stores the metrics in the region in which the VM exists.

This will be used in the next step. 

{

      "agent": {

        "metrics_collection_interval": 60,

        "logfile": "/opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log"

      },

"metrics": {

    "metrics_collected": {

      "mem": {

        "measurement": [

          {"name": "used", "rename": "MemoryUsed"},

          {"name": "mem_available", "rename": "MemoryAvailable"}

        ]

        }

      },

    "append_dimensions": {

      "InstanceId": "${aws:InstanceId}"

    }

  }

}

Also, you can configure optional parameters of the CloudWatch namespace (CWAgent is the default when not defined), and region (default region where data is located in the region where the instance runs).  Note adding more metrics to be collected will not be reflected in Turbonomic – only Used Memory will be collected currently. Refer to AWS Documentation for more details.

Creating Parameter Store

Now you need to create a parameter store for the above configuration so it can be deployed across multiple EC2 instances 

• Navigate to the 'System Manager' service under the account and region you'd like to configure  
• In the navigation pane, choose Parameter Store and then click Create Parameter
  
• Copy and paste the JSON agent configuration you created in the above step into the Value field (leave all options at default)
  
  
• Under the Name field, you can enter any name, but you must use the same name in the next step, for example, we used CloudWatchLinux
• Now click Create Parameter -  you should see a message saying it has been created and the parameter listed 
  

Deploying The CloudWatch Configuration File

• In the 'System Manager' navigation pane, choose Run Command and then click Run a Command
• In the Command document list, choose AmazonCloudWatch-ManageAgent
• Under Action select configure 
• Under Mode leave it as ec2
• Change the Optional Configuration Source to ssm
• Under Optional Configuration Location, enter the same name of the parameter you created in the Parameter Store (previous section). In this example, the parameter is called CloudWatchLinux
• Optional Restart should be set to Yes (This restart the CloudWatch agent, not the instance)
  
• In the Targets area, choose the instance or multiple instances on which you want to deploy CloudWatch Configuration
  
• Now click Run - In the end, you should see something like this.
  

Installation complete

To confirm that the instance is sending the Memory Metrics, head to the CloudWatch page, click Metrics and locate the CWAgent namespace (Only be created if the process worked successfully), click InstanceId > there should be 2 Metrics per Linux instance, click 'MemeoryUsed' and confirm data is collected:

Note it can take up to 30 minutes in some cases for Memory metric to show up in the Turbonomic user interface.