Control groups for Linux

Control groups (cgroups), allow you to allocate resources such as CPU time, system memory, network bandwidth, or combinations of these resources for a service, so that an activity per service instance can be ran and constrained by cgroups on the system.

In the application template and in the service profile, you can specify resources, such as CPU and memory, to be limited for the activity per service instance. The cgroup is created per service on the PEM host when the service instance is started.

The following structure represents cgroups:

/cgroup      
            |_memory (also separate for cpu, diskIO, etc.)
                |_IBM_PLATFORM_EGO
                    |_<cluster name>  
                      |_<consumer name>
                        |_<allocation_id>
                           |_<activity_id>
                             |_memory.limit_in_bytes 
                              |_tasks (a file containing activity process and it's childrens’ ID)

The service instance has an environment variable to indicate the cgroup path, which is called EGOSC_INSTANCE_CGROUP_PATH.

In configure cgroups in the application template, you use the IBM::ASC::Cgroups type for the service_container resource and add the following properties:

core_limit (CPULimit in the service profile): Sets the maximum number of cores that the activity can use.
disk:io_limit: (DiskIOLimit in the service profile): Sets the operation (read or write), device, and content (input/output (I/O) limit) in bytes per second for the activity.
mem: mem_limit_mb: (MemoryLimit in the service profile): Sets the maximum memory "RAM" represented in megabytes (MB) that the activity can use. When the service instance reaches the memory limit, the service instance ends with an out-of-memory error message.
mem: mem_swap_limit_mb: (MemorySwapLimit in the service profile): Sets the maximum memory swap (RAM+swap) represented in MB that the activity can use. When the service instance reaches the memory swap limit, the service instance ends with an out-of-memory error message.
Network: classid: (NetworkClassID in the service profile): Optional network class ID parameter that represents the net_cls.classid.

Note: SUSE and Ubuntu do not enable swapaccount by default. To use the memorySwapLimit parameter, follow SUSE's or Ubuntu's documentation to enable the swapaccount parameter first. Ubuntu does not mount net_cls by default, so you can follow Ubuntu's documentation to ensure that the cls_cgroup module is loaded before you mount net_cls.

For example:

serviceB_container
             type: IBM::ASC::Cgroups
              properties:
                  autoset: false
                  cpu: 
                    core_limit: 5
                  mem:
                    mem_limit_mb: 1024
                    mem_swap_limit_mb: 2048
                  disk:
                    - device: "8:0"
                    io_limit: 100
                    operation: read
                    - device: "9.0"
                    io_limit: 500
                    operation: write
                  network:
                    classid: "0x10001"

In the service profile, you can either have <ego:Container type="cgroups"/> to indicate that container resources are gathered from the multi-dimensional scheduling plan; or you can add the following optional resources under ego:Containertype:For example:

<sc:ActivityDescription>
<ego:ActivitySpecification>
<ego:Container="cgroup"> 
     <ego:cpuLimit>3</ego:cpuLimit>
     <ego:memoryLimit">200</ego:memoryLimit>
     <ego:memorySwapLimit">500</ego:memorySwapLimit>
     <ego:diskIOLimit operation="read or write" device="disk device id, e.g, 8:0">
         500</ego:diskIOLimit>
     <ego:NetworkClassID> Cgroup classid for net_cls.classid, e.g, 0x100001</ego:NetworkClassID>
</ego:Container>
<ego:JobMonitorMaxUpdateInterval>60</ego:JobMonitorUpdateInterval>
</ego:ActivitySpecification>
</sc:ActivityDescription>

When both multi-dimensional scheduling and ego:Container are enabled in the service definition, you can still add resource parameters. You can add the same resource parameters to ego:Container even if they are already covered in the multi-dimensional scheduling plan. If the same resource parameter is specified in both the ego:Container and multi-dimensional scheduling plan, the one with the more strict limitation is applied.

Note: If you only configure <ego:Container type="cgroups"/>, you must specify a resource group in the service profile in order for the multi-dimensional scheduling plan to work.