IBM Support

Which cgroup subsystems will be used by LSF?

Question & Answer


Question

Which cgroup subsystems will be used by LSF?

Answer

By default, LSF will use 3 cgroup subsystems: freezer, cpuacct, memory.
If the job has CPU affinity resource requirements, LSF will also utilize cgroup cpuset subsystem to contain job processes.

The details are as below:
[freezer]: LSF uses the Linux cgroup freezer subsystem to track job process for job control.
Process tracking through cgroups can capture job processes that are not in the existing job's process tree and have process group IDs that are different from the existing ones, or job processes that run very quickly, before LSF has a chance to find them in the regular or on-demand process table scan issued by PIM.

[cpuacct]: LSF gets cpu accounting data from cpuacct subdirectory of this job directly. Sbatchd checks if cgroup cpuacct subsystem had been mounted, if failed, an error message will be logged and cgroup based cpu and memory accounting will all be disabled.

[memory]: LSF can impose strict host-level memory and swap limits on systems that support Linux cgroups. These limits cannot be exceeded. All LSF job processes are controlled by the Linux cgroup system. If job processes on a host use more memory than the defined limit, the job is immediately killed by the Linux cgroup memory subsystem. Memory is enforced on a per job and per host basis, not per task.

[cpuset]: When CPU affinity binding through Linux cgroups is enabled, LSF will create a cpuset to contain job processes if the job has affinity resource requirements, so that the job processes cannot escape from the allocated CPUs. Each affinity job cpuset includes only the CPU and memory nodes that LSF distributes.

[{"Product":{"code":"SSWRJV","label":"IBM Spectrum LSF"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Component":"--","Platform":[{"code":"PF016","label":"Linux"}],"Version":"10.1;10.1.0.1;10.1.0.2;10.1.0.3","Edition":"","Line of Business":{"code":"LOB10","label":"Data and AI"}}]

Document Information

Modified date:
17 June 2018

UID

isg3T1026338