Configuring process resource limit on proxy nodes

On IBM Cloud Private proxy nodes, a Nginx ingress controller is used to transmit external request to user workloads that are running in the IBM Cloud Private cluster. For the Nginx ingress controller to work properly, the rlimit for the maximum number of open files that is allowed on your proxy node must be set.

Nginx ingress controller calculates the maximum number of open files that is allowed on your proxy node by using the values of the configured number of worker processes and the sysctlFSFileMax setting.

The default value for the number of worker processes, used by Nginx, is equal to the number of CPU processors on the host. To get the number of CPU processors in your host, run the following command:

nproc

To change the value for the number of worker processes, modify the worker-processes parameter in the nginx-ingress section of the config.yaml file.

  # Nginx Ingress Controller chart configuration
  nginx-ingress:
    ingress:
      image:
        repository: "<cluster_CA_domain>:8500/ibmcom/nginx-ingress-controller"
      config:
        disable-access-log: 'true'
        keep-alive-requests: '10000'
        upstream-keepalive-connections: '64'
        worker-processes: "2"
      extraArgs:
        publish-status-address: "{{ proxy_external_address }}"
        enable-ssl-passthrough: true

    defaultBackend:
      image:
        repository: "<cluster_CA_domain>:8500/ibmcom/defaultbackend"

sysctlFSFileMax is the system setting that is stored in the fs.file-max. You can get the sysctlFSFileMax value by running the following command:
```
 sysctl -a | grep fs.file-max
```

The maximum number of open files for the worker process is calculated by using the following formula:

maxOpenFiles = (sysctlFSFileMax / worker-processes) - 1024

If the calculated value of maxOpenFiles is less than 1024, maxOpenFiles is set to 1024.

Linux systems have default values for the maximum number of open files. The following output is a summary of a few default Linux settings.

 # prlimit
 RESOURCE   DESCRIPTION                             SOFT      HARD UNITS
 AS         address space limit                unlimited unlimited bytes
 CORE       max core file size                         0 unlimited blocks
 CPU        CPU time                           unlimited unlimited seconds
 DATA       max data size                      unlimited unlimited bytes
 FSIZE      max file size                      unlimited unlimited blocks
 LOCKS      max number of file locks held      unlimited unlimited
 MEMLOCK    max locked-in-memory address space     65536     65536 bytes
 MSGQUEUE   max bytes in POSIX mqueues            819200    819200 bytes
 NICE       max nice prio allowed to raise             0         0
 NOFILE     max number of open files                1024     65535
 NPROC      max number of processes                31861     31861
 RSS        max resident set size              unlimited unlimited pages
 RTPRIO     max real-time priority                     0         0
 RTTIME     timeout for real-time tasks        unlimited unlimited microsecs
 SIGPENDING max number of pending signals          31861     31861
 STACK      max stack size                       8388608 unlimited bytes

From the default settings, you can see that the soft and hard limits for NOFILE are 1024 and 65535 respectively. The calculated value of maxOpenFiles must be in the range of the soft and hard limit.

If maxOpenFiles is larger than the hard limit, Nginx fails to set rlimit for the worker process. A Operation not permitted error is returned.
In some cases your system might require that maxOpenFiles is larger than the soft limit of 1024. To modify the soft limit, you can adjust the system configuration fs.file-max in /etc/sysctl.conf. Then, restart your host to ensure that the changes are picked up.