Question & Answer
Question
Is it safe to increase operating system ulimits?
Cause
IBM often asks clients to increase process limits (ulimits) on POSIX operating systems such as AIX, Linux, Solaris, and HP-UX. The default process limits often cause truncated core dumps (also known as system dumps). Core dump truncation means that diagnostics produced by crashes and OutOfMemoryErrors are usually unusable and the problems need to be reproduced with increased ulimits. This technote will answer these primary questions:
- Why are the default operating system limits so restrictive?
- Is it safe to increase operating system process limits?
- What are the best practices for setting default operating system limits?
- What are other performance considerations for unlimited ulimits?
- How do you confirm that ulimits are set correctly?
- Why use non-unlimited ulimits at all?
Answer
Why are the default operating system limits so restrictive?
Modern operating systems are based on the concept of multi-user, time-sharing systems. Operating systems use three key features to isolate users and processes from each other: user mode, virtual address spaces, and process/resource limits. Before these innovations, it was much easier for users and processes to affect each other, whether malicious or not.
User mode forces processes to use system calls provided by the kernel instead of directly interacting with memory, devices, etc. This feature is ultimately enforced by the processor itself. Kernel code runs in a trusted, unrestricted mode, allowing it to do certain things that a user-mode process cannot do. A user-mode process can make a system call into the kernel to request such functions and this allows the kernel to enforce constraints and share limited resources.
Virtual address spaces allow each process to have its own memory space instead of managing and sharing direct memory accesses. The processor and kernel act in concert to allocate physical memory and paging space and translate virtual addresses to physical addresses. Again, this provides the ability to restrict which memory a process can access and in what way.
Process and resource limits are a subset of the restrictions enforced by the kernel and virtual address spaces which can be configured on a per-process or per-user basis. The two key limits related to this discussion are the core file size limit and the maximum file size limit. (On AIX, the fullcore and pre430core system configuration settings are also important.) Process limits come in two flavors - soft and hard - but for the purposes of this discussion, both should be set. Note that IBM Java™ will increase its core file and maximum file size soft limits up to the allowed hard limits at runtime.
The core file ulimit and maximum file size ulimit are independent mechanisms to limit the maximum size of a core dump, since a core dump is just a regular file. These default values are restricted for many reasons, including:
- Disk space: Core dumps are very big because they dump most of the virtual address space. Even a simple program may have a virtual address space of a few hundred megabytes, and the average Java program has a virtual address space of a few gigabytes. By default, a core dump will be created in the current working directory of the process. If core dumps are not managed properly, then they can consume available disk space which may cause other problems.
- Security: Core dumps dump out most of the memory contents for that virtual address space for an instant in time. This may contain sensitive user information.
- Historical inertia: Some of these defaults date back three or four decades.
Is it safe to increase operating system process limits?
If you increase core and maximum file size ulimits and you have insufficient disk space, then your application may be affected (and the core dump itself may be truncated). For example, the default working directory of a WAS process is the profile directory and a core file will be written to the default working directory. This directory has many artifacts such as configuration, transaction information, logs, and more, so if core files fill up this disk, then application updates and other functionality may fail. Therefore, IBM recommends that a dedicated filesystem is created for system dumps. This filesystem should be on a fast, local disk. Then you can either use -Xdump arguments to change the path of system dumps, or change the "Working directory" under Application Servers > SERVER > Process Definition.
You can change the default path of system dumps using the IBM Java -Xdump "defaults" option in a generic JVM argument -Xdump:system:defaults:file=/somepath/core.%Y%m%d.%H%M%S.%pid.%seq.dmp
The -Xdump parameter is only available on IBM Java, so it cannot be used on Solaris and HP
If you decide to change the working directory instead, be aware that this will also change where javacores and other artifacts go, so you should document this change for your administrators' awareness.
We recommend unlimited core file and maximum file size ulimits because we cannot know ahead of time how large the virtual address space will become. For example, if you have a native memory leak, then the virtual address space can become very large before a problem occurs, and determining the problem with a truncated core dump will be very difficult or impossible.
What are the best practices for setting default operating system limits?
You shouldn't wait until you have a problem. IBM recommends that proper ulimits are set ahead of time for any WAS installation. In general, the easiest way is to update the relevant global ulimit configuration file. Then you will need to restart all WAS processes (and if you were logged in to a shell from which you restart such processes, you will need to log out and log back in).
For example, on Linux, if Java is run under the wasadm user name, you can add these lines to /etc/security/limits.conf:
wasadm soft core unlimited
wasadm hard core unlimited
wasadm soft fsize unlimited
wasadm hard fsize unlimited
Alternatively, you can set these ulimits in the shell that spawns the process before the Java command:
ulimit -c unlimited
ulimit -f unlimited
If you are starting servers from the Administrative Console or using wsadmin, then make sure that you set these ulimits in the node agent (startNode.sh) and restart it so that the server will inherit the node agent's ulimits. If you are manually starting servers using startServer.sh, then you must update startServer.sh itself or ensure ulimits are set in the shell where you launch startServer.sh.
Other Performance considerations for unlimited ulimits
While writing a core dump, the operating system will completely pause the process. After the core dump is finished, the process will resume where it left off. The time it takes to write the core dump is mostly proportional to the virtual address space size, available file cache in physical memory, and disk speed.
You can determine the virtual address space size in various ways (where VSZ is in KB):
AIX: ps -o pid,vsz -L PID
Linux: ps -o pid,vsz -p PID
Solaris: ps -o pid,vsz -p PID
HP-UX: UNIX95="" ps -o pid,vsz -p PID
The file cache is an area of physical memory (RAM) that is used as a write-behind cache for some virtual file system operations. If a file is created, written to, or read from, the operating system may try to perform some or all of these operations through physical memory and then flush any changes asynchronously. This dramatically improves performance of file I/O at the risk of losing file updates if a machine crashes before the data is flushed to disk. The best way to try to improve performance of writing system dumps is to ensure that physical memory and file cache have at least the amount of the virtual size of the process available. This means that the operating system will write the core to RAM, continue the process, and then asynchronously write it to disk. The filecache is very operating system specific and cannot be dedicated to particular file operations, so even if you increase your memory and file cache, other file I/O operations may fill it up (for example, if you are tracing heavily at the same time).
Finally, disk speed is a major factor in core dump writing performance for obvious reasons and you can increase performance by dedicating a faster disk for core dump processing.
How do you confirm that ulimits are set correctly?
The following IBM Java versions introduced a ulimit section in a javacore: Java 5 SR11 (WAS 6.1.0.29), Java 6 SR6 (WAS 7.0.0.7), Java 626 (WAS 8). You can take a javacore using "kill -3 PID" or other ways and search for this section:
1CIUSERLIMITS User Limits (in bytes except for NOFILE and NPROC)
NULL --------------------------------------------------------------------
NULL type soft limit hard limit
2CIUSERLIMIT RLIMIT_CORE unlimited unlimited
2CIUSERLIMIT RLIMIT_FSIZE unlimited unlimited
On some versions of Linux, you can simply cat /proc/PID/limits
On Solaris, you can use the "plimit" command to print the current ulimits for a running process. On both Solaris and HP-UX, the hs_err_pid crash file will contain ulimits at the time of the crash.
Why use non-unlimited ulimits at all?
There is a valid philosophical question of whether to set every ulimit to unlimited for particular users such as those running WAS (note that some values do not accept "unlimited" such as maximum open files on Linux but instead take a specific maximum value; see links). Changing ulimit values is not really a "tuning" exercise since these values are simply restrictions on what a process can do. If a machine is dedicated for particular purposes (WAS, IHS, etc.) then it makes sense to run these processes unrestricted and only restrict other processes. It is true that some of these resources directly or indirectly use kernel memory which is a shared, limited resource; however, a well-behaved kernel should kill any process that exhausts its resources and the resulting core dump should have obvious symptoms of the offending resource usage or leak. Until that point, it's not clear why potentially legitimate resource usage is constricted by these arbitrary default ulimits (or equally arbitrary ulimits you may find in other tuning documents).
Summary
In summary, IBM recommends the following actions for all WAS installations:
- Use the operating system global ulimit configuration file (or otherwise, the ulimit commands) to set unlimited soft and hard ulimits for core size and maximum file size for the users running WAS, along with operating system specific settings such as fullcore and pre430core on AIX, and
- Optionally, create a dedicated, fast, local disk for system dumps and configure WAS to send system dumps to this disk.
Related Information
Was this topic helpful?
Document Information
Modified date:
15 June 2018
UID
swg21638553