File system buffer tuning

The following ioo and vmstat -v parameters can be useful in detecting I/O buffer bottlenecks and tuning disk I/O:

Counters of blocked I/Os due to a shortage of buffers

The vmstat -v command displays counters of blocked I/Os due to a shortage of buffers in various kernel components. Here is part of an example of the vmstat –v output:

...
              0 paging space I/Os blocked with no psbuf
              2740 filesystem I/Os blocked with no fsbuf
              0 external pager filesystem I/Os blocked with no fsbuf

...

The paging space I/Os blocked with no psbuf and the filesystem I/Os blocked with no fsbuf counters are incremented whenever a bufstruct is unavailable and the VMM puts a thread on the VMM wait list. The external pager filesystem I/Os blocked with no fsbuf counter is incremented whenever a bufstruct on an Enhanced JFS file system is unavailable

The numfsbufs parameter

If there are many simultaneous or large I/Os to a filesystem or if there are large sequential I/Os to a file system, it is possible that the I/Os might bottleneck at the file system level while waiting for bufstructs. You can increase the number of bufstructs per file system, known as numfsbufs, with the ioo command. The value takes effect only when a file system is mounted; so if you change the value, you must then unmount and mount the file system again. The default value for numfsbufs is currently 93 bufstructs per file system.

The j2_nBufferPerPagerDevice parameter

Note: When vmstat -v shows a shortage of file system buffstructs for enhanced JFS, the j2_dynamicBufferPreallocation tunable should be tuned first before making any change to j2_nBufferPerPagerDevice parameter.

In Enhanced JFS, the number of bufstructs is specified with the j2_nBufferPerPagerDevice parameter. The default number of bufstructs for an Enhanced JFS filesystem is currently 512. The number of bufstructs per Enhanced JFS filesystem (j2_nBufferPerPagerDevice) can be increased using the ioo command. The value takes effect only when a file system is mounted.

The lvm_bufcnt parameter

If an application is issuing very large raw I/Os rather than writing through the file system, the same type of bottleneck as for file systems could occur at the LVM layer. Very large I/Os combined with very fast I/O devices would be required to cause the bottleneck to be at the LVM layer. But if it does happen, a parameter called lvm_bufcnt can be increased by the ioo command to provide for a larger number of "uphysio" buffers. The value takes effect immediately. The current default value is 9 "uphysio" buffers. Because the LVM currently splits I/Os into 128 K each, and because the default value of lvm_bufcnt is 9, the 9*128 K can be written at one time. If your I/Os are larger than 9*128 K, increasing lvm_bufcnt might be advantageous.

The pd_npages parameter

The pd_npages parameter specifies the number of pages that should be deleted in one chunk from RAM when a file is deleted. Changing this value may only be beneficial to real-time applications that delete files. By reducing the value of the pd_npages parameter, a real-time application can get better response time because few number of pages will be deleted before a process/thread is dispatched. The default value is the largest possible file size divided by the page size (currently 4096); if the largest possible file size is 2 GB, then the value of the pd_npages parameter is 524288 by default.

The v_pinshm parameter

When you set the v_pinshm parameter to 1, it causes pages in shared memory segments to be pinned by VMM, if the application, which does the shmget(), specifies SHM_PIN as part of the flags. The default value is 0.

Applications can choose to have a tunable which specifies whether the application should use the SHM_PIN flag (for example, the lock_sga parameter in Oracle 8.1.5 and later). Avoid pinning too much memory, because in that case no page replacement can occur. Pinning is useful because it saves overhead in async I/O from these shared memory segments (the async I/O kernel extension is not required to pin the buffers).