Improvements in performance in very large file systems

Read about how to improve the performance of the mmapplypolicy command in very large file systems.

The following actions can improve performance:
  • Put the system pool on the fastest storage available, which can be either solid-state storage or hard disks. In either case, spread the system pool over multiple storage devices, so that they can seek in parallel, independently of one another. Verify that logical disks do not map to the same physical disk.

    The policy engine requires many I/O operations against file system metadata. Storing metadata on the fastest storage possible can improve the performance of policy execution.

  • Include both the -N parameter and the -g parameter on the mmapplypolicy command line.
    • The -N parameter specifies a list of nodes that run parallel instances of policy code.
    • The -g parameter specifies a global work directory that can be accessed by the nodes that are specified by the -N parameter.
    • When both -N and -g are specified, mmapplypolicy uses high-performance and fault-tolerant protocols during execution.
  • If the exact order in which files are processed is not important, consider specifying the --choice-algorithm fast algorithm, which works with the -N and -g options for parallel processing.
  • If the order in which files are processed is not important at all, specify WEIGHT(0) in your MIGRATE, LIST, and DELETE policy rules.
  • Update the file system format to format level 13.01 (GPFS™ 3.5.0.1) or higher. File systems at this level can support the following two features, among others:
    • Storing small directories and small files in the inode.
    • Fast extended attributes. For this feature, you must also update the file system by running mmmigratefs.
    These two features can improve the performance of the mmapplypolicy command. See the following links: