An example of how the IBM Spectrum LSF offering on IBM Cloud can be used for hosting Computational Fluid Dynamics (CFD) application workloads.

The IBM Spectrum LSF offering on IBM Cloud makes it easy to deploy and configure high performance computing (HPC) clusters to host HPC workloads on the virtual private cloud (VPC) infrastructure. In this blog post, we provide a summary of the results from an evaluation of a set of benchmark runs using one of the workloads included in the OpenFOAM toolkit. For more detailed technical information about the evaluation, see the related white paper here.

The OpenFOAM CFD toolkit

OpenFOAM is a widely used open-source toolkit for computational fluid dynamics (CFD). In this post, we provide results from executing the simpleFoam motorbike tutorial, which is included in the toolkit, on IBM Cloud. This test case exercises key iterative solvers in OpenFOAM, including the geometric agglomerated algebraic multigrid (GAMG) solver. The motorbike tutorial also provides a mechanism for exploring OpenFOAM performance over a wide range of grid sizes. 

Environment characteristics

The Spectrum LSF offering on IBM Cloud was used to create all the necessary resources and configure the HPC cluster for evaluating the OpenFOAM workload. The basic elements of the cluster are illustrated in Figure 1. There is a jump host (login system), one or more LSF master nodes, one NFS server node for storage and a static number of LSF worker nodes. Once the base cluster was created, the OpenFOAM software and its dependencies were installed, configured and compiled on the LSF master system. 

We measured the execution time for the simpleFoam motorbike benchmark on two IBM Cloud configurations, each using 128 virtual CPUs on a different number of instances (i.e., worker nodes): (1) a single instance of type cx2-128×256 with shared-memory communication and (2) four instances of type cx2-32×64 with TCP/Ethernet networking between the four instances:

simpleFoam motorbike benchmark run results

The figure below shows the simpleFoam processing rate (higher is better) as a function of the size of the grid, where we define processing rate as the number of simpleFoam steps times the number of grid cells divided by the execution time:

For grids larger than ~10M cells, the two different virtual machine configurations provide almost identical performance. For these relatively large grids, communication performance is not very critical, and overall performance is dominated by computation, where bandwidth to memory is a major factor. For grids of ~5M cells or smaller, communication becomes more important, and the single instance cx2-128×256 provides better performance due to more efficient communication via shared memory.

Summary of the OpenFOAM workload evaluation

Good performance was achieved on the simpleFoam motorbike benchmark runs executed in the IBM Cloud environment deployed using the Spectrum LSF offering. Based on these results, we offer a few recommendations for configurations to use depending on the characteristics of the OpenFOAM problem to be solved. 

For OpenFOAM problems with grids of ~10M cells or less, we recommend using the single virtual server instance approach, with one instance of type cx2-128×256. OpenFOAM problems with larger grids may fit on one cx2-128×256 instance, but time to solution will be limited by the number of cores (64) or the number of sockets (4). IBM Cloud offers 64-core, four-socket instance types with far more memory — bx2-128×512 (128 virtual CPUs with 512 GB memory) and mx2-128×1024 (128 virtual CPUs with 1024 GB memory) — and these larger memory instance types can support much larger OpenFOAM problems. However, if time to solution is important, OpenFOAM problems with ~10M grid cells or more can be scaled out to engage more cores, using TCP/Ethernet connected virtual machines. With the current networking infrastructure on IBM Cloud, we recommend limiting scaling to >~ 2*10^5 grid cells per core, or >~ 3*10^6 grid cells per socket. This will ensure that OpenFOAM performance remains in the favorable compute-bound region of the scaling curve. 

Get started with IBM Spectrum LSF on IBM Cloud.