My IBM Log in

September 15, 2021 By Sophia Wen 2 min read

A look at the auto-scaling capability offered by IBM Spectrum LSF in the HPC cluster solution.

The HPC cluster blog post and tutorial show you how to create an HPC cluster on IBM Cloud. This post is to focus on the auto-scaling capability offered by IBM Spectrum LSF in the HPC cluster solution.

The parameters related to auto-scaling are worker_node_min_count and worker_node_max_count:

The HPC cluster blog post and tutorial show you how to create an HPC cluster on IBM Cloud. This post is to focus on the auto-scaling capability offered by IBM Spectrum LSF in the HPC cluster solution.   The parameters related to auto-scaling are worker_node_min_count and worker_node_max_count.

Your cluster will keep worker_node_min_count workers regardless of whether there are jobs running in the cluster. When jobs request computing resources more than worker_node_min_count workers, LSF auto-scaling will automatically add more workers — up to worker_node_max_count workers in the cluster — to satisfy jobs’ demands. These additional computing resources (i.e., total number of workers – worker_node_min_count ) will be removed from the cluster after the job demand has diminished and the resources have been idling for 10 minutes.

Updating auto-scaling parameters

After you create an HPC cluster, you can still adjust the auto-scaling parameters. This can be done by changing the LSF configuration files inside the cluster. This page shows you how to change worker_node_max_count. You can also adjust the default 10-minute idling time if you want to keep these additional computing resources created by auto-scaling for a longer time. We would suggest keeping 10 minutes as the minimum value. This timer can be adjusted following the steps here.

One of the nice features of LSF auto-scaling is to allow you to mix different types of workers in one cluster. The instructions to add multiple compute profiles are provided here. You can then select the best choice of computing resources depending on the performance characteristics of your workloads.

Conclusions

This blog post shows you how to configure IBM Spectrum LSF auto-scaling parameters with the HPC cluster solution. Try to create a HPC cluster and see how LSF auto-scaling manages computing resources dynamically. You can learn more about LSF auto-scaling here.

Was this article helpful?
YesNo

More from Cloud

Attention new clients: exciting financial incentives for VMware Cloud Foundation on IBM Cloud

4 min read - New client specials: Get up to 50% off when you commit to a 1- or 3-year term contract on new VCF-as-a-Service offerings, plus an additional value of up to USD 200K in credits through 30 June 2025 when you migrate your VMware workloads to IBM Cloud®.1 Low starting prices: On-demand VCF-as-a-Service deployments begin under USD 200 per month.2 The IBM Cloud benefit: See the potential for a 201%3 return on investment (ROI) over 3 years with reduced downtime, cost and…

24 IBM offerings winning TrustRadius 2024 Top Rated Awards

2 min read - TrustRadius is a buyer intelligence platform for business technology. Comprehensive product information, in-depth customer insights and peer conversations enable buyers to make confident decisions. “Earning a Top Rated Award means the vendor has excellent customer satisfaction and proven credibility. It’s based entirely on reviews and customer sentiment,” said Becky Susko, TrustRadius, Marketing Program Manager of Awards. Top Rated Awards have to be earned: Gain 10+ new reviews in the past 12 months Earn a trScore of 7.5 or higher from…

IBM Tech Now: April 8, 2024

< 1 min read - ​Welcome IBM Tech Now, our video web series featuring the latest and greatest news and announcements in the world of technology. Make sure you subscribe to our YouTube channel to be notified every time a new IBM Tech Now video is published. IBM Tech Now: Episode 96 On this episode, we're covering the following topics: IBM Cloud Logs A collaboration with IBM watsonx.ai and Anaconda IBM offerings in the G2 Spring Reports Stay plugged in You can check out the…

IBM Newsletters

Get our newsletters and topic updates that deliver the latest thought leadership and insights on emerging trends.
Subscribe now More newsletters
Overview Annual report Corporate social responsibility Diversity & inclusion Financing Investor Newsroom Security, privacy & trust Senior leadership Careers with IBM Website Blog Publications Automotive Banking Consumer Good Energy Government Healthcare Insurance Life Sciences Manufacturing Retail Telecommunications Travel Our strategic partners Find a partner Become a partner - Partner Plus Partner Plus log in IBM TechXChange Community LinkedIn X Instagram YouTube Subscription Center Participate in user experience research Podcasts United States — English Contact IBM Privacy Terms of use Accessibility