Selection string

The selection string specifies the characteristics that a host must have to match the resource requirement. It is a logical expression that is built from a set of resource names. The selection string is evaluated for each host; if the result is non-zero, then that host is selected. When used in conjunction with a cu string, hosts not belonging to compute unit are not considered.

Syntax

The selection string can combine resource names with logical and arithmetic operators. Non-zero arithmetic values are treated as logical TRUE, and zero as logical FALSE. Boolean resources (for example, server to denote LSF server hosts) have a value of one if they are defined for a host, and zero if they are not defined for the host.

The resource names swap, idle, login, and cpu are accepted as aliases for swp, it, ls, and r1m respectively.

The ut index measures CPU utilization, which is the percentage of time spent running system and user code. A host with no processes running has a ut value of 0 percent; a host on which the CPU is completely loaded has a ut of 100 percent. You must specify ut as a floating-point number between 0.0 and 1.0.

For the string resources type and model, the special value any selects any value and local selects the same value as that of the local host. For example, type==local selects hosts of the same type as the host submitting the job. If a job can run on any type of host, include type==any in the resource requirements.

If no type is specified, the default depends on the command. For bhosts, bsub, lsplace, lsrun, and lsgrun the default is type==local unless a string or Boolean resource is specified, in which case it is type==any. For lshosts and lsload, the default is type==any.

Tip:

When PARALLEL_SCHED_BY_SLOT=Y in lsb.params, the resource requirement string keyword ncpus refers to the number of slots instead of the number of CPUs, however lshosts output will continue to show ncpus as defined by EGO_DEFINE_NCPUS in lsf.conf.

You can also filter hosts by using 'slots' or 'maxslots' in the select string of resource requirements. For example:

select[slots>4 && maxslots < 10 || mem > 10] order[-slots:maxslots:maxmem:ut]

Specify multiple -R options

bsub accepts multiple -R options for the select section in simple resource requirements.

Restriction:

Compound resource requirements do not support multiple -R options.

You can specify multiple resource requirement strings instead of using the && operator. For example:

bsub -R "select[swp > 15]" -R "select[hpux]"

LSF merges the multiple -R options into one string and dispatches the job if all of the resource requirements can be met. By allowing multiple resource requirement strings and automatically merging them into one string, LSF simplifies the use of multiple layers of wrapper scripts.

You cannot specify more than one select section in the same -R option. Use the logical and (&&) operator to specify multiple selection strings in the same select section. For example, the following command submits a job called myjob to run on a host that has more than 15 MB of swap space available, and maximum RAM larger than 100 MB. The job is expected to reserve 100 MB memory on the host:

% bsub -R "select [swp > 15 &&  maxmem > 100]  rusage[mem = 100] " myjob

The number of -R option sections is unlimited.

Select shared string resources

You must use single quote characters (') around string-type shared resources. For example, use lsload -s to see the shared resources that are defined for the cluster:

lsload -s
RESOURCE                                VALUE       LOCATION
os_version                                4.2       pc36
os_version                                4.0       pc34
os_version                                4.1       devlinux4
cpu_type                                   ia       pc36
cpu_type                                   ia       pc34
cpu_type                              unknown       devlinux4

Use a select string in lsload -R to specify the shared resources you want to view, enclosing the shared resource values in single quotes. For example:

lsload -R "select[os_version=='4.2' || cpu_type=='unknown']" 
HOST_NAME       status  r15s   r1m  r15m   ut    pg  ls    it   tmp   swp   mem
pc36                ok   0.0   0.2   0.1   1%   3.4   3     0  895M  517M  123M
devlinux4           ok   0.0   0.1   0.0   0%   2.8   4     0 6348M  504M  205M

Note: When reserving resources based on host status (bsub -R "status==ok"), the host status must be the one displayed by running bhosts not lsload.

Operators

These operators can be used in selection strings. The operators are listed in order of decreasing precedence.

Syntax	Meaning
(a)	When LSF_STRICT_RESREQ=Y is configured in lsf.conf, an expression between parentheses has higher priority than other operators.
-a !a	Negative of a Logical not: 1 if a==0, 0 otherwise
a * b a / b	Multiply a and b Divide a by b
a + b a - b	Add a and b Subtract b from a
a > b a < b a >= b a <= b	1 if a is greater than b, 0 otherwise 1 if a is less than b, 0 otherwise 1 if a is greater than or equal to b, 0 otherwise 1 if a is less than or equal to b, 0 otherwise
a == b a != b	1 if a is equal to b, 0 otherwise 1 if a is not equal to b, 0 otherwise
a && b	Logical AND: 1 if both a and b are non-zero, 0 otherwise
a \|\| b	Logical OR: 1 if either a or b is non-zero, 0 otherwise

Examples

select[(swp > 50 && type == x86_64) || (swp > 35 && type == LINUX)]
select[((2*r15s + 3*r1m + r15m) / 6 < 1.0) && !fs && (cpuf > 4.0)]

Specify shared resources with the keyword “defined”

A shared resource may be used in the resource requirement string of any LSF command. For example, when submitting an LSF job that requires a certain amount of shared scratch space, you might submit the job as follows:

bsub -R "avail_scratch > 200 && swap > 50" myjob

The above assumes that all hosts in the cluster have access to the shared scratch space. The job is only scheduled if the value of the "avail_scratch" resource is more than 200 MB and goes to a host with at least 50 MB of available swap space.

It is possible for a system to be configured so that only some hosts within the LSF cluster have access to the scratch space. To exclude hosts that cannot access a shared resource, the defined(resource_name) function must be specified in the resource requirement string.

For example:

bsub -R "defined(avail_scratch) && avail_scratch > 100 && swap > 100" myjob

would exclude any hosts that cannot access the scratch resource. The LSF administrator configures which hosts do and do not have access to a particular shared resource.

Supported resource names in the defined function

Only resource names configured in lsf.shared, except dynamic NUMERIC resource names with INTERVAL fields defined are accepted as the argument in the defined (resource_name) function.

The following resource names are not accepted in the defined (resource_name) function:

The following built-in resource names:

r15s r1m r15m ut pg io ls it tmp swp mem ncpus ndisks maxmem
maxswp maxtmp cpuf type model status rexpri server and hname

Dynamic NUMERIC resource names configured in lsf.shared with INTERVAL fields defined. In the default configuration, these are mode, cntrl, it_t.)
Other non-built-in resource names not configured in lsf.shared.

Specify exclusive resources

An exclusive resource may be used in the resource requirement string of any placement or scheduling command, such as bsub, lsplace, lsrun, or lsgrun. An exclusive resource is a special resource that is assignable to a host. This host will not receive a job unless that job explicitly requests the host. For example, use the following command to submit a job requiring the exclusive resource bigmem:

bsub -R "bigmem" myjob

Jobs will not be dispatched to the host with the bigmem resource unless the command uses the -R option to explicitly specify "bigmem".

To configure an exclusive resource, first define a static Boolean resource in lsf.shared. For example:

Begin Resource
...
bigmem Boolean () ()
End Resource

Assign the resource to a host in the Host section of lsf.cluster.cluster_name for static hosts or LSF_LOCAL_RESOURCES for dynamic hosts. Prefix the resource name with an exclamation mark (!) to indicate that the resource is exclusive to the host. For example:

Begin Host
HOSTNAME   model  type  server r1m pg tmp RESOURCES       RUNWINDOW
...
hostE      !      !     1      3.5 () ()  (linux !bigmem) ()
...
End Host
 
LSF_LOCAL_RESOURCES="[resource linux] [!bigmem]"

Strict syntax for resource requirement selection strings

When LSF_STRICT_RESREQ=Y is configured in lsf.conf, resource requirement strings in select sections must conform to a more strict syntax. The strict resource requirement syntax only applies to the select section. It does not apply to the other resource requirement sections (order, rusage, same, span, or cu). When LSF_STRICT_RESREQ=Y in lsf.conf, LSF rejects resource requirement strings where an rusage section contains a non-consumable resource.

Strict select string syntax usage notes

The strict syntax is case-sensitive.

Boolean variables, such as fs, hpux, cs, can only be computed with the following operators

&& || !

String variables, such as type, can only be computed with the following operators:

= == != < > <= >=

For function calls, blanks between the parentheses "( )" and the resource name are not valid. For example, the following is not correct:

defined(   mg   )

Multiple logical NOT operators (!) are not valid. For example, the following is not correct:

!!mg

The following resource requirement is valid:

!(!mg)

At least one blank space must separate each section. For example, the following are correct:

type==any rusage[mem=1024]
select[type==any] rusage[mem=1024]
select[type==any]rusage[mem=1024]

but the following is not correct:

type==anyrusage[mem=1024]

Only a single select section is supported by the stricter syntax. The following is not supported in the same resource requirement string:

select[mem>0] select[maxmem>0]

Escape characters (like '\n') are not supported in string literals.

A colon (:) is not allowed inside the select string. For example, select[mg:bigmem] is not correct.

inf and nan can be used as resource names or part of a resource name.

Single or double quotes are only supported around the whole resource requirement string, not within the square brackets containing the selection string. For example, in lsb.queues, RES_REQ='swp>100' and RES_REQ="swp>100" are correct. Neither RES_REQ=select['swp>100'] nor RES_REQ=select["swp>100"] are supported.

The following are correct bsub command-level resource requirements:

bsub -R "'swp>100'"
bsub -R '"swp>100"'

The following are not correct:

bsub -R "select['swp>100']"
bsub -R 'select["swp>100"]'

Some incorrect resource requirements are no longer silently ignored. For example, when LSF_STRICT_RESREQ=Y is configured in lsf.conf, the following are rejected by the resource requirement parser:

microcs73 is rejected:
```
linux rusage[mem=16000] microcs73
```
select[AMD64] is rejected:
```
mem < 16384 && select[AMD64]
```
linux is rejected:
```
rusage[mem=2000] linux
```
Using a colon (:) to separate select conditions, such as linux:qscw.
The restricted syntax of resource requirement select strings that are described in the lsfintro(1) man page is not supported.

Explicit and implicit select sections

An explicit select section starts from the section keyword and ends at the begin of next section, for example: the select section is select[selection_string]. An implicit select section starts from the first letter of the resource requirement string and ends at the end of the string if there are no other resource requirement sections. If the resource requirement has other sections, the implicit select section ends before the first letter of the first section following the selection string.

All explicit sections must begin with a section keywords (select, order, span rusage, or same). The resource requirement content is contained by square brackets ([) and (]).

An implicit select section must be the first resource requirement string in the whole resource requirement specification. Explicit select sections can appear after other sections. A resource requirement string can have only one select section (either an explicit select section or an implicit select section). A section with an incorrect keyword name is not a valid section.

An implicit select section must have the same format as the content of an explicit select section. For example, the following commands are correct:

bsub -R "select[swp>15] rusage[mem=100]" myjob
bsub -R "swp > 15 rusage[mem=100]" myjob
bsub -R "rusage[mem=100] select[swp >15]" myjob

Examples

The following examples illustrate some correct resource requirement select string syntax.

bsub -R "(r15s * 2 + r15m) < 3.0 && !(type == IBMAIX4) || fs" myjob
If swap space is equal to 0, the following means TRUE; if swap space is not equal to 0, it means FALSE:
```
bsub -R "!swp" myjob 
```
Select hosts of the same type as the host submitting the job:
```
bsub -R "type == local" myjob
```
Select hosts that are not the same type as the host submitting the job:
```
bsub -R "type != local" myjob
```
bsub -R "r15s < 1.0 || model ==local && swp <= 10" myjob
Since && has a higher priority than ||, this example means:
```
r15s < 1.0 || (model == local  && swp <=10)
```

This example has different meaning from the previous example:

bsub -R "(r15s < 1.0 || model == local) && swp <= 10" myjob

This example means:

(r15s < 1.0 || model == local) && swp <= 10

Check resource requirement syntax

Use the BSUB_CHK_RESREQ environment variable to check the compatibility of your existing resource requirement select strings against the stricter syntax enabled by LSF_STRICT_RESREQ=Y in lsf.conf.

Set the BSUB_CHK_RESREQ environment variable to any value enable bsub to check the syntax of the resource requirement selection string without actually submitting the job for scheduling and dispatch. LSF_STRICT_RESREQ does not need to be set to check the resource requirement selection string syntax.

bsub only checks the select section of the resource requirement. Other sections in the resource requirement string are not checked.

If resource requirement checking detects syntax errors in the selection string, bsub returns an error message. For example:

bsub -R "select[type==local] select[hname=abc]" sleep 10
Error near "select": duplicate section. Job not submitted.
echo $?
255

If no errors are found, bsub returns a successful message and exit code zero. For example:

env | grep BSUB_CHK_RESREQ
BSUB_CHK_RESREQ=1
bsub -R "select[type==local]" sleep 10
Resource requirement string is valid.
echo $?
0

If BSUB_CHK_RESREQ is set, but you do not specify -R, LSF treats it as empty resource requirement. For example:

bsub sleep 120
Resource requirement string is valid.
echo $?
0

Resizable jobs

Resize allocation requests are scheduled using hosts as determined by the select expression of the merged resource requirement. For example, to run an autoresizable job on 1-100 slots, but only on hosts of type X86_64, the following job submission specifies this resource request:

bsub -ar -app <appplicaion_file> -n "1,100" -R "rusage[swp=100,license=1]" myjob

Every time the job grows in slots, slots are requested on hosts of the specified type.

Note: Resizable jobs cannot have compound or alternative resource requirements.