Project mode preemption

Preemption occurs only when there are no free licenses. During preemption, a project releases a borrowed license to the project that owns the license (and now has demand).

Jobs that use licenses that support job suspension release their tokens and automatically resume from where they were suspended. Jobs that use licenses that do not support suspension are killed and restarted from the beginning.

Preemption applies only to project mode, and depending on your configuration takes the following into consideration:
  • Run time (a job that has the smallest run time gets preempted first, in general)
  • Fair share settings
  • Ownership
  • Priority
  • Minimal job preemption

Depending on how your projects are set up (regardless of whether they are all at the same level) your preemption is either flat or hierarchical.

Basic preemption with projects configured

When preemption occurs, License Scheduler calculates token usage for each project. The calculation considers tokens in use, tokens that are required, and token ownership value.

Based on the token usage, License Scheduler determines the projects that require tokens, and the projects that have too many.

  • Jobs belonging to projects that require tokens are scheduled first, ordered by project fair share settings.

  • Jobs belonging to projects with extra tokens are preempted first, if needed, ordered by project fair share settings and the length of time each job is running.

With PRIORITY

If project PRIORITY is configured in the Project section, the sort order of projects is based on priority, where a higher priority project is preempted last.

With PREEMPT_ORDER

If PREEMPT_ORDER is set to BY_OWNERSHIP in the Feature section, the projects are sorted by ownership.
  • Projects with the highest ownership are scheduled first.
  • Projects with the smallest ownership are preempted first.

This setting overrides basic preemption and PRIORITY.

With ENABLE_MINJOB_PREEMPTION

If ENABLE_MINJOB_PREEMPTION=Y, the number of preempted jobs is minimized. Projects with extra tokens are sorted by PRIORITY (if configured) or fair share. The jobs are then sorted by RUSAGE.

Jobs with higher RUSAGE are preempted first to minimize the number of jobs preempted.

This setting is used in addition to basic preemption or PRIORITY.

Hierarchical preemption with project groups configured

When project groups are configured, introducing a hierarchy into the project configuration, hierarchical preemption applies.

There are two methods of hierarchical preemption:
  1. Top-down (default): Preemption occurs between cousins rather than siblings. The result is to balance preemption between the entire hierarchy of projects.
  2. Bottom-up (if LS_PREEMPT_PEER=Y): Siblings can preempt each other. The result is to balance preemption within a family of projects first.
For example, your projects are set up as follows:

In top-down preemption, if P8 needs a token, it preempts from P1, P2, or P3 (who are more distant relations), not from P6 or P7 (siblings of P8).

In bottom-up preemption, P8 preempts instead from its siblings (P6 or P7).

Limits

Hierarchical preemption is also affected by any limits that are placed on the projects. If a limit is already reached (at any level of the hierarchy), License Scheduler considers the next possible node for preemption instead.