Platform LSF Version 6.0 - Administering Platform LSF

[ Platform Documentation ] [ Title ] [ Contents ] [ Previous ] [ Next ] [ Index ]

Advance Reservation

Contents

About Advance Reservation
Configuring Advance Reservation
Using Advance Reservation

[ Top ]

About Advance Reservation

Advance reservations ensure access to specific hosts during specified times. An advance reservation is essentially a lock on a number of processors.

Each reservation consists of the number of processors to reserve, a list of hosts for the reservation, a start time, an end time, and an owner. You can also specify a resource requirement string instead of or in addition to a list of hosts.

During the time the reservation is active, only users or groups associated with the reservation have access to start new jobs on the reserved hosts. The reservation is active only within the time frame specified, and any given host may have several reservations in place, some of which may be active at the same time.

When an advance reservation becomes active, LSF attempts to start all jobs that reference the reservation. By default, jobs that are already running on the hosts may continue, even though they do not reference the reservation. However, if a job that references a reservation is pending because the host has reached its job slot limit, LSF frees up a job slot on the host by suspending one of the jobs that does not reference the reservation. This is the only case where advance reservation overrides another LSF job scheduling policy.

LSF treats advance reservation like other deadlines, such as dispatch windows or run windows; LSF does not schedule jobs that are likely to be suspended when a reservation becomes active. Jobs referencing the reservation are killed when the reservation expires. LSF administrators can prevent running jobs from being killed when the reservation expires by changing the termination time of the job using the reservation (bmod -t) before the reservation window closes.

Reservations can also be created for system maintenance. If a system reservation is active, no other jobs can use the reserved hosts, and LSF does not dispatch jobs to the specified hosts while the reservation is active.

Only LSF administrators or root can create or delete advance reservations. Any LSF user can view existing advance reservations.

[ Top ]

Configuring Advance Reservation

Advance reservation plugin

To enable advance reservation in your cluster, configure the advance reservation scheduling plugin schmod_advrsv in lsb.modules.

Configuring lsb.modules

Begin PluginModule SCH_PLUGIN RB_PLUGIN SCH_DISABLE_PHASES schmod_default () () schmod_advrsv () () End PluginModule

Advance reservation license

Advance reservation requires the lsf_sched_advance_reservation license feature in your license file and LSF_Sched_Advance_Reservation configured in the PRODUCTS line of lsf.cluster.cluster_name.

Allowing users to create advance reservations

Advance reservation user policies

By default, only LSF administrators or root can add or delete advance reservations. To allow other users to use brsvadd to create advance reservations and brsvdel to delete advance reservations, use the ResourceReservation section of lsb.resources to configure advance reservation policies for users.

ResourceReservation section (lsb.resources)

A ResourceReservation section specifies:

Users or user groups that can create reservations
Hosts that can be used for the reservation
Time window when reservations can be created

Each advance reservation policy is defined in a separate ResourceReservation section, so it is normal to have multiple ResourceReservation sections in lsb.resources.

Examples
Only user1 and user2 can make advance reservations on hostA and hostB. The reservation time window is between 8:00 a.m. and 6:00 p.m. every day:
Begin ResourceReservation NAME = dayPolicy USERS = user1 user2 # optional HOSTS = hostA hostB # optional TIME_WINDOW = 8:00-18:00 # weekly recurring reservation End ResourceReservation

user1 can add the following reservation for user user2 to use on hostA every Friday between 9:00 a.m. and 11:00 a.m.:

% user1@hostB> brsvadd -m "hostA" -n 1 -u "user2" -t "5:9:0-5:11:0" Reservation "user2#2" is created

Users can only delete reservations they created themselves. In the example, only user user1 can delete the reservation; user2 cannot. Administrators can delete any reservations created by users.
All users in user group ugroup1 except user1 can make advance reservations on any host in hgroup1, except hostB, between 10:00 p.m. and 6:00 a.m. every day:
Begin ResourceReservation NAME = nightPolicy USERS = ugroup1 ~user1 HOSTS = hgroup1 ~hostB TIME_WINDOW = 20:00-8:00 End ResourceReservation
The not operator (~) does not exclude LSF administrators from the policy.

USER_ADVANCE_RESERVATION is obsolete (lsb.params)

USER_ADVANCE_RESERVATION in lsb.params is obsolete in LSF Version 6.0. Use the ResourceReservation section configuration in lsb.resources to configure advance reservation policies for your cluster.

[ Top ]

Using Advance Reservation

Advance reservation commands

Use the following commands to work with advance reservations:

brsvadd

Add a reservation

brsvdel

Delete a reservation

brsvs

View reservations

Adding and removing reservations

By default, only LSF administrators or root can add or delete advance reservations.

brsvadd command

Use brsvadd to create new advance reservations. You must specify the following for the reservation:

Number of processors to reserve
This number should less than or equal to the actual number of CPUs for the hosts defined in the reservation.

Hosts for the reservation
Owners of the reservation
Time period for the reservation: Either:

Begin time and end time for a one-time reservation

OR

Time window for a recurring reservation

The brsvadd command returns a reservation ID that you use when you submit a job that uses the reserved hosts. Any single user or user group can have a maximum of 100 reservation IDs.

Specifying hosts for the reservation

Use one or both of the following brsvadd options to specify hosts for which processors are reserved:

The -m option lists the hosts needed for the reservation
The hosts listed the -m option can be local to the cluster or hosts leased from remote clusters. At job submission, LSF considers the hosts in the specified order.

If you also specify a resource requirement string with the -R option, -m is optional.

The -R option selects hosts for the reservation according to a resource requirements string
Only hosts that satisfy the resource requirement expression are reserved. -R accepts any valid resource requirement string, but only the select string takes effect.

If you also specify a host list with the -m option, -R is optional.

Adding a one-time reservation (-b and -e)

Use the -b and -e options of brsvadd to specify the begin time and end time of a one-time advance reservation. One-time reservations are useful for dedicating hosts to a specific user or group for critical projects.

The day and time are in the form:

[[[year:]month:]day:]hour:minute

with the following ranges:

year: any year after 1900 (YYYY)
month: 1-12 (MM)
day of the month: 1-31 (dd)
hour: 0-23 (hh)
minute: 0-59 (mm)

You must specify at least hour:minute. Year, month, and day are optional. Three fields are assumed to be day:hour:minute, four fields are assumed to be month:day:hour:minute, and five fields are year:month:day:hour:minute.

If you do not specify a day, LSF assumes the current day. If you do not specify a month, LSF assumes the current month. If you specify a year, you must specify a month.

You must specify a begin and an end time. The time value for -b must use the same syntax as the time value for -e. It must be earlier than the time value for -e, and cannot be earlier than the current time.

Examples
The following command creates a one-time advance reservation for 1024 processors on host hostA for user user1 between 6:00 a.m. and 8:00 a.m. today:
% brsvadd -n 1024 -m hostA -u user1 -b 6:0 -e 8:0 Reservation "user1#0" is created

The hosts specified by -m can be local to the cluster or hosts leased from remote clusters.
The following command creates a one-time advance reservation for 1024 processors on a host of any type for user user1 between 6:00 a.m. and 8:00 a.m. today:
% brsvadd -n 1024 -R "type==any" -u user1 -b 6:0 -e 8:0 Reservation "user1#1" is created
The following command creates a one-time advance reservation that reserves 12 CPUs on hostA between 6:00 p.m. on 01 December 2003 and 6:00 a.m. on 31 January 2004:
% brsvadd -n 12 -m hostA -u user1 -b 2003:12:01:18:00 -e 2004:01:31:06:00 Reservation user1#2 is created
Adding a recurring reservation (-t)

Use the -t option of brsvadd to specify a recurring advance reservation. The -t option specifies a time window for the reservation. Recurring reservations are useful for scheduling regular system maintenance jobs.

The day and time are in the form:

[day:]hour[:minute]

with the following ranges:

day of the week: 0-6
hour: 0-23
minute: 0-59

Specify a time window one of the following ways:

hour-hour
hour:minute-hour:minute
day:hour:minute-day:hour:minute

You must specify at least the hour. Day of the week and minute are optional. Both the start time and end time values must use the same syntax. If you do not specify a minute, LSF assumes the first minute of the hour (:00). If you do not specify a day, LSF assumes every day of the week. If you do specify the day, you must also specify the minute.

When the job starts running, the run limit of the reservation is set to the minimum of the job run limit (if specified), the queue run limit (if specified), or the duration of the reservation time window. LSF administrators can prevent running jobs from being killed when the reservation expires by changing the termination time of the job using the reservation (bmod -t) before the reservation window closes.

Examples
The following command creates an advance reservation for 1024 processors on two hosts hostA and hostB for user group groupA every Wednesday from 12:00 midnight to 3:00 a.m.:
% brsvadd -n 2048 -m "hostA hostB" -g groupA -t "3:0:0- 3:3:0" Reservation "groupA#0" is created
The following command creates an advance reservation for 1024 processors on hostA for user user2 every weekday from 12:00 noon to 2:00 p.m.:
% brsvadd -n 1024 -m "hostA" -u user2 -t "12:0-14:0" Reservation "user2#0" is created
The following command creates a system reservation on hostA every Friday from 6:00 p.m. to 8:00 p.m.:
% brsvadd -n 1024 -m hostA -s -t "5:18:0-5:20:0" Reservation "system#0" is created

While the system reservation is active, no other jobs can use the reserved hosts, and LSF does not dispatch jobs to the specified hosts.
The following command creates an advance reservation for 1024 processors on hosts hostA and hostB with more that 50 MB of swap space for user user2 every weekday from 12:00 noon to 2:00 p.m.:
% brsvadd -n 1024 -R "swp > 50" -m "hostA hostB" -u user2 -t "12:0-14:0" Reservation "user2#1" is created
Removing an advance reservation (brsvdel)

Use brsvdel to delete reservations. Specify the reservation ID for the reservation you want to delete. For example:

% brsvdel user1#0 Reservation user1#0 is being deleted

You can only delete one reservation at a time.

For more information

See Time Syntax and Configuration for more information about specifying time windows in LSF.

Viewing reservations

brsvs command

Use brsvs to show current reservations:

% brsvsRSVID TYPE USER NCPUS RSV_HOSTS TIME_WINDOW user1#0 user user1 1024 hostA:1024 11/12/6/0-11/12/8/0 user2#0 user user2 1024 hostA:1024 12:0-14:0 * groupA#0 group groupA 2048 hostA:1024 3:0:0-3:3:0 * hostB:1024 system#0 sys system 1024 hostA:1024 5:18:0-5:20:0 *

In the TIME_WINDOW column:
A one-time reservation displays fields separated by slashes (month/day/hour/minute). For example:
11/12/14/0-11/12/18/0
A recurring reservation displays fields separated by colons (day:hour:minute). An asterisk (*) indicates a recurring reservation. For example:
5:18:0 5:20:0 *
Showing a weekly planner (brsvs -p)

Use brsvs -p to show a weekly planner for specified hosts using advance reservation. The all keyword shows the planner for all hosts with reservations. MAX indicates the configured maximum number of job slots for the host.

% brsvs -p allRSVID TYPE USER NCPUS RSV_HOSTS TIME_WINDOW user1#0 user user1 1024 hostA:1024 11/12/6/0-11/12/8/0 user2#0 user user2 1024 hostA:1024 12:0-14:0 * groupA#0 group groupA 2048 hostA:1024 3:0:0-3:3:0 * hostB:1024 system#0 sys system 1024 hostA:1024 5:18:0-5:20:0 * HOST: hostA (MAX = 1024) Week: 11/11/2004 - 11/17/2004 Hour:Min Sun Mon Tue Wed Thu Fri Sat ------------------------------------------------------------------- 0:0 0 0 0 1024 0 0 0 0:10 0 0 0 1024 0 0 0 0:20 0 0 0 1024 0 0 0 ... 2:30 0 0 0 1024 0 0 0 2:40 0 0 0 1024 0 0 0 2:50 0 0 0 1024 0 0 0 3:0 0 0 0 0 0 0 0 3:10 0 0 0 0 0 0 0 3:20 0 0 0 0 0 0 0 ... 5:30 0 0 0 0 0 0 0 5:40 0 0 0 0 0 0 0 5:50 0 0 0 0 0 0 0 6:0 0 1024 0 0 0 0 0 6:10 0 1024 0 0 0 0 0 6:20 0 1024 0 0 0 0 0 ... 7:30 0 1024 0 0 0 0 0 7:40 0 1024 0 0 0 0 0 7:50 0 1024 0 0 0 0 0 8:0 0 0 0 0 0 0 0 8:10 0 0 0 0 0 0 0 8:20 0 0 0 0 0 0 0 ... 11:30 0 0 0 0 0 0 0 11:40 0 0 0 0 0 0 0 11:50 0 0 0 0 0 0 0 12:0 1024 1024 1024 1024 1024 1024 1024 12:10 1024 1024 1024 1024 1024 1024 1024 12:20 1024 1024 1024 1024 1024 1024 1024 ... 13:30 1024 1024 1024 1024 1024 1024 1024 13:40 1024 1024 1024 1024 1024 1024 1024 13:50 1024 1024 1024 1024 1024 1024 1024 14:0 0 0 0 0 0 0 0 14:10 0 0 0 0 0 0 0 14:20 0 0 0 0 0 0 0 ... 17:30 0 0 0 0 0 0 0 17:40 0 0 0 0 0 0 0 17:50 0 0 0 0 0 0 0 18:0 0 0 0 0 0 1024 0 18:10 0 0 0 0 0 1024 0 18:20 0 0 0 0 0 1024 0 ... 19:30 0 0 0 0 0 1024 0 19:40 0 0 0 0 0 1024 0 19:50 0 0 0 0 0 1024 0 20:0 0 0 0 0 0 0 0 20:10 0 0 0 0 0 0 0 20:20 0 0 0 0 0 0 0 ... 23:30 0 0 0 0 0 0 0 23:40 0 0 0 0 0 0 0 23:50 0 0 0 0 0 0 0 HOST: hostB (MAX = 1024) Week: 11/11/2004 - 11/17/2004 Hour:Min Sun Mon Tue Wed Thu Fri Sat ------------------------------------------------------------------- 0:0 0 0 0 1024 0 0 0 0:10 0 0 0 1024 0 0 0 0:20 0 0 0 1024 0 0 0 ... 2:30 0 0 0 1024 0 0 0 2:40 0 0 0 1024 0 0 0 2:50 0 0 0 1024 0 0 0 3:0 0 0 0 0 0 0 0 3:10 0 0 0 0 0 0 0 3:20 0 0 0 0 0 0 0 ... 23:30 0 0 0 0 0 0 0 23:40 0 0 0 0 0 0 0 23:50 0 0 0 0 0 0 0

bjobs command

Use bjobs -l to show the reservation ID used by a job:

% bjobs -lJob <1152>, User <user1>, Project <default>, Status <PEND>, Queue <normal>, Reservation <user1#0>, Command <myjob> Mon Nov 12 5:13:21: Submitted from host <hostB>, CWD </home/user1/jobs>;

Submitting and modifying jobs using advance reservations

Submitting and running jobs (bsub -U)

Use the -U option of bsub to submit jobs with a reservation ID. For example:

%bsub -U user1#0 myjob

The job can only use hosts reserved by the reservation user1#0. By default, LSF selects only hosts in the reservation. Use the -m option to specify particular hosts within the list of hosts reserved by the reservation; you can only select from hosts that were included in the original reservation.

If you do not specify hosts (bsub -m) or resource requirements (bsub -R), the default resource requirement is to select hosts that are of any host type (LSF assumes "type==any" instead of "type==local" as the default select string.)

A job can only use one reservation. There is no restriction on the number of jobs that can be submitted to a reservation; however, the number of slots available on the hosts in the reservation may run out. For example, reservation user2#0 reserves 1024 slots on hostA. When all 1024 slots on hostA are used by jobs referencing user2#0, hostA is no longer available to other jobs using reservation user2#0. Any single user or user group can have a maximum of 100 reservation IDs.

Jobs referencing the reservation are killed when the reservation expires. LSF administrators can prevent running jobs from being killed when the reservation expires by changing the termination time of the job using the reservation (bmod -t) before the reservation window closes.

Modifying jobs (bmod -U)

Administrators can use the -U option of bmod to change a job to another reservation ID. For example:

%bmod -U user1#0 1234

To cancel the reservation, use the -Un option of bmod. For example:

%bmod -Un 1234

Changing job termination time (bmod -t)

Before the reservation window closes, administrators can use the -t option of bmod to change the termination time of a running job that is using the reservation. This prevents the job from being killed when the reservation expires; it does not extend the actual reservation window. For example:

%bmod -t 15:0 1234

You must set LSB_MOD_ALL_JOBS=Y in lsf.conf to use bmod -t for advance reservations.

IMPORTANT

bmod -t will not change the termination time of a pending job.

Job resource usage limits

A job using a reservation is subject to all job resource usage limits. If a limit is reached on a particular host in a reservation, jobs using that reservation cannot start on that host.

Preemption and fairshare

Higher priority jobs can only preempt other jobs that use the same reservation. In fairshare, a lower priority job can run as long as no other higher priority share holders can access the same reservation as the lower priority job.

Forcing a job to run before a reservation is active

LSF administrators can use brun to force jobs to run before the reservation is active, but the job must finish running before the time window of the reservation expires.

For example, if the administrator forces a job with a reservation to run one hour before the reservation is active, and the reservation period is 3 hours, a 4 hour run limit takes effect.

Advance reservations across clusters

You can create and use advance reservation for the MultiCluster job forwarding model. To enable this feature, you must upgrade all clusters to LSF Version 6.0 or later.

See the Using Platform LSF MultiCluster for more information.

Viewing historical accounting information for advance reservations

bacct -U

Use the -U option of the bacct command to display accounting information about advance reservations. bacct -U displays information similar to the brsvs command:
The reservation ID specified on the -U option.
The type of reservation: user or system
The user names of users who used the brsvadd command to create the advance reservations
The user names of the users who can use the advance reservations (with bsub -U)
Number of CPUs reserved
List of hosts for which processors are reserved
Time window for the reservation.
A one-time reservation displays fields separated by slashes (month/day/hour/minute). For example:
11/12/14/0-11/12/18/0
A recurring reservation displays fields separated by colons (day:hour:minute). For example:
5:18:0 5:20:0
Example

% bacct -U user1#2Accounting for: - advanced reservation IDs: user1#2, - advanced reservations created by user1, ----------------------------------------------------------------------------- RSVID TYPE CREATOR USER NCPUS RSV_HOSTS TIME_WINDOW user1#2 user user1 user1 1 hostA:1 9/16/17/36- 9/16/17/38 SUMMARY: Total number of jobs: 4 Total CPU time consumed: 0.5 second Maximum memory of a job: 4.2 MB Maximum swap of a job: 5.2 MB Total duration time: 0 hour 2 minute 0 secon

[ Top ]

[ Platform Documentation ] [ Title ] [ Contents ] [ Previous ] [ Next ] [ Index ]

Date Modified: January 12, 2004
Platform Computing: www.platform.com

Platform Support: support@platform.com
Platform Information Development: doc@platform.com

Copyright © 1994-2004 Platform Computing Corporation. All rights reserved.