| [ Platform Documentation ] | [ Title ] | [ Contents ] | [ Previous ] | [ Next ] | [ Index ] |
[ Top ]
Job Dependency Scheduling
About job dependency scheduling
Sometimes, whether a job should start depends on the result of another job. For example, a series of jobs could process input data, run a simulation, generate images based on the simulation output, and finally, record the images on a high-resolution film output device. Each step can only be performed after the previous step finishes successfully, and all subsequent steps must be aborted if any step fails.
Some jobs may not be considered complete until some post-job processing is performed. For example, a job may need to exit from a post-execution job script, clean up job files, or transfer job output after the job completes.
In LSF, any job can be dependent on other LSF jobs. When you submit a job,
you use bsub -w to specify a dependency expression, usually based on the
job states of preceding jobs.
LSF will not place your job unless this dependency expression evaluates to TRUE. If you specify a dependency on a job that LSF cannot find (such as a job that has not yet been submitted), your job submission fails.
Specifying a job dependency
To specify job dependencies, use bsub -w to specify a dependency expression
for the job.
bsub -w 'dependency_expression'
The dependency expression is a logical expression composed of one or more dependency conditions. For syntax of individual dependency conditions, see Dependency Conditions.
To make dependency expression of multiple conditions, use the following logical operators:
- Use parentheses to indicate the order of operations, if necessary.
- Enclose the dependency expression in single quotes (') to prevent the shell from interpreting special characters (space, any logic operator, or parentheses). If you use single quotes for the dependency expression, use double quotes for quoted items within it, such as job names.
- Job names specify only your own jobs, unless you are an LSF administrator.
- Use double quotes (") around job names that begin with a number.
- In Windows, enclose the dependency expression in double quotes (") when the expression contains a space. For example:
-
In the job name, specify the wildcard character (*) at the end of a string, to indicate all jobs whose name begins with the string. For example, if you use
jobA*as the job name, it specifies jobs namedjobA,jobA1,jobA_test,jobA.log, etc.
Multiple jobs with the same name
By default, if you use the job name to specify a dependency condition, and more than one of your jobs has the same name, all of your jobs that have that name must satisfy the test.
To change this behavior, set JOB_DEP_LAST_SUB in lsb.params to 1. Then,
if more than one of your jobs has the same name, the test is done on the one
submitted most recently.
[ Top ]
Dependency Conditions
The following dependency conditions can be used with any job:
-
done(job_ID |"job_name") -
ended(job_ID |"job_name") -
exit(job_ID [,[op] exit_code]) -
exit("job_name"[,[op] exit_code]) -
external(job_ID |"job_name","status_text") -
job_ID |
"job_name" -
post_done(job_ID| "job_name") -
post_err(job_ID| "job_name") -
started(job_ID| "job_name")
done
done(job_ID | "job_name")
The job state is DONE.
ended
ended(job_ID | "job_name")
The job state is EXIT or DONE.
exit
exit(job_ID | "job_name"[,[operator] exit_code])
where operator represents one of the following relational operators:
The job state is EXIT, and the job's exit code satisfies the comparison test.
If you specify an exit code with no operator, the test is for equality (== is assumed).
If you specify only the job, any exit code satisfies the test.
-
exit (myjob)The job named
myjobis in the EXIT state, and it does not matter what its exit code was. -
exit (678,0)The job with job ID 678 is in the EXIT state, and terminated with exit code 0.
-
exit ("678",!=0)The job named
678is in the EXIT state, and terminated with any non-zero exit code.
external
external(job_ID | "job_name", "status_text")
Specify the first word of the job status or message description (no spaces). Only the first word is evaluated.
The job has the specified job status, or the text of the job's status begins with the specified word.
Job ID or job name
job_ID | "job_name"
If you specify a job without a dependency condition, the test is for the DONE state (LSF assumes the "done" dependency condition by default).
post_done
post_done(job_ID | "job_name")
The job state is POST_DONE (the post-processing of specified job has completed without errors).
post_err
post_err(job_ID | "job_name")
The job state is POST_ERR (the post-processing of specified job has completed with errors).
started
started(job_ID | "job_name")
The job state is:
Advanced dependency conditions
If you use job arrays, you can specify additional dependency conditions that only work with job arrays.
To use other dependency conditions with array jobs, specify elements of a job array in the usual way.
Job dependency examples
-
The simplest kind of dependency expression consists of only one dependency condition. For example, if
JobAdepends on the successful completion ofJobB, submit the job as shown:bsub -J "JobA" -w 'done(JobB)'command -
-w 'done(312) && (started(Job2)||exit("99Job"))'The submitted job will not start until the job with the job ID of 312 has completed successfully, and either the job named
Job2has started, or the job named99Jobhas terminated abnormally. -
-w '"210"'The submitted job will not start unless the job named 210 is finished. The numeric job name should be doubly quoted, since the UNIX shell treats
-w "210"the same as-w 210, which would evaluate the job with the job ID of 210.
[ Top ]
| [ Platform Documentation ] | [ Title ] | [ Contents ] | [ Previous ] | [ Next ] | [ Index ] |
|
Date Modified: January 12, 2004
Platform Computing: www.platform.com Platform Support: support@platform.com Platform Information Development: doc@platform.com Copyright © 1994-2004 Platform Computing Corporation. All rights reserved. |