Using the Grid/Job submission
This page shows how to create and submit a simple job to the grid using the WMS.
This job involves no large input or output files.
Interacting with the WMS is done using the glite-wms-* commands.
For job submission you use: glite-wms-job-submit
|
NOTE: Make sure you have started your session . On the Lifescience Grid you can use the script startGridSession. You can do this manually by creating a valid proxy certificate (credential) with voms-proxy-init, storing that in the proxy server with myproxy-init and then delegate your credential by using glite-wms-job-delegate-proxy. |
A Simple JDL Job
To submit a job to the Workload Management System (WMS), a text file is needed in which the job is described. For this purpose a special language, Job Description Language (JDL) is used. The JDL describes the job and its requirements in simple attribute value pairs.
Here is an example of the contents of a JDL file:
// specify the jobtype
Type = "Job";
JobType = "Normal";
// specify the executable
Executable = "/bin/hostname";
Arguments = "-f";
// specify the names of the standard output and error
StdOutput = "hostname.out";
StdError = "hostname.err";
// specify your output sandbox contents
OutputSandbox = {"hostname.err","hostname.out"};
// number of retries
ShallowRetryCount = 3;
// additional requirements in this case we are aiming for a short queue
Requirements = (other.GlueCEPolicyMaxWallClockTime >=30 );
When this file is submitted to the WMS, a job will be created and sent to a Worker Node in the Grid by the WMS. There it will execute the command "/bin/hostname -f". It will write its standard output in the file hostname.out and its standard error in the file hostname.err, as specified in the JDL file. This will all take place in on the remote Worker Node. In able to get results back an OutputSandbox is used. Note that the OutputSandbox is only intended to be used for small files. If you have files of 5 MB or larger please see the section on data management on how to the use the Storage Elements.
The Executable attribute specifies the command to be run on the Worker Node. The OutputSandbox attribute indicates the files to be copied back after job execution; normally these are files where output and error streams are redirected to; their names are determined by the StdOutput and StdError attributes respectively. Also the number of retries is specified, in case of failures.
We've also specified an additional requirement about the Wallclocktime. We've made an estimate this job will take less then 30 minutes to finish. It will be scheduled to run on a queue specially meant for short jobs. see also
JDL is more fully described here: Job Description Language.
The examples below assume that you have this hostname JDL file - please copy the listing above to create your own hostname.jdl.
Job Submission
A simple job can be submitted by the command:
glite-wms-job-submit -d <delegationId> -o <jobidfile> <jdlname>
The -d option uses the delegationID. This is a string which identifies your credential delegation (this has been done by the startGridsession script), further explained in Credential delegation.
The -o option is used to save the id of the job. This jobId is needed to get the status and retrieve the output from the WMS.
When you created the hostname.jdl file on the server, you can submit this job as follows:
$ glite-wms-job-submit -d $USER -o myjobs hostname.jdl Connecting to the service https://wms.grid.sara.nl:7443/glite_wms_wmproxy_server ====================== glite-wms-job-submit Success ====================== The job has been successfully submitted to the WMProxy Your job identifier is: https://wms.grid.sara.nl:9000/T1JVMOHOgl3xCIv_cN4bVg The job identifier has been saved in the following file: /home/mgjansen/myjobs ==========================================================================
The file /home/mgjansen/myjobs contains the output of the submission process as the result of the -o option. It contains the jobID(s) returned by the submission process. Here the string
https://wms.grid.sara.nl:9000/T1JVMOHOgl3xCIv_cN4bVg
is the Job Identifier. Note that this URL cannot be visted by a webbrowser unless you have valid grid credentials in your browser.
If another job is submitted using the same -o value, its jobID is appended to the same jobID file. Try it yourself.
|
NOTE: Omitting the -o option means that the jobID is not saved in a file. It is returned in the output exactly as in the above example. Note that you need the jobID to get your status and retrieve your output. When you do not save this id you will effectively loose the output of your job! |
Read how to view the status of the job in "Job status" and how to retrieve the output in "Job output" .