Using the Grid/Job script execution
Introduction
In this example you will create a small executable script and run it on the Grid. You can execute any unix command in this script. However, remember that your home directory (and also your environment) is not available on a Grid node, so you need to specify any files you need in the JDL file.
Script and JDL file
|
NOTE: To prevent that all examples will end up in the same directory: make a seperate directory "example_sleepjob" and work from there |
For this example we will need a script. Type the following in a text editor and save it into a file "sleepjob.sh":
#!/bin/bash date /bin/hostname sleep 10 date
This script will output the date and time, the hostname of the machine it is running on, then wait 10 seconds and output the date and time again.
Now make this file executable:
chmod 700 sleepjob.sh
Now create the file "sleep_job.jdl" with these contents:
Type="Job";
JobType="Normal";
Executable = "/bin/sh";
Arguments = "sleepjob.sh";
StdError = "stderr.log";
StdOutput = "stdout.log";
InputSandbox = "$HOME/example_sleepjob/sleepjob.sh";
OutputSandbox = {"stderr.log", "stdout.log"};
// optionally specify how long your job will run Requirements = (other.GlueCEPolicyMaxWallClockTime<=20);
Submit the job
Now you can submit your job. Remember to start your session (for example with the command startGridSession)
startGridSession glite-wms-job-submit -d $USER -o myjobs $HOME/example_sleepjob/sleep_job.jdl
To check its status:
glite-wms-job-status -i myjobs
You will see its status changing from "ready" to "scheduled", to "running" to "done".
Retrieve output
To retrieve its output (use the option --dir to output to a place other then /scratch):
glite-wms-job-output -i myjobs
The output for that command looks like:
================================================================================
JOB GET OUTPUT OUTCOME
Output sandbox files for the job:
https://wms.grid.sara.nl:9000/lco7GbB8fVj2TCp9qGkA
have been successfully retrieved and stored in the directory:
/scratch/jlennon_lco7GbB8fVj2TCp9qGkA
================================================================================
You can access the results in that directory:
ls -l /scratch/jlennon_lco7GbB8fVj2TCp9qGkA
will give you something similar to:
-rw-r--r-- 1 jlennon jlennon 0 Jun 17 11:31 stderr.log -rw-r--r-- 1 jlennon jlennon 72 Jun 17 11:31 stdout.log
The output of the script was saved in stdout.log:
cat /scratch/jlennon_lco7GbB8fVj2TCp9qGkA/stdout.log
will show something similar to:
Tue Jun 17 11:28:14 CEST 2008 gb-wn01-ams Tue Jun 17 11:28:24 CEST 2008
Clean up
When you are done with the results, you can permanently remove them:
rm -rf /scratch/jlennon_lco7GbB8fVj2TCp9qGkA/