Using the Grid/Job script execution

Introduction

In this example you will create a small executable script and run it on the Grid. You can execute any unix command in this script. However, remember that your home directory (and also your environment) is not available on a Grid node, so you need to specify any files you need in the JDL file.

Script and JDL file

NOTE: To prevent that all examples will end up in the same directory: make a seperate directory "example_sleepjob" and work from there

For this example we will need a script. Type the following in a text editor and save it into a file "sleepjob.sh":

#!/bin/bash
date
/bin/hostname
sleep 10
date

This script will output the date and time, the hostname of the machine it is running on, then wait 10 seconds and output the date and time again.

Now make this file executable:

chmod 700 sleepjob.sh

Now create the file "sleep_job.jdl" with these contents:

Type="Job";
JobType="Normal";

Executable = "/bin/sh";
Arguments = "sleepjob.sh";

StdError = "stderr.log";
StdOutput = "stdout.log";

InputSandbox = "$HOME/example_sleepjob/sleepjob.sh";
OutputSandbox = {"stderr.log", "stdout.log"};

// optionally specify how long your job will run 
Requirements = (other.GlueCEPolicyMaxWallClockTime<=20);

Submit the job

Now you can submit your job. Remember to start your session (for example with the command startGridSession)

startGridSession
glite-wms-job-submit  -d $USER -o myjobs $HOME/example_sleepjob/sleep_job.jdl 

To check its status:

glite-wms-job-status -i myjobs

You will see its status changing from "ready" to "scheduled", to "running" to "done".

Retrieve output

To retrieve its output (use the option --dir to output to a place other then /scratch):

glite-wms-job-output -i myjobs

The output for that command looks like:

================================================================================

                        JOB GET OUTPUT OUTCOME

Output sandbox files for the job:
https://wms.grid.sara.nl:9000/lco7GbB8fVj2TCp9qGkA
have been successfully retrieved and stored in the directory:
/scratch/jlennon_lco7GbB8fVj2TCp9qGkA

================================================================================

You can access the results in that directory:

ls -l /scratch/jlennon_lco7GbB8fVj2TCp9qGkA

will give you something similar to:

-rw-r--r--  1 jlennon jlennon  0 Jun 17 11:31 stderr.log
-rw-r--r--  1 jlennon jlennon 72 Jun 17 11:31 stdout.log

The output of the script was saved in stdout.log:

cat /scratch/jlennon_lco7GbB8fVj2TCp9qGkA/stdout.log

will show something similar to:

Tue Jun 17 11:28:14 CEST 2008
gb-wn01-ams
Tue Jun 17 11:28:24 CEST 2008

Clean up

When you are done with the results, you can permanently remove them:

rm -rf /scratch/jlennon_lco7GbB8fVj2TCp9qGkA/