Using the Grid/Job binary execution

This page has been marked as Todo, which means it needs serious work. Please feel free to add to this page. Contribute help

• Problem: you want to execute your own binary program on the grid.
• Solution: use a script to execute your program and use JDL to send your program with your job

When you have a binary program that you want to execute on the Grid you can send it along with the job submission. This can be done when the executable is not too large (the limit is about 10MB, for larger executables: follow the procedures for large files). To send such an executable along we use the InputSandBox in the job description. The program itself will be executed by a simple shell script ("wrapping"). There are several reasons to wrap the call to your executable with a script. One important one is that the executable file might not have executable permissions after it is copied to the node. A second is that it is more flexible in the use of (script) parameters and to redirect the output. In short, this script provides the correct environment for the execution of the binary.

NOTE on compilation: It is advisable to compile your programs on the User Interface (UI) Machine. Many Grid nodes have similar environments and the chances of your job running on a remote Worker Node are larger when your program is able to run on the UI.

The binary file myexecutable will print out some welcome message and an argument received as inputs. Create the file "myexecutable.c" and type the following text:

#include <stdio.h>
#include <stdlib.h>
int main (int argc, char *argv[])
  {
  if (argc == 2) {
    printf("Hello %s \n",argv[1]);
  } else {
    printf ("Usage: myexecutable myname\n");
    exit (-1);
  }
}

Compile that file:

gcc myexecutable.c -o myexecutable

The script "script.sh" is sent along in the InputSandbox. It executes the executable with the argument specified in the Arguments: atribute script.sh INSERT_YOUR_NAME. Feel free to fill in your own name as the argument. The content of the script is as follows:

cat script.sh
#!/bin/sh

# On the worker node you may have not
# execution rights
echo "setting right permissions"
chmod 755 myexecutable

# run example passing as argument the shell script argument
# and redirecting its output on exe.out file
echo "executing program now..."
./myexecutable $1 > exe.out

Now you can create the job file "yourexe.jdl" and submit it:

$ cat yourexe.jdl
Type = "Job";
JobType = "Normal";
Executable = "/bin/sh";
Arguments = "script.sh INSERT_YOUR_NAME";
StdOutput = "script.out";
StdError = "script.err";
InputSandbox = {"script.sh","myexecutable"};
OutputSandbox = {"script.out","script.err","exe.out"};
ShallowRetryCount = 1;

Prev:

Up:

Next: