Using the Grid/Access introduction

From BiGGrid Wiki
Jump to navigation Jump to search

People affiliated with Dutch universities and institutes are eligible for using the NL Grid infrastructure; See http://www.biggrid.nl/getting-involved/forms/ for more information.

How it works

As a user you connect to the Grid by using a so-called UI (User Interface). Once you have received the right credentials (i.e. an account on a user interface and a grid certificate) you are set to go.

First, you divide your problem into smaller units, called jobs. These jobs are the unit of computation and can be submitted to the Grid. The way to do this is to describe each job in terms of a Job Description Language (JDL). This is not a programming language but consists of attribute value pairs which describe your job. Here you list which program should be executed and what data it should operate on. Your program and data can be send with the job when necessary.

Each job in the form of a JDL file is then submitted to the Workload Management System. This system schedules your jobs and knows which compute clusters in the Grid are ready to accept your job. These clusters each consist of several machines. The Compute Element (CE) is the server which communicates with the WMS and accepts jobs. It then distributes the jobs to other machines in the cluster, called Worker Nodes (WNs). These WNs are the machines which do the actual work. When finished with a job they will report back to the CE, which in turn will inform the user about the status of the job. In addition, Clusters have a storage server, called the Storage Element (SE). These servers can be used to store files on a permanent basis. Data on the SE's can be replicated at other sites and jobs can be told only to land on Worker Nodes which are close to data the jobs will operate upon.

On these wiki pages you can find more information about this whole process.

To use or not to use Grid

Grid is best suited for your problem when it is in principle embarrassingly parallel. This means that the problem should be split up relatively easily in multiple, independent parts. When you have software running MPI, or on multiple, intercommunicating threads you may find it difficult to port this to a Grid. MPI jobs can be run on our Grid, but will run on a single cluster. Of course, SARA is willing to help you bring your application to the Grid.

All our clusters run CentOS 5 and every script or program which runs on this can in be principle be submitted to the Grid.

At present all grid nodes run 64 bits Operating System en Grid middleware (gLite). The amount of memory a single job can address is about 3GB RAM.

In addition, each job should only run on a single core. When you have problems which are not suited for grid you could take a look at the Huygens supercomputeror the National Compute Cluster Lisa.

A Grid certificate

Grid Certificate