Introduction Grid Computing Lab Course Overview

From PDP/Grid Wiki
Revision as of 11:35, 28 August 2005 by Davidg@nikhef.nl (talk | contribs)
Jump to navigationJump to search

Structure

The aim of the lab courses will be to install, deploy and operate a mini-grid, with some applications and services. The entire minigrid will be build and run by the students partipating in the course (of course with some help from the tutors). At the end of the lab course you'll know what a grid is, be able to build one, and what is needed to make it useful for applications.

A grid contains a few components that you cannot do without:

  • a common trust domain (authentication)
  • communities of resources and users (authorization)
  • an information service

and of course some services to make the grid useful, like

  • a job submission service
  • data movement or indexing
  • workload management
  • database access
  • your favourice custom services ...

For each of these, literature and documentation are given below, together with one or two projects (assignments) to be picked up by a team of students (say, 2-3 students per project).

Authentication

Trust in the grid today is established via a Public Key Infrastructure (PKI). Every entity in the system is issues with a "certificate" that links an identifier (the persons name, or a DNS name) to a piece of unique cryptographic data (an RSA keypair, for instance). These certificates usually have a limited lifetime when stored in a file, or are carried on hardware tokens like smart-cards and USB keys.

Commercial providers, like Verisign, Thawte, or Entrust, operate a Certification Authority and sellX.509 public key certificates.

You can also setup an X.509 Certification Authority (CA) yourself. The simplest is to use the OpenSSL commands, that even come with a shellscript to automate the task. More complete functionality can be found in OpenCA. Recent version of the Globus Toolkit also come with a package called "globus-simple-ca".

But there is more to authentication than just issuing certificates to users and hosts. Keys can be compromised or lost, the data in the certificate may become invalid, etc. These issues must be considered, also for the course's CA service.

Literature

  • [Media:ITU-T-X500-series-UvA.zip ITU-T X.500 Document series], Open Systems Interconnection — The Directory: Overview of Concepts, Models, and Services, Recommendation X.500, ISO/IEC 9594-1
  • Global Grid Forum CAOPS-WG
  • RFC 2459, Certificate Revocation Lists
  • RFC 2560, On-line Certificate Status Protocol
  • EUGridPMA, the European Grid Authentication Policy Management Authority in e-Science
  • IGTF, the International Grid Trust Federation
  • Minimum CA Requirements for Traditional X.509 Public Key Certification Authorities with secured infrastructure.

Project proposals

  • Build a simple CA service, e.g. based on OpenSSL, that can be used by your fellow students to obtain certificates.
  • Describe the way in which you would identify entities, and what the level of trust in your certificates should be. Describe what the limitations, vulnerabilities, and possible attack vectors.
  • Build a more scalable system, incorporating Registration Authorities, and on-line checking of the status of your certificates (using an independent client program).
  • Integrate on-line checks in a piece of middleware (optional)


Authorization

Users and resources in a grid are grouped in Virtual Organisations. These can be based on directories of users stored in LDAP directories, on attributes issued to the user by the VO, and embedded in the proxy certificate, like in VOMS, or by having a Community Authorization Service (CAS) issue the proxy to the user.

The proxy certificate is the basis for grid authorization today, and enables single sign-on. To access these proxy certs from web portals (and for proxy renewal for long-running jobs), a MyProxy service has been built. This MyProxy service is required for portal operations.

Literature

Project proposals

  • Provide a VO management service for the two grid clusters that will be built lateron (this can best be done with a VO-LDAP server).
  • Old-style systems required the system administrators of a grid site to maintain a file (grid-mapfile) with a list of the authorized users. With VO-LDAP and VOMS, the membership list can be maintained in a central directory for the VO. What else is needed for smooth operation with a VO-LDAP, i.e. how to prevent the sysadmin from having to type something for each new member? (keywords: gridmapdir, LCMAPS, WorkSpace Service/WSS).
  • Setup a CAS service (with GT4) and CAS-enable an example service.

Information Services

A grid consists of many autonomous resources, that come and go. A resource information system to find the resources available for you is therefor vitally important. The system must be stable, scalable to several hunderd sites, hunderds of queries per second, and universally understood.

Information systems have evolved significantly over the years. The Globus Toolkit shipped originally with the "Metacomputing Directory Service" (later renamed to Monitoring and Discovery Service, MDS). The information was presented via an LDAP interface with a proprietary schema. This system later evolved into the BDII for increased performance.

R-GMA (a relational implementation of the GGF Grid Monitoring Archirecture [REF]) uses a structure, SQL based virtual database across all sites in the grid to propagae information in a producer-consumer paradigm.

The Web services based GT4 release contains a completely new version of MDS, that's based on a notification/subscription mechanism and the new WS-notification standards. [ADD REFs]. And with Condor you get it's own monitoring system Hawkeye.

Note also the existence of UDDI [ref], but that is a registry only (not an information system, look and find out why!)

Essential for all of them is a common way to express the information in a schema. There are many schema in use. The most popular today is the GLUE schema.

Besides there are various management presentation tools like GridICE, MapCenter, GOC Monitor &c.

Literature

Projects

  • extract resource information from a host (or a cluster when available) and express it in one of the information systems listed above.
  • once both clusters are operational to some degree, make sure you get the same information fromn both systems, so that the users can decide which one is the best to use (i.e. make a brokering decision based on the information)

Job Management and Clusters

The first resources to populate a grid were compute clusters (a computational grid). These resources were inintially supercomputers, but since we don't have one handy we stick with clusters.


[NEED DESCRIPTION AND REFS]


Projects

(for two teams of ~3 students each)

  • build a PBS/Torque based cluster with a single head-node a two worker nodes.
    • can you run multi-node jobs?
    • what happens if a node fails (try pulling the network plug!)
  • build a Condor based cluster with three nodes in total, of which one is also used for other tasks?
    • can use use idle cycles on the shared node?
    • what happens in case of failure?
    • can you do job migration?
  • add MPI support to both clusters
  • add a GT2 or GT4 GRAM service to both clusters (use the same on both initially!)


Integration

Scheduling and brokering in a grid.

Projects

  • build a broker that looks at the info system and find the empty cluster (find a grid scheduler like Condor-G)
  • try multi-cluster MPI with MPICH-G2 (and a GT2 GRAM on each cluster)

Portals

Spare assignment:

Projects

  • Build a web portal with MyProxy and a portal engine like GridPort (NPACI)
Retrieved from "https://wiki.nikhef.nl/grid/index.php?title=Introduction_Grid_Computing_Lab_Course_Overview&oldid=62"