Namespaced Quattor configuration

Setup

The current Quattor 1.3 setup is entirely different from the Quattor 1.2 setup. It is therefore located in a different tree in CVS: nl.nikhef.ndpf.quattor-config/conf-ns/. This tree contains the templates and the infrastructure needed to maintain, build and deploy templates. Various tools assume that there is an environment variable $L that points to the location where the contents of this tree in CVS are present.

The build and deploy infrastructure is derived from SCDB: [1]

The major difference to standard SCDB is that our setup is independent of SubVersion, whereas SCDB relies on the presence of a SubVersion repository. The setup is still built around an ant build script build.xml, which governs various tasks.

Building and deploying templates is easiest with the shell scripts makexprof and pushxprof, respectively. Note that the latter is a symlink to the former. Syntax:

makexprof -f <facility-name> [template1] [template2] [...]
pushxprof -f <facility-name> [template1] [template2] [...]

The option -f with argument <facility-name> is required to select one of the facility names. Currently defined facilities: itb (Installation Test Bed), prd (Production cluster) and generic (miscellaneous Quattor-managed servers).

Updating the templates that describe the contents of the package repositories is done by directly executing the task update.rep.templates in the Ant script:

( cd $L ; ant update.rep.templates )

To generate update templates (i.e., templates that replace older version of software), the tool $L/../bin/rpmUpdates.pl can be used:

rpmUpdates.pl /path/to/update/rpm/directory  > updates.tpl

creates an update template (prepared for namespace rpms/updates) based on the contents of /path/to/update/rpm/directory.

Template Hierarchy

The current Quattor templates are organized in namespaced hierarchies (see below). The namespaces are inspired by those used by the Quattor Working Group ([2]), but there may be (large) deviations. The current setup is likely to change in the future, aiming to come to a more consistent approach.

The templates are all located under the directory $L/cfg. The Ant build file (build.xml) requires the presence of the directory facility/<facility-name>. In this directory, a file cluster.pan.includes may exist that specifies which include roots should be used when compiling the templates. For example:

cluster.pan.includes=sites/ndpf grid os standard

specifies that under directory $L/cfg, the include roots sites/ndpf, grid, os and standard are considered in the specified order. If two or more of the include roots contain the namespaced template, only the first one found is taken into account. The include roots specified are appended to facility/<facility-name>, which is searched recursively for all templates (also allowing for namespaced templates). Note that the above specification only includes everything directly in the listed directories and does not recursively descend into all subtrees, unlike for facility/<facility-name>. Recursively including all files under somedir is possible by specifying somedir/**/*.

The setup is based on the concept of facilities (comparable to cluster in SCDB). A facility is currently a collection of grid hosts that act as a standalone grid cluster with similar (grid) configuration. One or more grid clusters are part of a site, which has some common properties among the facilities it contains. The distinction between facility and site is not strict and may be changed in the future; in principle any kind of grouping is possible.

The above concept has led to the currently defined include roots:

facility/<facility-name>
grid
os
standard
sites

facility/<facility-name>

This subtree contains the information for a specific grouping of machines. The typical hierarchy for a facility:

+- profiles
+- local
+- repository

Per machine there has to be an object template in the profiles subdirectory. The object templates are kept as simple as possible in order not to reuse code as much as possible, and for flexibility. The subdirectory local contains miscellaneous configuration templates for the hosts in the facility. In repositories, the package repositories that are used to resolve requested packages are defined (as references to the repositories under sites/ndpf/repository/. Addition of new directories allows to override namespaces that are defined somewhere else in the template hierarchy.

grid

The grid tree contains service configuration and software needed for the various grid services and machine types. The current configuration has the following subtrees:

+- common
+- glite-3.0
+- glite-3.1
+- machine-types
+- users
+- vle
+- vo

Under common, various generic (non-grid) services are present, such as openldap and nameserver configuration, but also Torque and Nagios.

The glite-x.y trees contain everything needed by the gLite services in terms of software packages and configuration. Because multiple gLite releases cannot coexist on the same machine, the trees are separate. They are typically added to the LOADPATH variable on a per host basis.

In machine-types the various functions of a physical machine are described. The machine type templates are directly included by an object template, and are specialized for one (set of) service(s) only. Warning: inclusion of more than one machine type from an object profile will most likely result in a misconfigured machine!

Directory users contains the user definitions.

The tree vle contains the software and configuration templates for the releases of the VL-e project ([www.vl-e.nl]). The VL-e releases may coexist on the same node.

The configuration of the various virtual organizations is contained in the tree vo.

os

The os include root contains subtrees that are named by the version of the operating system and the architecture:

+- centos3_i386
+- centos4_i386
+- centos5_x86_64

The names are free format. One of these directories is selected via the LOADPATH for inclusion.

Typically, an OS/architecture-specific subtree contains the following:

+- config
   +- os
   +- quattor
+- glite    
+- os
+- repository
+- rpms

Under config the configuration specific for the OS, such as kernel version or Quattor's AII is defined. The glite branch allows for OS-specific patches to gLite dependencies. The os tree contains the templates corresponding to the package groups for the OS distribution. In repository, the repository templates to be used are selected (note that there are only references here; the actual contents are defined in sites/ndpf/repository. The rpms tree contains the updates to the OS packages.

standard

The standard tree contains the templates that are directly related to Pan and Quattor. Some entries are symbolic links to templates installed by Quattor component rpms, other entries define the Quattor software to install (per OS) or some generic Quattor configuration.

sites

At the moment, there is only one site defined: ndpf. The common configuration and local functions for the site are collected under sites/ndpf:

+- common
+- functions
+- hardware
+- repositories
+- site

common contains generic configuration for a service; this may be removed in the future. Directory functions contains some site-specific utility functions used by other templates. The hardware definitions are present under the hardware tree. In repositories, per software repository there is a generated entry (not in CVS!) that contains all packages in that repository location. The directory site contains some generic configuration for the site.

Namespaces

Quattor release 1.3 uses the concept of namespaces for organizing templates. The name of template in a namespace corresponds to its parent directory (or directories) that contain it with respect to the include root. An example: assume the include root for the Pan compiler is $I, then a template with path $I/my/namespaced/template.tpl should have the name my/namespaced/template in Pan:

template my/namespaced/template;

# contents

Combining namespaced templates with the Pan feature called loadpath gives tremendous flexibility in defining various template tree parallel to each other with the possibility to select from a template which of the trees to use for compilation. The loadpath is set by built-in Pan variable LOADPATH, which is a list strings to consider to resolve a requested template.

A concrete example is the selection of the operating system. Assume the file system hierarchy under the include root $I is as follows:

$I
   +- centos3_i386
      +- some
         +- template
   +- centos4_i386
      +- some
         +- template

Then it is possible to

include some/template;

from other templates if either centos3_i386 or centos4_i386 was added to the Pan variable LOADPATH.