Namespaced Quattor configuration

Setup

The current Quattor 1.3 setup is entirely different from the Quattor 1.2 setup. It is therefore located in a different tree in CVS: nl.nikhef.ndpf.quattor-config/conf-ns/. This tree contains the templates and the infrastructure needed to maintain, build and deploy templates. Various tools assume that there is an evironment variable $L that points to the location where the contents of this tree in CVS are present.

The build and deploy infrastructure is derived from SCDB: [1]

The major difference to standard SCDB is that our setup is independent of SubVersion, whereas SCDB relies on the presence of a SubVersion repository. The setup is still built around an ant build script build.xml, which governs various tasks.

Building and deploying templates is easiest with the shell scripts makexprof and pushxprof, respectively. Note that the latter is a symlink to the former. Syntax:

makexprof -f <facility-name> [template1] [template2] [...]
pushxprof -f <facility-name> [template1] [template2] [...]

The option -f with argument <facility-name> is required to select one of the facility names. Currently defined facilities: itb (Installation Test Bed), prd (Production cluster) and generic (miscellaneous Quattor-managed servers).

Updating the templates that describe the contents of the package repositories is done by directly executing the task update.rep.templates in the Ant script:

( cd $L ; ant update.rep.templates )

To generate update templates (i.e., templates that replace older version of software), the tool $L/../bin/rpmUpdates.pl can be used:

rpmUpdates.pl /path/to/update/rpm/directory  > updates.tpl

creates an update template (prepared for namespace rpms/updates) based on the contents of /path/to/update/rpm/directory.

Templates

Work In Progress

The current Quattor templates use namespaces (see below).

The templates are all located under the directory $L/cfg. The Ant build file (build.xml) requires the presence of the directory facility/<facility-name>. In this directory, a file cluster.pan.includes may exist that specifies which include roots should be used when compiling the templates. For example:

cluster.pan.includes=sites/ndpf grid os standard

specifies that under directory $L/cfg, the include roots sites/ndpf, grid, os and standard are considered in the specified order. If two or more of the include roots contain the namespaced template, only the first one found is taken into account. The include roots specified are appended to facility/<facility-name>, which is searched recursively for all templates (also allowing for namespaced templates). Note that the above specification only includes everything directly in the listed directories and does not recursively descend into all subtrees, unlike for facility/<facility-name>. Recursively including all files under somedir is possible by specifying somedir/**/*.

The setup is based on the concept of facilities (comparable to cluster in SCDB). A facility is currently a collection of grid hosts that act as a standalone grid cluster with similar (grid) configuration. One or more grid clusters are part of a site, which has some common properties among the facilities it contains. The distinction between facility and site is not strict and may be changed in the future; in principle any kind of grouping is possible.

The above concept has led to the currently defined include roots:

facility/<facility-name>
grid
os
standard
sites

Directory facility/<facility-name> contains the information for a specific grouping of machines. Per machine there has to be an object template in the profiles subdirectory. The subdirectory local contains miscellaneous configuration templates for the hosts in the facility. In repositories, the package repositories that are used to resolve requested packages are defined. Addition of new directories allows to override namespaces that are defined somewhere else in the template hierarchy.

<more to come>.

Namespaces

Quattor release 1.3 uses the concept of namespaces for organizing templates. The name of template in a namespace corresponds to its parent directory (or directories) that contain it with respect to the include root. An example: assume the include root for the Pan compiler is $I, then a template with path $I/my/namespaced/template.tpl should have the name my/namespaced/template in Pan:

template my/namespaced/template;

# contents

Combining namespaced templates with the Pan feature called loadpath gives tremendous flexibility in defining various template tree parallel to each other with the possibility to select from a template which of the trees to use for compilation. The loadpath is set by built-in Pan variable LOADPATH, which is a list strings to consider to resolve a requested template.

A concrete example is the selection of the operating system. Assume the file system hierarchy under the include root $I is as follows:

$I
   +- centos3_i386
      +- some
         +- template
   +- centos4_i386
      +- some
         +- template

Then it is possible to

include some/template;

from other templates if either centos3_i386 or centos4_i386 was added to the Pan variable LOADPATH.