Quattor package management with yum-based ncm-spma
The update of ncm-spma, Quattor's package management component, to a version that uses yum instead SPMA and rpmt-py, has significantly simplified the configuration of package lists, changed the requirements on software repositories and changed the workflow of updating software.
The major advantage of the new component lies in the simplification of the software that needs to be installed. Where the original version required every package and all of its dependent packages to be exactly specified, often referred to as the RPM dependency hell, the new component only needs the packages that are desired (version can be specified if required, but that is not needed). The dependencies are handled automatically by yum, just as yum will take care of the selection of the optimal or most recent version of each package. Consequently, there will be less need for modifications in the software installation templates to upgrade and the trial-and-error approach to satisfy all dependencies will become an unpleasant relic of the past. Furthermore, since package versions do not need to be specified exactly any more, the painful process to generate template for all package groups when a a new minor version of the distribution is released should also become simpler.
There is a modest price to pay for the simplification of the software templates: part of the complexity goes into maintaining the software repositories and one loses the compile-time verification of the presence of the packages to be installed in the available repositories. The latter may lead to unexpected problems when deploying.
One of Quattor's was the ability to roll back an installation or configuration to a previous state. Performing a roll back to an earlier set of packages is not trivial when using yum. Nevertheless, it is very desirable not to lose this feature. By using snapshots of software repositories, it is possible to let yum consider an earlier set of packages when managing the installation, implying that downgrading to previous versions is required.
This article gives a description of the new situation as implemented at Nikhef.
Repository Management
Repositories are managed on the Quattor installation server stal.
There are two new aspects on repository management:
- Repositories have to be populated with care
- Snapshotting of repositories to enable rolling back
Populating Repositories with Packages
Before using yum, a Quattor software repository was simply a web page where a collection of packages was offered. The collection could consist of a mix of packages from different operating system version or architectures. It was the administrator's responsibility to select a working mix of packages that could be found in the available repositories.
It is not so simple when using yum.
Yum will try to select the most recent version of a package. If two versions of the same package, intended for different distributions, exist in a repository, yum will always select the most recent, i.e., the lexically largest file name. For example, if a repository contains mypackage-1.0.el5.noarch.rpm and mypackage-1.0.el6.noarch.rpm, installing mypackage will always lead to selection of mypackage-1.0.el6.noarch.rpm, irrespective of the platform used. That can lead to very, very wrong dependency resolutions!
To prevent such problems, the repositories must be organized per distribution and optionally per architecture and may only contain packages corresponding to (i.e., built for) the related distribution and architecture. The following example presents the repositories for locally developed tools which independent of the hardware architecture:
tools/ +-- el5 +-- el6
To add new (versions of) packages to a repository, the above layout must be taken into consideration. Furthermore, new packages should be added to the repository mirror. The repository mirror is a dedicated directory (location) which acts as the source from which repository snapshots (see next section) are derived. This directory will contain all packages that are available within the context of the repository. The actual selection is made in the snapshots of the repository.
At the Quattor install server stal, the mirror directory is /project/mirror.
Snapshotting Repositories
To enable restoring a previous set of package versions, snapshots of software repositories are kept on the Quattor installation server. Per repository mirror location, there are one or more snapshots. Each snapshot location contains hard links to the packages present in the mirror location on a given moment. The snapshot location is converted into a yum repository by generating the repodata index files.
By convention, the repository snapshots are stored on the Quattor installation server under directory /project/repositories. The snapshot directory hierarchy is included in the periodic rsync-synchronisation to the slave servers behind alias stalkaars.
After adding a new package to a mirror, a new snapshots can be made with the script makesnapshot:
makesnapshot [options] <repository> <label>
The script will create a snapshot of the actual contents of the mirror repository. The snapshot will be named <label>. The details of the snapshot creation are described per repository a configuration file in directory /project/quattor/etc/repository-snapshots/. This configuration file at least contains the source for the snapshot (by convention the mirror location) and the destination (by convention the snapshot base directory). A list of arguments for rsync can be provided.
For example, creating a snapshot with command makesnapshot nikhef-tools-el5 20130930 where the configuration file /project/quattor/etc/repository-snapshots/nikhef-tools-el5.conf is:
source = /project/mirror/nikhef/tools/el5 destination = /project/repositories/nikhef/tools/el5 arguments = --exclude nikhef-yaim-core-1.0.1-1.noarch.rpm
will create a new snapshot under directory /project/repositories/nikhef/tools/el5/20130930 with the actual contents of /project/mirror/nikhef/tools/el5, ignoring the presence of the package nikhef-yaim-core-1.0.1-1.noarch.rpm.
Package Selection
The selection of packages to be installed on a particular host is defined in the Quattor templates. Before using the yum-based SPMA, all packages had to be specified exactly, providing the name, version and architecture(s). Furthermore, SPMA required all dependencies of each package to be included in the list of packages to be installed.
With the new SPMA, this has become much more simple. Not only will yum select the most recent version of a package, assuming that a package's dependencies were correctly specified at build time, yum will include all dependencies in the installation transaction.
Repository Snapshot Selection
Documentation from the Quattor website:
