Koji Testbed

The Koji setup on the testbed

The Koji system is used by Fedora for builds of packages for Fedora Core and EPEL. It is open source and available as a standard package, and documentation for running a private installation is available.

Koji consists of a number of interrelated components:

• Koji Hub, the central XMLRPC service.
• A postgresql database as a backend for the hub
• Koji Web, a user interface front-end for the hub (other interaction is by the command-line client)
• Kojira, managing repository building and clean-up.
• One or more builders, on separate machines.

It is possible to install most of these components on separate machines, but our simple setup has only two:

• koji-hub.testbed, holding the hub, web, kojira and database
• koji-builder.testbed, for building i386 and x86_64 packages. We could install more builders if needs be.

Both machines share access to a large NFS share on put.testbed:/mnt/put/koji for storing results and buildroots. Currently both machines are deployed on the M610 blades (blade13 and blade14) with the possibility for hot migration.

The installation mostly followed the very complete guidance on [1], choosing the SSL authentication option.

SSL Authentication

The authentication system is picky when it comes to Common Names. For users, they have to be the same as user ids, and for servers (only koji-hub.testbed, really) it has to be the fully qualified host name.

A new CA was created for the purpose of handing out certificates (as reuse of existing certificates was hard because of the above restriction, but we may still find a way around that sometime). The CA installation is currently found on bleek.testbed:/srv/koji-pki.

For access to koji-web, the user needs to export the certificate file from there to a pkcs12 file and import it in the browser. For command-line koji use, put it in ~/.koji/clientcert.crt.

Building from git

This step still needs to be automated with a commit hook. The git repository on git://bleek.testbed/koji.git has one subdirectory per component, and each directory contains:

• a SPEC file
• a Makefile with a 'sources' target, which will fetch (with curl) the sources for the package.

After pushing the updates to this repository, record the commit hash and start the koji build like so (replace the hash with yours):

koji build dist-epel5 'git://bleek.testbed/koji.git?lcmaps/#7a2b193fdd8a26875a16303da609952cccc4cba8'

The format is particular to koji; it is parsed as an URL and that is why the ? and # are used to mark certain elements.

Adding tags, targets and repositories

See the main documentation on the Fedora wiki.

The builds are done against targets, which have source and target tags, which have buildroots populated from repositories. The initial buildroots are set up from virtual packages called 'build' and 'srpm-build' created in koji.

A dump of the history to see what steps are required

  99  koji add-tag dist-epel5
 100  koji add-tag --parent dist-epel5 --arches "i386 x86_64" dist-epel5-build
 102  koji add-external-repo -t dist-epel5-build dist-epel5-external-repo http://mirrors.nl.eu.kernel.org/fedora-epel/5/\$arch/
 103  koji add-external-repo -t dist-epel5-build dist-epel5-centos5-repo http://spiegel.nikhef.nl/mirror/centos/5/os/\$arch/
 104  koji add-target dist-epel5 dist-epel5-build

These groups are going to be passed to 'yum groupinstall' when the buildroot is created.

 106  koji add-group dist-epel5-build build
 107  koji add-group dist-epel5-build srpm-build

They need to be populated

 109  koji -s http://koji.fedoraproject.org/koji list-groups dist-epel5-build
 110  koji -s https://koji.fedoraproject.org/koji list-groups dist-epel5-build

Clone from a typical value for the Fedora Koji systems.

 112  koji add-group-pkg dist-epel5-build build `cat epel5-build-pkgs `
 114  koji add-group-pkg dist-epel5-build srpm-build `cat epel5-srpm-build-pkgs `

Don't forget to add the package!

 140  koji add-pkg --owner okoeroo dist-epel5 lcmaps
 124  koji build dist-epel5 git://bleek.testbed/koji.git\?lcmaps/#a942e490ace1da950cedae2976bf59fa3b3f5038

The git repository must be added to the allowed repositories on the builder in kojid.conf.