Difference between revisions of "SAC Debian packaging"

From PDP/Grid Wiki
Jump to navigationJump to search
 
(49 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 +
<sidebar>
 +
* navigation
 +
** Main Page|Main Page
 +
** SAC software procedures|Software Procedures
 +
** GLExec|SAC software for users
 +
</sidebar>
 +
 
The following procedures extend the [[SAC software procedures]] with respect to the creation and distribution of Debian and Ubuntu packages.
 
The following procedures extend the [[SAC software procedures]] with respect to the creation and distribution of Debian and Ubuntu packages.
  
 
== packaging a new release of an existing component ==
 
== packaging a new release of an existing component ==
 +
 +
When a component has been updated and released, it should also be repackaged for Debian at some point. The following steps outline what needs to be done.
 +
 +
=== Update the Debian source package ===
 +
 +
Source packages are [https://ndpfsvn.nikhef.nl/cgi-bin/viewvc.cgi/mwsec/packaging/debian/trunk/ maintained in SVN]. Check out the component and run uscan to fetch the upstream tarball.
 +
 +
cd $mwsec/packaging/debian/trunk/$component
 +
svn update
 +
uscan --report # optional
 +
uscan --destdir ../tarballs
 +
 +
Update debian/changelog.
 +
dch -v $version-1 -D UNRELEASED "New upstream release"
 +
 +
=== Review and test the packaging ===
 +
 +
Check the upstream changelog if there is anything that may affect the build. Changed paths, modified shared libraries, etc. Test a local build:
 +
svn-buildpackage --svn-ignore-new -us -uc
 +
debc ../build-area/${component}_${version}-1_${arch}.changes | less
 +
lintian -IiE --pedantic ../build-area/${component}_${version}-1_${arch}.changes
 +
 +
When you're happy commit to svn.
 +
svn commit
 +
 +
=== Prepare the source package for uploading ===
 +
 +
When the packaging is in good shape, it is time to prepare a Debian source package (.dsc). This will be used in later steps for building for different distributions.
 +
svn-buildpackage -S -sa
  
 
== releasing packages for multiple distributions ==
 
== releasing packages for multiple distributions ==
 +
 +
The Debian packages can be uploaded to several places:
 +
* [http://www.debian.org/releases/sid/ Debian unstable] (via [http://mentors.debian.net/ Debian mentors])
 +
* [https://help.launchpad.net/Packaging/PPA Ubuntu PPA] (personal package archive)
 +
* http://software.nikhef.nl/dist/debian/
 +
 +
The normal lifecycle of a package is that it starts in Debian unstable. From there it will (eventually) reach Debian stable and Ubuntu. However, since this takes (much) time, we make our packages available for users of several distributions; this is called backporting. The backport targets are given in the table below.
 +
 +
The target distribution is set in debian/changelog. We have to repackage the source package to set the distribution ''and'' the release number, which is coding the backport.
 +
 +
The first Debian release of a package always has ''1'' as the release number, so upstream version 1.2.3 becomes 1.2.3-1. If the Debian packaging for the same upstream version is updated, the release number increments. The table below indicates what the release numbers should be for various backports. The use of the '~' is special: when comparing versions the tilde ''precedes'' any other symbol ''including the empty space'', e.g. 1.2.3-1 is considered newer than 1.2.3-1~backport. This is a very useful trick to make sure that the distribution's own package of the same version is preferred over a backport. The number after the backport codename is another sequence number that increments with every backport-specific rebuild of the same upstream-version-and-debian-release.
 +
 +
{| class="wikitable" border="0" cellpadding="8"
 +
! distribution
 +
! version
 +
! code name
 +
! release tag
 +
! notes
 +
|-
 +
| Debian
 +
| unstable
 +
| sid
 +
| 1
 +
| Sid (Debian unstable) is not a backport.
 +
|-
 +
|
 +
| 6
 +
| squeeze
 +
| 1~bpo60+1
 +
| This is the [http://backports-master.debian.org/Contribute/ Debian style for backports].
 +
|-
 +
| Ubuntu
 +
| 8.04 LTS
 +
| hardy
 +
| 1~hardy1
 +
| Hardy is difficult, because the debhelper package is too old (see <code>man dh</code>).
 +
|-
 +
|
 +
| 10.04 LTS
 +
| lucid
 +
| 1~lucid1
 +
| The Ubuntu style for backporting is slightly simpler.
 +
|-
 +
|
 +
| 10.10
 +
| maverick
 +
| 1~maverick1
 +
|-
 +
|
 +
| 11.04
 +
| natty
 +
| 1~natty1
 +
|-
 +
|
 +
| 11.10
 +
| oneiric
 +
| 1~oneiric1
 +
|}
 +
 +
=== Preparing the source packages for backporting ===
 +
 +
The following script extracts the given UNRELEASED source packages (the output of the first step), updates the changelog, and repackages so they can be build by cowbuilder or cowpoke.
 +
cat > prepare-backports.sh <<'EOF'
 +
#!/bin/sh
 +
 +
# prepare backports for packages
 +
# arguments: list of .dsc files
 +
 +
# action: for each dsc file and for each distribution:
 +
# unpack the dsc file
 +
# dch -v $version-release~$backportcode --distribution $dist "Rebuild for xxx"
 +
 +
# arguments: source package, dist (directory name), target distribution, release code
 +
makesrc() {
 +
    cd $2
 +
    dpkg-source -x ../$1
 +
    pkg=`basename $1`
 +
    cd `echo $pkg | sed -e 's/\([^_]*\)_\([^-]*\).*/\1-\2/'`
 +
    version=`dpkg-parsechangelog | sed -n 's/^Version: //p'`
 +
    dch --distribution $3 -b -v ${version}~${4} "Rebuild for $2"
 +
    dpkg-buildpackage -S -sa
 +
    cd ../..
 +
 +
}
 +
 +
ubuntu="hardy lucid maverick natty oneiric"
 +
mkdir $ubuntu squeeze sid
 +
 +
while [ $# -ge 1 ]; do
 +
    for dist in $ubuntu; do
 +
makesrc $1 $dist $dist-backports ${dist}1
 +
    done
 +
    # similar but different for debian
 +
    makesrc $1 squeeze squeeze bpo60+1
 +
 +
    # treat sid specially
 +
    cd sid
 +
    dpkg-source -x ../$1
 +
    pkg=`basename $1`
 +
    cd `echo $pkg | sed -e 's/\([^_]*\)_\([^-]*\).*/\1-\2/'`
 +
    dch --distribution unstable --release ""
 +
    dpkg-buildpackage -S -sa
 +
    cd ../..
 +
 +
    shift
 +
done
 +
EOF
 +
Running this script unpacks and repacks the sources multiple times in various subdirectories. After building with cowbuilder or cowpoke (cowbuilder on a remote machine) it is no longer required to keep them around.
 +
 +
=== Starting the remote build for a backport ===
 +
 +
The reference machine ref-debian6-64.testbed has been configured for cowbuilder for all distributions, in the architectures i386 and amd64. The cowpoke program uploads the sources and launches a remote build, downloads the results and signs the changes files after the build. (See the configuration section on how to set up cowpoke.)
 +
 +
( cd sid
 +
  cowpoke --dist=sid --arch=amd64 ${component}_{version}-1.dsc
 +
  cowpoke --dist=sid --arch=i386 --dpkg-opts=-B ${component}_{version}-1.dsc )
 +
( cd squeeze
 +
  cowpoke --dist=squeeze --arch=amd64 ${component}_{version}-1~bpo60+1.dsc
 +
  cowpoke --dist=squeeze --arch=i386 --dpkg-opts=-B ${component}_{version}-1~bpo60+1.dsc
 +
)
 +
 +
The other builds follow the same pattern. '''Note:''' the -B option only builds the architecture-specific packages. This prevents regenerating the source package and architecture-independent packages, which are already produced by the amd64 build. (The ordering is not specific, but if both builds produce the same packages we'll get in trouble later on when uploading to the repository. It refuses to accept packages that it already has, and on top of that the checksums will be different.)
 +
 +
=== Uploading the builds ===
 +
 +
The build results are found under ref-debian6-64.testbed:/var/cache/pbuilder/$dist-$arch/results.
 +
The dput command is used to perform the uploading (see the configuration section how to set up dput).
 +
 +
==== local repository ====
 +
 +
dput mwsec-squeeze /var/cache/pbuilder/squeeze-amd64/results/${component}_${version}-1~bpo60+1_amd64.changes
 +
dput mwsec-squeeze /var/cache/pbuilder/squeeze-i386/results/${component}_${version}-1~bpo60+1_i386.changes
 +
# similar for other backports
 +
rsync -a -vP --delete /var/local/debian-repository/debian/ software.nikhef.nl:/project/srv/www/site/software/html/dist/debian/
 +
 +
==== Debian mentors ====
 +
 +
In order to get the packages in the main Debian distribution, they need to be uploaded to Debian mentors. Subsequently a sponsor must be found to upload the packages to unstable. Always upload the unstable packages to mentors.
 +
 +
dput debexpo /var/cach/pbuilder/sid-amd64/result/${component}_${version}-1_amd64.changes
 +
 +
There is no need to upload for all architectures as Debian will rebuild the packages anyway.
 +
 +
==== Ubuntu ppa ====
 +
 +
There is no ppa configured for mwsec at the moment.
  
 
== Setting up a personal build environment ==
 
== Setting up a personal build environment ==
Line 15: Line 197:
 
=== Environment variables ===
 
=== Environment variables ===
  
−
Many of the tools source ~/.devscripts for environment variables. Of course they may be put in ~/.profile as well. The following variables are relevant:
+
The [http://www.debian.org/doc/manuals/maint-guide/first.en.html#dh-make Debian new maintainers guide] suggests to put the following variables in ${HOME}/.bashrc:
−
* DEBSIGN_KEYID, this should be the identifier of your GPG key. (You can find it by typing gpg --list-secret-keys; e.g. if the output lists 4096R/8AAE244E then 8AAE244E is the key id.)
+
export DEBEMAIL=youremail@nikhef.nl
−
* DEBEMAIL, your e-mail address. This must match the one in your gpg info.
+
export DEBFULLNAME="John D. Developer"
−
* DEBFULLNAME, your full name (exactly as listed in the gpg info)
+
You must use values that exactly match your GPG key identity.
 +
 
 +
For using the debsign tool later on (for signing packages) put these settings in ${HOME}/.devscripts:
 +
DEBSIGN_KEYID=8AAE244E
 +
but fill in the key id of your own GPG key. (You can find it by typing gpg --list-secret-keys.)
  
 
=== Machine environment ===
 
=== Machine environment ===
  
−
Development should be done on either a Debian or Ubuntu machine.
+
Development should be done on either a Debian or Ubuntu machine. Root privileges are not required, but it is necessary that a certain number of developer's tools are installed (besides the build dependencies of the packages you work on, obviously). The [http://www.debian.org/doc/manuals/maint-guide/start.en.html#needprogs New Maintainers' Guide] suggests several packages.
  
−
== packaging a new component for the first time ==
+
apt-get install build-essentials dh-make devscripts autotools-dev cdbs debhelper fakeroot lintian pbuilder \
 +
                cowbuilder quilt
  
−
== system configurations ==
+
Take care to install recent versions of at least lintian, devscripts and debhelper. Older distributions may not have the required versions, in which case you should consider installing a backport of the right versions.
  
−
=== cowbuilder ===
+
Setting up pbuilder and cowbuilder is rather advanced, but very worthwhile to do clean-room installations.
  
−
=== reprepro ===
+
=== Working directory layout ===
  
−
== Anatomy of a Debian source package ==
+
You are almost done. The last thing to do is setting up the work area for doing the packaging.
  
−
This material is provided to help understand how to package for Debian. The most important thing is to familiarize yourself with the available material for Debian maintainers and developers, but as this can be a bit overwhelming, these notes are somewhat tailored to our situation and may help to clarify.
+
All of the packaging of MWSEC is done in [https://ndpfsvn.nikhef.nl/cgi-bin/viewvc.cgi/mwsec/packaging/debian/trunk/ SVN]. However, to keep the repository clean, the original sources are not included. The Debian way of packaging is based on the original sources plus a debian subdirectory, so our separated setup means that things will have to be merged before actual building can take place.
  
−
== older material ==
+
Fortunately, this mode of operation is supported by the tool svn-buildpackage. If the property svn:mergeWithUpstream is set on the debian directory, the tool will understand that it must make the merge before commencing the build. In this setup, all the original tarballs are put in ../tarballs and the merge is done in ../build-area.
  
−
TODO move these to the relevant sections above
+
svn checkout svn+ssh://svn@ndpfsvn.nikhef.nl/repos/mwsec/packaging/debian/trunk
 +
cd trunk
 +
mkdir tarballs
 +
mkdir build-area
  
−
Debian packaging requires a <tt>debian/</tt> subdirectory in the root of the upstream source directory. The structure of this directory is described in the [http://www.debian.org/doc/manuals/maint-guide/ Debian New Maintainer's Guide]. For the SAC middleware, the debian packaging is [https://ndpfsvn.nikhef.nl/cgi-bin/viewvc.cgi/mwsec/packaging/debian/trunk/ maintained in SVN]. Debian does not prescribe the exact method of creating packages, as is the case with RPM, and several helper libraries have emerged over time. The most notable are [https://en.wikipedia.org/wiki/Debhelper debhelper] and the newer [http://build-common.alioth.debian.org/cdbs-doc.html CDBS]. Both should make it a snap to build packages with.
+
The locations of tarballs and build-area is configurable. These settings in ${HOME}/.svn-buildpackage.conf are only a suggestion.
 +
 +
svn-override=origDir=$HOME/debian/upstream,buildarea=$HOME/debian/build-area
  
 +
== packaging a new component for the first time ==
  
−
The following steps are required only once for every source packaged to be debianized. All of this could be done manually, but it is a lot of hassle and prone to errors, so better leave it to the automated tools. Should any package decide to radically change it's nature (hopefully this will never happen!) it may be required to repeat these steps.
+
The terse version:
−
 
 
−
'''Required System: Debian/Ubuntu. Required tools: dh_make.'''
 
  
 
* Unpack the upstream sources.
 
* Unpack the upstream sources.
−
* Run <code>dh_make -r cdbs</code> (and answer some questions).
+
* Run <code>dh_make -c apache</code> (and answer some questions).
 
* Review the contents of the <tt>debian/</tt> subdirectory.
 
* Review the contents of the <tt>debian/</tt> subdirectory.
 
* Create a subdirectory in SVN under mwsec/packaging/debian/trunk for the new package.
 
* Create a subdirectory in SVN under mwsec/packaging/debian/trunk for the new package.
Line 54: Line 245:
 
* set the svn:MergeWithUpstream property.
 
* set the svn:MergeWithUpstream property.
  
−
(TODO: review and verify this part!)
+
The longer version.
 +
 
 +
Download the new component from http://software.nikhef.nl/security/$component. Unpack it and run dh_make.
 +
dh_make -c apache -f ../$component-$version.tar.gz
 +
 
 +
Dh_make asks a couple of questions and then presents a lot of template files in the debian/ directory. Many of these are probably irrelevant and should be deleted.
 +
 
 +
Things that need to be filled in in various places:
 +
{| class="wikitable"
 +
|-
 +
| debian/control
 +
| Homepage
 +
| https://wiki.nikhef.nl/grid/Site_Access_Control
 +
|-
 +
| debian/control
 +
| Upstream Author
 +
| Nikhef Grid Middleware Security Team <grid-mw-security@nikhef.nl>
 +
|-
 +
| debian/control
 +
| Vcs-Svn
 +
| http://ndpfsvn.nikhef.nl/ro/mwsec/packaging/debian/trunk/''$component''
 +
|-
 +
| debian/control
 +
| Vcs-Browser
 +
| http://ndpfsvn.nikhef.nl/cgi-bin/viewvc.cgi/mwsec/packaging/debian/trunk/''$component''
 +
 
 +
|}
 +
 
 +
For guidance, read the [http://www.debian.org/doc/manuals/maint-guide/ Debian New Maintainers' Guide].
  
−
=== Setting up pbuilder for building Debian packages ===
+
Try to build the package and use tools like dpkg-depcheck, debc and lintian to figure out what needs to be done. If you are reasonably satisfied it is time to incorporate the work in SVN.
  
−
Debian's pbuilder is a system for building packages in a clean environment. It can use chroot or copy-on-write. In order to set it up so it can build for multiple distributions, some [http://wiki.debian.org/PbuilderTricks enhancements to the configuration] need to be applied.
+
The packaging of Nikhef Grid Middleware is maintained in [https://ndpfsvn.nikhef.nl/cgi-bin/viewvc.cgi/mwsec/packaging/debian/trunk/ SVN]. Check out svn+ssh://svn@ndpfsvn.nikhef.nl/repos/mwsec/packaging/debian/trunk and create a new directory named after the new component, but without a version number. Inside this directory, create a directory named 'debian'.
 +
svn co svn+ssh://svn@ndpfsvn.nikhef.nl/repos/mwsec/packaging/debian/trunk
 +
cd trunk
 +
mkdir -p $component/debian
 +
svn add $component
 +
svn propset svn:mergeWithUpstream 1 $component/debian
 +
Now copy the files from the prepared source package's debian directory to the debian directory in SVN. This way you don't accidentally copy anything in there that you will have to remove later on anyway. Commit the changes.
 +
Building the package must now be done with svn-buildpackage. Because the original sources and the debian directory are separated, they must first be reintegrated and that is what the svn:mergeWithUpstream setting is for. The original tarball is expected in the ../tarballs directory, and the actual building is going to be rooted in ../build-area. After setting this up, try to build the package:
 +
svn-buildpackage -us -uc
  
−
Because pbuilder needs to be run as root, users are allowed to run /usr/sbin/pbuilder through <tt>sudo</tt>.
+
=== Copyright ===
  
−
The main installation is on ref-debian6-64.testbed.
+
It's important to get the copyright statements correct. For most of the middleware it should be something like this:
 +
Copyright: 2004-2010 Members of the EGEE Collaboration
 +
            2010-2012 Members of EMI collaboration
 +
The dates may vary. For the Debian packaging itself, the copyright should read:
 +
Copyright: 2012 Stichting FOM <grid-mw-security@nikhef.nl>
  
−
FIXME: the new machine has things set up slightly differently than before. We're moving to cowbuilder + cowpoke...
+
=== Patching ===
  
−
To accommodate Ubuntu builds, the ubuntu-archive-keyring package was installed manually from the Ubuntu archives. Otherwise debootstrap will fail.
+
Sometimes a package needs to be patched in order to make it work for Debian. Since we are our own upstream providers, fixes are best put in the original sources in a new release. Occasionally it might concern something that is simply too Debian specific to handle upstream. In those cases,
 +
quilt should be used as a patch system.
  
−
To build a package, it should suffice to go to the SVN directory <tt>mwsec/packaging/debian/trunk/''component''/</tt> and run:
+
It's important that this must be done on the unpacked and merged sources (use svn-buildpackage --svn-export to do the merge but no build). The command sequence is
−
DIST=stable svn-buildpackage --svn-builder pdebuild --pbuilder cowbuilder
+
quilt new fix-this-urgently-for-debian.patch
 +
quilt add the-file-that-needs-fixing
 +
(edit the file as needed)
 +
(possibly repeat the add and edit steps for other files)
 +
quilt refresh
 +
This should result in a patch file under debian/patches/ as well as a series file. Copy the whole thing to the SVN work area. It should be automatically picked up the next time you run svn-buildpackage.
 +
There is a proposal to make [http://dep.debian.net/deps/dep3/ annotations to the patches] a requirement. At the moment it is highly recommended.
  
−
The ''cowbuilder'' will use the copy-on-write root in /var/cache/pbuilder/stable-amd64/base.cow. For other distributions just change <tt>DIST=stable</tt>, e.g. <tt>DIST=oneiric</tt> for the 11.10 Ubuntu release.
+
== Configuration ==
 +
 
 +
=== pbuilder/cowbuilder ===
 +
 
 +
Pbuilder is a system to build packages in a clean, chroot environment. It can use a base.tgz tarball, but the slightly more efficient way is to do 'copy on write' a.k.a. COW with cowbuilder. There are [https://wiki.ubuntu.com/PbuilderHowto excellent] [http://wiki.debian.org/cowbuilder documents] on how to set this up.
 +
 
 +
The cowbuilder environment is already configured on ref-debian6-64.testbed; this documentation exists to reproduce the setup in case you need a personal cowbuilder environment.
 +
 
 +
Both pbuilder and cowbuilder require root privileges, because they need to do <code>chroot()</code>. The most convenient way is by configuring sudo. User visudo and add something along these lines:
 +
Cmnd_Alias PBUILDERS =  /usr/sbin/pbuilder, /usr/sbin/cowbuilder
 +
johnd  ALL= NOPASSWD: SETENV: PBUILDERS
 +
 
 +
Following the guidance at the [http://wiki.debian.org/cowbuilder#Building_your_package_for_many_distributions_at_once Debian cowbuilder wiki], create a ${HOME}/.pbuilderrc based on the [https://wiki.ubuntu.com/PbuilderHowto Ubuntu Pbuilder HOWTO], and uncomment
 +
BASEPATH="/var/cache/pbuilder/$NAME/base.cow/
 +
 
 +
You should make a separate cowbuilder environment for every distribution you need to make builds for.
 +
 
 +
sudo cowbuilder \
 +
--create \
 +
--basepath /var/cache/pbuilder/squeeze-i386/base.cow\
 +
  --distribution squeeze\
 +
  --debootstrapopts --arch --debootstrapopts i386
 +
 
 +
('''Update:''' this command didn't work on ref-debian6-64.testbed for precise; cowbuilder runs 'pbuilder create' with a wrong mirror (Debian instead of Ubuntu). The fix is to run pbuilder manually like so):
 +
sudo pbuilder create --debootstrapopts --arch --debootstrapopts i386 --debootstrapopts \
 +
--keyring --debootstrapopts /usr/share/keyrings/ubuntu-archive-keyring.gpg --buildplace \
 +
/var/cache/pbuilder/precise-i386/base.cow --mirror http://archive.ubuntu.com/ubuntu/ --architecture i386 \
 +
--distribution precise --no-targz --extrapackages cowdancer
 +
 +
Inside the cowbuilder environment, add our own software repository to the /etc/apt/sources.list in order to resolve build dependencies on our own software. Add the repositories GPG key to apt as well.
 +
sudo ARCH=i386 DIST=squeeze cowbuilder --architecture=i386 --login --save-after-login
 +
cat >> /etc/apt/sources.list <<EOF
 +
deb http://software.nikhef.nl/dist/debian/ squeeze main
 +
EOF
 +
wget -O- http://software.nikhef.nl/dist/debian/DEB-GPG-KEY-MWSEC.asc | apt-key add -
 +
apt-get update
 +
 
 +
[http://wiki.debian.org/Hardening Debian now requires the use of hardening flags in the compiler]. For the MWsec middleware this requires the use of a recent dpkg-dev package; at least version 1.16. Since this is not available
 +
for squeeze, we need to install it manually from the [http://backports-master.debian.org/Instructions/ squeeze-backports repository].
 +
So in addition to the above steps, also do the following:
 +
cat >> /etc/apt/sources.list <<EOF
 +
deb http://backports.debian.org/debian-backports squeeze-backports main
 +
EOF
 +
apt-get update
 +
apt-get -t squeeze-backports install dpkg-dev
 +
 
 +
To build a package, it should suffice to go to the SVN directory <tt>mwsec/packaging/debian/trunk/''$component''/</tt> and run:
 +
DIST=squeeze svn-buildpackage --svn-builder pdebuild --pbuilder cowbuilder
 +
 
 +
The ''cowbuilder'' will use the copy-on-write root in /var/cache/pbuilder/squeeze-amd64/base.cow. For other distributions just change <tt>DIST=squeeze</tt>, e.g. <tt>DIST=oneiric</tt> for the 11.10 Ubuntu release.
  
 
In order to build a i386 build after the amd64 build, select -B to do a binary build only. Passing arguments is a bit tricky:
 
In order to build a i386 build after the amd64 build, select -B to do a binary build only. Passing arguments is a bit tricky:
Line 77: Line 364:
 
  ARCH=i386 DIST=stable svn-buildpackage --svn-builder pdebuild --pbuilder cowbuilder --debbuildopts "-us -uc -B"
 
  ARCH=i386 DIST=stable svn-buildpackage --svn-builder pdebuild --pbuilder cowbuilder --debbuildopts "-us -uc -B"
  
−
=== Creating a repository of packages to be downloaded ===
+
=== cowpoke ===
 +
 
 +
The cowpoke utility starts a cowbuilder on a remote machine and takes care of signing. Put the following in your ${HOME}/.cowpoke:
 +
BUILDD_HOST=ref-debian6-64.testbed
 +
SIGN_KEYID=$YOUR_GPG_KEY_ID
 +
DEBOOTSTRAP=multistrap
 +
RETURN_DIR="."
 +
for i in squeeze wheezy sid precise oneiric natty maveric lucid karmic jaunty intrepid hardy
 +
do
 +
    for j in i386 amd64
 +
    do
 +
eval ${j}_${i}_RESULT_DIR="$PBUILDER_BASE/${i}-${j}/result"
 +
eval ${j}_${i}_BASE_PATH="$PBUILDER_BASE/${i}-${j}/base.cow"
 +
    done
 +
done
  
−
Debian offers several tools for generating repositories; some are meant for full-size Debian archives and others for simple, private use. For the SAC middleware the <tt>reprepro</tt> utility offers enough features while at the same time it is easy to install and maintain.
+
=== reprepro ===
  
−
Basically follow the [http://www.debian-administration.org/articles/286 guidelines on how to set up a repository]. TODO: link the pbuilder output to this.
+
The reprepro Debian repository is a lightweight version of a real Debian repository. There are other options, for personal use you may consider mini-dinstall, but reprepro is what's set up on ref-debian6-64.testbed.
  
−
Put the build results in the 'incoming' directory. Use this in <tt>$HOME/.dput.cf</tt>
+
The repository lives in /var/local/debian-repository/. The place where new packages are uploaded is /var/local/debian-repository/incoming/$dist and the configuration file is /var/local/debian-repository/debian/conf/distributions.
−
  [mwsec]
+
This is part of it (there is more but it's along the same lines).
 +
Origin: Nikhef Grid Security Middleware
 +
Label: Nikhef Grid Security Middleware for oneiric
 +
Codename: oneiric
 +
Architectures: i386 amd64 source
 +
Components: main
 +
Description: Apt repository for Nikhef Grid Security Middleware
 +
  compatible with Debian oneiric
 +
DebOverride: override.oneiric
 +
DscOverride: override.oneiric
 +
SignWith: 8AAE244E
 +
 +
Origin: Nikhef Grid Security Middleware
 +
Label: Nikhef Grid Security Middleware for squeeze
 +
Codename: squeeze
 +
Architectures: i386 amd64 source
 +
Components: main
 +
Description: Apt repository for Nikhef Grid Security Middleware
 +
  compatible with Debian squeeze
 +
DebOverride: override.squeeze
 +
DscOverride: override.squeeze
 +
SignWith: 8AAE244E
 +
 
 +
The /var/local/debian-repository/debian/conf/options file:
 +
verbose
 +
basedir /var/local/debian-repository/debian
 +
ask-passphrase
 +
 
 +
The SignWith key is a passwordless GPG key that is used by the repository to automatically sign the Release.gpg file. See the dput configuration on how to upload packages to reprepro.
 +
 
 +
=== dput ===
 +
 
 +
The dput utility uploads Debian source and binary packages, based on what's given an a ''changes'' file (those are output by dpkg-buildpackage).
 +
 
 +
The ${HOME}/.dput.cf contains something along these lines:
 +
 
 +
[mwsec-oneiric]
 +
method = local
 +
incoming = /var/local/debian-repository/incoming/oneiric
 +
allow_non-us_software = 1
 +
run_dinstall = 0
 +
post_upload_command = reprepro -b /var/local/debian-repository/debian processincoming oneiric
 +
 +
  [mwsec-squeeze]
 
  method = local
 
  method = local
−
  incoming = /var/www/repos/apt/incoming
+
  incoming = /var/local/debian-repository/incoming/squeeze
−
  allow_unsigned_uploads = 1
+
  allow_non-us_software = 1
 
  run_dinstall = 0
 
  run_dinstall = 0
 +
post_upload_command = reprepro -b /var/local/debian-repository/debian processincoming squeeze
 +
 +
If you have an account on http://mentors.debian.net/, you can find information about uploading to mentors for your account there.
 +
 +
== Anatomy of a Debian source package ==
 +
 +
This material is provided to help understand how to package for Debian. The most important thing is to familiarize yourself with the available material for Debian maintainers and developers, but as this can be a bit overwhelming, these notes are somewhat tailored to our situation and may help to clarify.
 +
 +
Debian packaging requires a <tt>debian/</tt> subdirectory in the root of the upstream source directory. The structure of this directory is described in the [http://www.debian.org/doc/manuals/maint-guide/ Debian New Maintainers' Guide]. For the SAC middleware, the debian packaging is [https://ndpfsvn.nikhef.nl/cgi-bin/viewvc.cgi/mwsec/packaging/debian/trunk/ maintained in SVN]. Debian does not prescribe the exact method of creating packages, as is the case with RPM, and several helper libraries have emerged over time. The most notable are [https://en.wikipedia.org/wiki/Debhelper debhelper] and the newer [http://build-common.alioth.debian.org/cdbs-doc.html CDBS]. Both should make it a snap to build packages with. In fact, the SAC middleware packaging has both systems in use. This is somewhat inefficient, so we should standardize on one or the other, probably debconf.
 +
 +
=== The changelog file ===
 +
 +
One of the most important files is the changelog file. It records what has changed to the packaging over time, but the latest entry (at the top) determines the version and distribution of the package.
 +
 +
It is advisable not to write entries by hand, as the tools are particular to the syntax; just use dch to update.
 +
 +
=== The rules file ===
 +
 +
Since debconf 7 this should be very short. if the package builds through 'configure; make; make install' the default rule is to call
 +
dh $@
 +
in any case. The rules file is actually a Makefile, and the build program calls it with a sequence of targets.
  
−
And run dput:
+
=== The control file ===
  
−
dput mwsec /var/cache/pbuilder/stable-amd64/result/lcmaps_1.4.34-1_amd64.changes
+
The control file has several paragraphs. One for the source package, and one or more for each binary package. This file controls the dependencies and descriptions of the package.
  
−
Process the 'incoming' directory (this should be automated sometime):
+
=== The copyright file ===
  
−
reprepro -b /var/www/repos/apt/debian processincoming squeeze lcmaps_1.4.34-1_amd64.changes
+
Debian is standardizing on using a [http://www.debian.org/doc/packaging-manuals/copyright-format/1.0/ machine-readable copyright] file. The standard is approved, but still fairly new so you may get lintian warnings about it for all but the latest version of lintian. But the actual, proper header for the copyright should be:
 +
Format: http://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
  
−
(First it may be required to set the distribution in the changes file to 'squeeze' with the <tt>changestool</tt> command).
+
== References ==
  
−
Sync the local repository to software.nikhef.nl:
+
TODO: find the links.
  
−
rsync -rvP /var/www/repos/apt/debian/ software.nikhef.nl:/project/srv/www/site/software/html/dist/debian/
+
* The [http://www.debian.org/doc/manuals/developers-reference/ Debian Developer's Reference]
 +
* The [http://www.debian.org/doc/manuals/maint-guide/ Debian New Maintainers' Guide]
 +
* The [http://www.debian.org/doc/debian-policy/ Debian Policy Manual]
 +
* The [http://www.netfort.gr.jp/~dancer/column/libpkg-guide/libpkg-guide.html Debian Library Packaging guide]

Latest revision as of 12:33, 6 March 2013

<sidebar>

  • navigation
    • Main Page|Main Page
    • SAC software procedures|Software Procedures
    • GLExec|SAC software for users

</sidebar>

The following procedures extend the SAC software procedures with respect to the creation and distribution of Debian and Ubuntu packages.

packaging a new release of an existing component

When a component has been updated and released, it should also be repackaged for Debian at some point. The following steps outline what needs to be done.

Update the Debian source package

Source packages are maintained in SVN. Check out the component and run uscan to fetch the upstream tarball. 

cd $mwsec/packaging/debian/trunk/$component
svn update
uscan --report # optional
uscan --destdir ../tarballs

Update debian/changelog. 

dch -v $version-1 -D UNRELEASED "New upstream release"

Review and test the packaging

Check the upstream changelog if there is anything that may affect the build. Changed paths, modified shared libraries, etc. Test a local build: 

svn-buildpackage --svn-ignore-new -us -uc
debc ../build-area/${component}_${version}-1_${arch}.changes | less
lintian -IiE --pedantic ../build-area/${component}_${version}-1_${arch}.changes

When you're happy commit to svn. 

svn commit

Prepare the source package for uploading

When the packaging is in good shape, it is time to prepare a Debian source package (.dsc). This will be used in later steps for building for different distributions. 

svn-buildpackage -S -sa

releasing packages for multiple distributions

The Debian packages can be uploaded to several places:

The normal lifecycle of a package is that it starts in Debian unstable. From there it will (eventually) reach Debian stable and Ubuntu. However, since this takes (much) time, we make our packages available for users of several distributions; this is called backporting. The backport targets are given in the table below.

The target distribution is set in debian/changelog. We have to repackage the source package to set the distribution and the release number, which is coding the backport.

The first Debian release of a package always has 1 as the release number, so upstream version 1.2.3 becomes 1.2.3-1. If the Debian packaging for the same upstream version is updated, the release number increments. The table below indicates what the release numbers should be for various backports. The use of the '~' is special: when comparing versions the tilde precedes any other symbol including the empty space, e.g. 1.2.3-1 is considered newer than 1.2.3-1~backport. This is a very useful trick to make sure that the distribution's own package of the same version is preferred over a backport. The number after the backport codename is another sequence number that increments with every backport-specific rebuild of the same upstream-version-and-debian-release.

distribution version code name release tag notes
Debian unstable sid 1 Sid (Debian unstable) is not a backport.
6 squeeze 1~bpo60+1 This is the Debian style for backports.
Ubuntu 8.04 LTS hardy 1~hardy1 Hardy is difficult, because the debhelper package is too old (see man dh).
10.04 LTS lucid 1~lucid1 The Ubuntu style for backporting is slightly simpler.
10.10 maverick 1~maverick1
11.04 natty 1~natty1
11.10 oneiric 1~oneiric1

Preparing the source packages for backporting

The following script extracts the given UNRELEASED source packages (the output of the first step), updates the changelog, and repackages so they can be build by cowbuilder or cowpoke. 

cat > prepare-backports.sh <<'EOF'
#!/bin/sh

# prepare backports for packages
# arguments: list of .dsc files

# action: for each dsc file and for each distribution:
# unpack the dsc file
# dch -v $version-release~$backportcode --distribution $dist "Rebuild for xxx"

# arguments: source package, dist (directory name), target distribution, release code
makesrc() {
    cd $2
    dpkg-source -x ../$1
    pkg=`basename $1`
    cd `echo $pkg | sed -e 's/\([^_]*\)_\([^-]*\).*/\1-\2/'`
    version=`dpkg-parsechangelog | sed -n 's/^Version: //p'`
    dch --distribution $3 -b -v ${version}~${4} "Rebuild for $2"
    dpkg-buildpackage -S -sa
    cd ../..

}

ubuntu="hardy lucid maverick natty oneiric"
mkdir $ubuntu squeeze sid

while [ $# -ge 1 ]; do
    for dist in $ubuntu; do
	makesrc $1 $dist $dist-backports ${dist}1
    done
    # similar but different for debian
    makesrc $1 squeeze squeeze bpo60+1

    # treat sid specially
    cd sid
    dpkg-source -x ../$1
    pkg=`basename $1`
    cd `echo $pkg | sed -e 's/\([^_]*\)_\([^-]*\).*/\1-\2/'`
    dch --distribution unstable --release ""
    dpkg-buildpackage -S -sa
    cd ../..

    shift
done
EOF

Running this script unpacks and repacks the sources multiple times in various subdirectories. After building with cowbuilder or cowpoke (cowbuilder on a remote machine) it is no longer required to keep them around.

Starting the remote build for a backport

The reference machine ref-debian6-64.testbed has been configured for cowbuilder for all distributions, in the architectures i386 and amd64. The cowpoke program uploads the sources and launches a remote build, downloads the results and signs the changes files after the build. (See the configuration section on how to set up cowpoke.) 

( cd sid
  cowpoke --dist=sid --arch=amd64 ${component}_{version}-1.dsc
  cowpoke --dist=sid --arch=i386 --dpkg-opts=-B ${component}_{version}-1.dsc )
( cd squeeze
  cowpoke --dist=squeeze --arch=amd64 ${component}_{version}-1~bpo60+1.dsc
  cowpoke --dist=squeeze --arch=i386 --dpkg-opts=-B ${component}_{version}-1~bpo60+1.dsc
)

The other builds follow the same pattern. Note: the -B option only builds the architecture-specific packages. This prevents regenerating the source package and architecture-independent packages, which are already produced by the amd64 build. (The ordering is not specific, but if both builds produce the same packages we'll get in trouble later on when uploading to the repository. It refuses to accept packages that it already has, and on top of that the checksums will be different.)

Uploading the builds

The build results are found under ref-debian6-64.testbed:/var/cache/pbuilder/$dist-$arch/results. The dput command is used to perform the uploading (see the configuration section how to set up dput).

local repository

dput mwsec-squeeze /var/cache/pbuilder/squeeze-amd64/results/${component}_${version}-1~bpo60+1_amd64.changes
dput mwsec-squeeze /var/cache/pbuilder/squeeze-i386/results/${component}_${version}-1~bpo60+1_i386.changes
# similar for other backports
rsync -a -vP --delete /var/local/debian-repository/debian/ software.nikhef.nl:/project/srv/www/site/software/html/dist/debian/

Debian mentors

In order to get the packages in the main Debian distribution, they need to be uploaded to Debian mentors. Subsequently a sponsor must be found to upload the packages to unstable. Always upload the unstable packages to mentors. 

dput debexpo /var/cach/pbuilder/sid-amd64/result/${component}_${version}-1_amd64.changes

There is no need to upload for all architectures as Debian will rebuild the packages anyway.

Ubuntu ppa

There is no ppa configured for mwsec at the moment.

Setting up a personal build environment

Before getting anything to work, it is necessary to make several preparations. Luckily, it's just a one-time endeavor.

GPG key

All Debian packaging tools use GPG signing by default. If you already have a gpg key, you can use that. If not, create one. Preferably using a key length of at least 2048 (although 4096 is common these days).

Environment variables

The Debian new maintainers guide suggests to put the following variables in ${HOME}/.bashrc: 

export DEBEMAIL=youremail@nikhef.nl
export DEBFULLNAME="John D. Developer"

You must use values that exactly match your GPG key identity.

For using the debsign tool later on (for signing packages) put these settings in ${HOME}/.devscripts: 

DEBSIGN_KEYID=8AAE244E

but fill in the key id of your own GPG key. (You can find it by typing gpg --list-secret-keys.)

Machine environment

Development should be done on either a Debian or Ubuntu machine. Root privileges are not required, but it is necessary that a certain number of developer's tools are installed (besides the build dependencies of the packages you work on, obviously). The New Maintainers' Guide suggests several packages. 

apt-get install build-essentials dh-make devscripts autotools-dev cdbs debhelper fakeroot lintian pbuilder \
                cowbuilder quilt

Take care to install recent versions of at least lintian, devscripts and debhelper. Older distributions may not have the required versions, in which case you should consider installing a backport of the right versions.

Setting up pbuilder and cowbuilder is rather advanced, but very worthwhile to do clean-room installations.

Working directory layout

You are almost done. The last thing to do is setting up the work area for doing the packaging.

All of the packaging of MWSEC is done in SVN. However, to keep the repository clean, the original sources are not included. The Debian way of packaging is based on the original sources plus a debian subdirectory, so our separated setup means that things will have to be merged before actual building can take place.

Fortunately, this mode of operation is supported by the tool svn-buildpackage. If the property svn:mergeWithUpstream is set on the debian directory, the tool will understand that it must make the merge before commencing the build. In this setup, all the original tarballs are put in ../tarballs and the merge is done in ../build-area. 

svn checkout svn+ssh://svn@ndpfsvn.nikhef.nl/repos/mwsec/packaging/debian/trunk
cd trunk
mkdir tarballs
mkdir build-area

The locations of tarballs and build-area is configurable. These settings in ${HOME}/.svn-buildpackage.conf are only a suggestion. 

svn-override=origDir=$HOME/debian/upstream,buildarea=$HOME/debian/build-area

packaging a new component for the first time

The terse version:

  • Unpack the upstream sources.
  • Run dh_make -c apache (and answer some questions).
  • Review the contents of the debian/ subdirectory.
  • Create a subdirectory in SVN under mwsec/packaging/debian/trunk for the new package.
  • add the debian directory to this subdirectory
  • set the svn:MergeWithUpstream property.

The longer version.

Download the new component from http://software.nikhef.nl/security/$component. Unpack it and run dh_make. 

dh_make -c apache -f ../$component-$version.tar.gz

Dh_make asks a couple of questions and then presents a lot of template files in the debian/ directory. Many of these are probably irrelevant and should be deleted.

Things that need to be filled in in various places:

debian/control Homepage https://wiki.nikhef.nl/grid/Site_Access_Control
debian/control Upstream Author Nikhef Grid Middleware Security Team <grid-mw-security@nikhef.nl>
debian/control Vcs-Svn http://ndpfsvn.nikhef.nl/ro/mwsec/packaging/debian/trunk/$component
debian/control Vcs-Browser http://ndpfsvn.nikhef.nl/cgi-bin/viewvc.cgi/mwsec/packaging/debian/trunk/$component

For guidance, read the Debian New Maintainers' Guide.

Try to build the package and use tools like dpkg-depcheck, debc and lintian to figure out what needs to be done. If you are reasonably satisfied it is time to incorporate the work in SVN.

The packaging of Nikhef Grid Middleware is maintained in SVN. Check out svn+ssh://svn@ndpfsvn.nikhef.nl/repos/mwsec/packaging/debian/trunk and create a new directory named after the new component, but without a version number. Inside this directory, create a directory named 'debian'. 

svn co svn+ssh://svn@ndpfsvn.nikhef.nl/repos/mwsec/packaging/debian/trunk
cd trunk
mkdir -p $component/debian
svn add $component
svn propset svn:mergeWithUpstream 1 $component/debian

Now copy the files from the prepared source package's debian directory to the debian directory in SVN. This way you don't accidentally copy anything in there that you will have to remove later on anyway. Commit the changes. Building the package must now be done with svn-buildpackage. Because the original sources and the debian directory are separated, they must first be reintegrated and that is what the svn:mergeWithUpstream setting is for. The original tarball is expected in the ../tarballs directory, and the actual building is going to be rooted in ../build-area. After setting this up, try to build the package: 

svn-buildpackage -us -uc

Copyright

It's important to get the copyright statements correct. For most of the middleware it should be something like this: 

Copyright: 2004-2010 Members of the EGEE Collaboration
           2010-2012 Members of EMI collaboration

The dates may vary. For the Debian packaging itself, the copyright should read: 

Copyright: 2012 Stichting FOM <grid-mw-security@nikhef.nl>

Patching

Sometimes a package needs to be patched in order to make it work for Debian. Since we are our own upstream providers, fixes are best put in the original sources in a new release. Occasionally it might concern something that is simply too Debian specific to handle upstream. In those cases, quilt should be used as a patch system.

It's important that this must be done on the unpacked and merged sources (use svn-buildpackage --svn-export to do the merge but no build). The command sequence is 

quilt new fix-this-urgently-for-debian.patch
quilt add the-file-that-needs-fixing
(edit the file as needed)
(possibly repeat the add and edit steps for other files)
quilt refresh

This should result in a patch file under debian/patches/ as well as a series file. Copy the whole thing to the SVN work area. It should be automatically picked up the next time you run svn-buildpackage. There is a proposal to make annotations to the patches a requirement. At the moment it is highly recommended.

Configuration

pbuilder/cowbuilder

Pbuilder is a system to build packages in a clean, chroot environment. It can use a base.tgz tarball, but the slightly more efficient way is to do 'copy on write' a.k.a. COW with cowbuilder. There are excellent documents on how to set this up.

The cowbuilder environment is already configured on ref-debian6-64.testbed; this documentation exists to reproduce the setup in case you need a personal cowbuilder environment.

Both pbuilder and cowbuilder require root privileges, because they need to do chroot(). The most convenient way is by configuring sudo. User visudo and add something along these lines: 

Cmnd_Alias PBUILDERS =  /usr/sbin/pbuilder, /usr/sbin/cowbuilder
johnd  ALL= NOPASSWD: SETENV: PBUILDERS

Following the guidance at the Debian cowbuilder wiki, create a ${HOME}/.pbuilderrc based on the Ubuntu Pbuilder HOWTO, and uncomment 

BASEPATH="/var/cache/pbuilder/$NAME/base.cow/

You should make a separate cowbuilder environment for every distribution you need to make builds for. 

sudo cowbuilder \
--create \
--basepath /var/cache/pbuilder/squeeze-i386/base.cow\
 --distribution squeeze\
 --debootstrapopts --arch --debootstrapopts i386

(Update: this command didn't work on ref-debian6-64.testbed for precise; cowbuilder runs 'pbuilder create' with a wrong mirror (Debian instead of Ubuntu). The fix is to run pbuilder manually like so): 

sudo pbuilder create --debootstrapopts --arch --debootstrapopts i386 --debootstrapopts \
--keyring --debootstrapopts /usr/share/keyrings/ubuntu-archive-keyring.gpg --buildplace \
/var/cache/pbuilder/precise-i386/base.cow --mirror http://archive.ubuntu.com/ubuntu/ --architecture i386 \
--distribution precise --no-targz --extrapackages cowdancer

Inside the cowbuilder environment, add our own software repository to the /etc/apt/sources.list in order to resolve build dependencies on our own software. Add the repositories GPG key to apt as well. 

sudo ARCH=i386 DIST=squeeze cowbuilder --architecture=i386 --login --save-after-login
cat >> /etc/apt/sources.list <<EOF
deb http://software.nikhef.nl/dist/debian/ squeeze main
EOF
wget -O- http://software.nikhef.nl/dist/debian/DEB-GPG-KEY-MWSEC.asc | apt-key add -
apt-get update

Debian now requires the use of hardening flags in the compiler. For the MWsec middleware this requires the use of a recent dpkg-dev package; at least version 1.16. Since this is not available for squeeze, we need to install it manually from the squeeze-backports repository. So in addition to the above steps, also do the following: 

cat >> /etc/apt/sources.list <<EOF
deb http://backports.debian.org/debian-backports squeeze-backports main
EOF
apt-get update
apt-get -t squeeze-backports install dpkg-dev

To build a package, it should suffice to go to the SVN directory mwsec/packaging/debian/trunk/$component/ and run: 

DIST=squeeze svn-buildpackage --svn-builder pdebuild --pbuilder cowbuilder

The cowbuilder will use the copy-on-write root in /var/cache/pbuilder/squeeze-amd64/base.cow. For other distributions just change DIST=squeeze, e.g. DIST=oneiric for the 11.10 Ubuntu release.

In order to build a i386 build after the amd64 build, select -B to do a binary build only. Passing arguments is a bit tricky: 

ARCH=i386 DIST=stable svn-buildpackage --svn-builder pdebuild --pbuilder cowbuilder --debbuildopts "-us -uc -B"

cowpoke

The cowpoke utility starts a cowbuilder on a remote machine and takes care of signing. Put the following in your ${HOME}/.cowpoke: 

BUILDD_HOST=ref-debian6-64.testbed
SIGN_KEYID=$YOUR_GPG_KEY_ID
DEBOOTSTRAP=multistrap
RETURN_DIR="."
for i in squeeze wheezy sid precise oneiric natty maveric lucid karmic jaunty intrepid hardy
do
    for j in i386 amd64
    do
	eval ${j}_${i}_RESULT_DIR="$PBUILDER_BASE/${i}-${j}/result"
	eval ${j}_${i}_BASE_PATH="$PBUILDER_BASE/${i}-${j}/base.cow"
    done
done

reprepro

The reprepro Debian repository is a lightweight version of a real Debian repository. There are other options, for personal use you may consider mini-dinstall, but reprepro is what's set up on ref-debian6-64.testbed.

The repository lives in /var/local/debian-repository/. The place where new packages are uploaded is /var/local/debian-repository/incoming/$dist and the configuration file is /var/local/debian-repository/debian/conf/distributions. This is part of it (there is more but it's along the same lines). 

Origin: Nikhef Grid Security Middleware
Label: Nikhef Grid Security Middleware for oneiric
Codename: oneiric
Architectures: i386 amd64 source
Components: main
Description: Apt repository for Nikhef Grid Security Middleware
 compatible with Debian oneiric
DebOverride: override.oneiric
DscOverride: override.oneiric
SignWith: 8AAE244E

Origin: Nikhef Grid Security Middleware
Label: Nikhef Grid Security Middleware for squeeze
Codename: squeeze
Architectures: i386 amd64 source
Components: main
Description: Apt repository for Nikhef Grid Security Middleware
 compatible with Debian squeeze
DebOverride: override.squeeze
DscOverride: override.squeeze
SignWith: 8AAE244E

The /var/local/debian-repository/debian/conf/options file: 

verbose
basedir /var/local/debian-repository/debian
ask-passphrase

The SignWith key is a passwordless GPG key that is used by the repository to automatically sign the Release.gpg file. See the dput configuration on how to upload packages to reprepro.

dput

The dput utility uploads Debian source and binary packages, based on what's given an a changes file (those are output by dpkg-buildpackage).

The ${HOME}/.dput.cf contains something along these lines: 

[mwsec-oneiric]
method = local
incoming = /var/local/debian-repository/incoming/oneiric
allow_non-us_software = 1
run_dinstall = 0
post_upload_command = reprepro -b /var/local/debian-repository/debian processincoming oneiric

[mwsec-squeeze]
method = local
incoming = /var/local/debian-repository/incoming/squeeze
allow_non-us_software = 1
run_dinstall = 0
post_upload_command = reprepro -b /var/local/debian-repository/debian processincoming squeeze

If you have an account on http://mentors.debian.net/, you can find information about uploading to mentors for your account there.

Anatomy of a Debian source package

This material is provided to help understand how to package for Debian. The most important thing is to familiarize yourself with the available material for Debian maintainers and developers, but as this can be a bit overwhelming, these notes are somewhat tailored to our situation and may help to clarify.

Debian packaging requires a debian/ subdirectory in the root of the upstream source directory. The structure of this directory is described in the Debian New Maintainers' Guide. For the SAC middleware, the debian packaging is maintained in SVN. Debian does not prescribe the exact method of creating packages, as is the case with RPM, and several helper libraries have emerged over time. The most notable are debhelper and the newer CDBS. Both should make it a snap to build packages with. In fact, the SAC middleware packaging has both systems in use. This is somewhat inefficient, so we should standardize on one or the other, probably debconf.

The changelog file

One of the most important files is the changelog file. It records what has changed to the packaging over time, but the latest entry (at the top) determines the version and distribution of the package.

It is advisable not to write entries by hand, as the tools are particular to the syntax; just use dch to update.

The rules file

Since debconf 7 this should be very short. if the package builds through 'configure; make; make install' the default rule is to call 

dh $@

in any case. The rules file is actually a Makefile, and the build program calls it with a sequence of targets.

The control file

The control file has several paragraphs. One for the source package, and one or more for each binary package. This file controls the dependencies and descriptions of the package.

The copyright file

Debian is standardizing on using a machine-readable copyright file. The standard is approved, but still fairly new so you may get lintian warnings about it for all but the latest version of lintian. But the actual, proper header for the copyright should be: 

Format: http://www.debian.org/doc/packaging-manuals/copyright-format/1.0/

References

TODO: find the links.

Retrieved from "https://wiki.nikhef.nl/grid/index.php?title=SAC_Debian_packaging&oldid=8921"