Difference between revisions of "User:Dennisvd@nikhef.nl/SoftwareOverview"

From PDP/Grid Wiki
Jump to navigationJump to search
 
(6 intermediate revisions by the same user not shown)
Line 12: Line 12:
 
* the software is maintained in a [https://ndpfsvn.nikhef.nl/viewvc/mwsec/ Subversion repository].
 
* the software is maintained in a [https://ndpfsvn.nikhef.nl/viewvc/mwsec/ Subversion repository].
 
** Access to SVN is configured in the central LDAP database.  
 
** Access to SVN is configured in the central LDAP database.  
−
* The software packages follow the GNU coding guidelines, and use automake/autoconf.
+
* The software packages loosely follow the [http://www.gnu.org/prep/standards/ GNU coding standards], and use automake/autoconf.
 
|
 
|
 
* Before tagging, go through [[SAC_software_procedures#Mandatory_Check_list|the mandatory checklist]]:
 
* Before tagging, go through [[SAC_software_procedures#Mandatory_Check_list|the mandatory checklist]]:
Line 35: Line 35:
 
* [mailto:grid-mw-security@nikhef.nl?subject=Release write a signed e-mail] with the name of the tarball and the checksums
 
* [mailto:grid-mw-security@nikhef.nl?subject=Release write a signed e-mail] with the name of the tarball and the checksums
 
|-
 
|-
−
! scope="row"| Packaging
+
! scope="row" rowspan=4| Packaging
 +
! colspan=2| RPM
 +
|-
 +
|
 +
* SPEC files are [https://ndpfsvn.nikhef.nl/viewvc/mwsec/packaging/fedora/trunk maintained in SVN]
 +
** [http://fedoraproject.org/wiki/PackagingGuidelines#Changelogs Update the changelog]
 +
* The local [[Koji_Testbed|koji instance]] automatically [[Koji_Testbed#Triggering_a_build_in_svn|triggers builds on committing a SPEC file]] to a tag
 +
** a SVN commit hook on '''sikkel.nikhef.nl''' will start the build
 +
** Koji has a central hub, '''koji-hub.testbed''' that may be [https://koji-hub.testbed/koji/ visited locally through a proxy] or [https://bleek.nikhef.nl:8443/koji/ externally through another].
 +
** There are two builders, '''koji-builder.testbed''' and '''koji-boulder.testbed''' that run all the builds.
 +
** There are nightly builds for all components based on the latest SVN revision.
 
|
 
|
−
{| class="wikitable"
+
[[SAC_software_procedures#Generating_RPM_packages_from_the_distributed_tarballs|Generating RPM packages]]
−
! RPM
+
* [[SAC_software_procedures#Update the SPEC file|update the SPEC file]],
−
! Debian
+
* [[SAC_software_procedures#Starting the Koji build by creating a new tag for the SPEC file|Starting the Koji build by creating a new tag for the SPEC file]],
 +
 
 +
|-
 +
! colspan=2| Debian
 
|-
 
|-
 
|
 
|
−
* SPEC files are [https://ndpfsvn.nikhef.nl/viewvc/mwsec/packaging/fedora/trunk maintained in SVN]  
+
To do Debian packaging you need to [[SAC_Debian_packaging#Setting_up_a_personal_build_environment|set up a work environment]] with the right toolchain. This can be on a local Debian machine, such as a Virtual machine, or through a testbed machine such as '''debian.testbed'''.
−
* a SPEC file together with the source tarballs can be rolled into a source RPM
+
* The ''debian'' directories are [https://ndpfsvn.nikhef.nl/viewvc/mwsec/packaging/debian/trunk/ maintained separately from their components in SVN]. svn-buildpackage will merge the directory with the source tarball.
−
* a source RPM can be rebuild with mock in a controlled environment
+
* The central build machine for Debian is '''ref-debian6-64.testbed'''. It has a cowbuilder configuration to build for various Debian-style distributions.
−
* our [[Koji_Testbed|koji instance]] automatically triggers builds on commits
+
See [[SAC_Debian_packaging]].
−
|
 
−
* The ''debian'' directories are [https://ndpfsvn.nikhef.nl/viewvc/mwsec/packaging/debian/trunk/ maintained separately from their components in SVN]. svn-buildpackage will merge the directory with the source tarball. See [[SAC_Debian_packaging]].
 
−
*
 
−
|}
 
 
|  
 
|  
 +
[[SAC_software_procedures#Generating_Debian_packages_from_the_distributed_source_tarballs|Generating Debian packages]]
 +
* check out the [https://ndpfsvn.nikhef.nl/viewvc/mwsec/packaging/debian/trunk/ component directory]
 +
* download the original tarball
 +
* generate an initial source package for the UNRELEASED distribution
 +
* prepare all required backports
 +
* start cowpoke for each backport
 
|-
 
|-
−
! scope="row"| Distribution
+
! scope="row" rowspan=5| Distribution
 +
|
 +
The built packages are collected in repositories for automated download and installation. Besides the local Nikhef repository,
 +
the packages are integrated downstream by middleware projects such as
 +
* EMI,
 +
* IGE,
 +
* UMD and
 +
* OSG.
 +
A subset of the packages is submitted for integration in mainline distributions such as
 +
* Fedora,
 +
* EPEL,
 +
* Debian and
 +
* Ubuntu
 +
Packages and repositories are GPG signed where possible.
 
|
 
|
−
* mwsec repository
 
−
* EMI, IGE, UMD and other middleware integrators
 
−
* Fedora, EPEL, Debian, Ubuntu
 
−
|
 
−
koji tags, sigul, mash, signed repositories, dput
 
 
* write Release notes and announce on [https://mailman.nikhef.nl/mailman/listinfo/grid-mw-security-announce the announce mailing list].
 
* write Release notes and announce on [https://mailman.nikhef.nl/mailman/listinfo/grid-mw-security-announce the announce mailing list].
−
 
+
|-
 +
! colspan=2| RPM
 +
|-
 +
|
 +
The koji builds that are ready for release are tagged, so they can be automatically picked up by mash.
 +
The signing system '''sigul.testbed''' is set up to sign all the RPMs belonging to a release tag.
 +
|
 +
* tag a build either ''testing'' or ''release''
 +
* move a package from ''testing'' to ''release''
 +
* sigulsign_mwsec.py the tag
 +
* write out the signed RPMs
 +
* update the mash repository
 +
* sync to '''software.nikhef.nl'''
 +
|-
 +
! colspan=2| Debian
 +
|-
 +
|
 +
The Debian packages are already signed by the packager. The packages are delivered to the reprepro repository on ref-debian6-64.testbed. This repository is signed with an automatic signing key.
 +
|
 +
* dput the .changes files to the right distribution
 +
* sync to '''software.nikhef.nl'''
 
|}
 
|}

Latest revision as of 23:30, 17 January 2013

Cheat Sheet

The following table gives an overview of the workflow and procedures involving development and delivery of our software.

resources procedures
Source
  • Before tagging, go through the mandatory checklist:
    • Update configure.ac with the right version
    • make distcheck
    • Update the files BUGS, INSTALL, NEWS, README
    • svn update ; svn log -v > ChangeLog
    • commit to SVN
  • Prior to preparing a release, the software should be tagged.
Release

Releasing the tarball requires interaction with several testbed machines. Make good use of the fact that they share the home directories.

  • svn export svn+ssh://svn@ndpfsvn.nikhef.nl/repos/mwsec/tags/component/tag component_tag
  • Run bootstrap on fc17.testbed, to use the newest possible automake/autoconf
  • Run 'make dist' on mwsecbuild.testbed; this machine has all the required dependencies for a build
Packaging RPM

Generating RPM packages

Debian

To do Debian packaging you need to set up a work environment with the right toolchain. This can be on a local Debian machine, such as a Virtual machine, or through a testbed machine such as debian.testbed.

  • The debian directories are maintained separately from their components in SVN. svn-buildpackage will merge the directory with the source tarball.
  • The central build machine for Debian is ref-debian6-64.testbed. It has a cowbuilder configuration to build for various Debian-style distributions.

See SAC_Debian_packaging.

Generating Debian packages

  • check out the component directory
  • download the original tarball
  • generate an initial source package for the UNRELEASED distribution
  • prepare all required backports
  • start cowpoke for each backport
Distribution

The built packages are collected in repositories for automated download and installation. Besides the local Nikhef repository, the packages are integrated downstream by middleware projects such as

  • EMI,
  • IGE,
  • UMD and
  • OSG.

A subset of the packages is submitted for integration in mainline distributions such as

  • Fedora,
  • EPEL,
  • Debian and
  • Ubuntu

Packages and repositories are GPG signed where possible.

RPM

The koji builds that are ready for release are tagged, so they can be automatically picked up by mash. The signing system sigul.testbed is set up to sign all the RPMs belonging to a release tag.

  • tag a build either testing or release
  • move a package from testing to release
  • sigulsign_mwsec.py the tag
  • write out the signed RPMs
  • update the mash repository
  • sync to software.nikhef.nl
Debian

The Debian packages are already signed by the packager. The packages are delivered to the reprepro repository on ref-debian6-64.testbed. This repository is signed with an automatic signing key.

  • dput the .changes files to the right distribution
  • sync to software.nikhef.nl
Retrieved from "https://wiki.nikhef.nl/grid/index.php?title=User:Dennisvd@nikhef.nl/SoftwareOverview&oldid=8792"