Difference between revisions of "User:Dennisvd@nikhef.nl/SoftwareOverview"

From PDP/Grid Wiki
Jump to navigationJump to search
 
(4 intermediate revisions by the same user not shown)
Line 35: Line 35:
 
* [mailto:grid-mw-security@nikhef.nl?subject=Release write a signed e-mail] with the name of the tarball and the checksums
 
* [mailto:grid-mw-security@nikhef.nl?subject=Release write a signed e-mail] with the name of the tarball and the checksums
 
|-
 
|-
−
! scope="row"| Packaging
+
! scope="row" rowspan=4| Packaging
 +
! colspan=2| RPM
 +
|-
 
|
 
|
−
{| class="wikitable"
+
* SPEC files are [https://ndpfsvn.nikhef.nl/viewvc/mwsec/packaging/fedora/trunk maintained in SVN]
−
! RPM
+
** [http://fedoraproject.org/wiki/PackagingGuidelines#Changelogs Update the changelog]
−
|-
+
* The local [[Koji_Testbed|koji instance]] automatically [[Koji_Testbed#Triggering_a_build_in_svn|triggers builds on committing a SPEC file]] to a tag
 +
** a SVN commit hook on '''sikkel.nikhef.nl''' will start the build
 +
** Koji has a central hub, '''koji-hub.testbed''' that may be [https://koji-hub.testbed/koji/ visited locally through a proxy] or [https://bleek.nikhef.nl:8443/koji/ externally through another].
 +
** There are two builders, '''koji-builder.testbed''' and '''koji-boulder.testbed''' that run all the builds.
 +
** There are nightly builds for all components based on the latest SVN revision.
 
|
 
|
−
* SPEC files are [https://ndpfsvn.nikhef.nl/viewvc/mwsec/packaging/fedora/trunk maintained in SVN]  
+
[[SAC_software_procedures#Generating_RPM_packages_from_the_distributed_tarballs|Generating RPM packages]]
−
* a SPEC file together with the source tarballs can be rolled into a source RPM
+
* [[SAC_software_procedures#Update the SPEC file|update the SPEC file]],
−
* a source RPM can be rebuild with mock in a controlled environment
+
* [[SAC_software_procedures#Starting the Koji build by creating a new tag for the SPEC file|Starting the Koji build by creating a new tag for the SPEC file]],
−
* our [[Koji_Testbed|koji instance]] automatically triggers builds on commits
+
 
 
|-
 
|-
−
! Debian
+
! colspan=2| Debian
 
|-
 
|-
 
|
 
|
−
* The ''debian'' directories are [https://ndpfsvn.nikhef.nl/viewvc/mwsec/packaging/debian/trunk/ maintained separately from their components in SVN]. svn-buildpackage will merge the directory with the source tarball. See [[SAC_Debian_packaging]].
+
To do Debian packaging you need to [[SAC_Debian_packaging#Setting_up_a_personal_build_environment|set up a work environment]] with the right toolchain. This can be on a local Debian machine, such as a Virtual machine, or through a testbed machine such as '''debian.testbed'''.
−
*
+
* The ''debian'' directories are [https://ndpfsvn.nikhef.nl/viewvc/mwsec/packaging/debian/trunk/ maintained separately from their components in SVN]. svn-buildpackage will merge the directory with the source tarball.
−
|}
+
* The central build machine for Debian is '''ref-debian6-64.testbed'''. It has a cowbuilder configuration to build for various Debian-style distributions.
 +
See [[SAC_Debian_packaging]].
 
|  
 
|  
−
{| class="wikitable"
+
[[SAC_software_procedures#Generating_Debian_packages_from_the_distributed_source_tarballs|Generating Debian packages]]
−
! RPM
+
* check out the [https://ndpfsvn.nikhef.nl/viewvc/mwsec/packaging/debian/trunk/ component directory]
−
! Debian
+
* download the original tarball
 +
* generate an initial source package for the UNRELEASED distribution
 +
* prepare all required backports
 +
* start cowpoke for each backport
 
|-
 
|-
 +
! scope="row" rowspan=5| Distribution
 
|
 
|
−
[[SAC_software_procedures#Generating_RPM_packages_from_the_distributed_tarballs|Generating RPM packages]]
+
The built packages are collected in repositories for automated download and installation. Besides the local Nikhef repository,
 +
the packages are integrated downstream by middleware projects such as
 +
* EMI,
 +
* IGE,
 +
* UMD and
 +
* OSG.
 +
A subset of the packages is submitted for integration in mainline distributions such as
 +
* Fedora,
 +
* EPEL,
 +
* Debian and
 +
* Ubuntu
 +
Packages and repositories are GPG signed where possible.
 +
|
 +
* write Release notes and announce on [https://mailman.nikhef.nl/mailman/listinfo/grid-mw-security-announce the announce mailing list].
 +
|-
 +
! colspan=2| RPM
 +
|-
 +
|
 +
The koji builds that are ready for release are tagged, so they can be automatically picked up by mash.
 +
The signing system '''sigul.testbed''' is set up to sign all the RPMs belonging to a release tag.
 
|
 
|
−
[[SAC_software_procedures#Generating_Debian_packages_from_the_distributed_source_tarballs|Generating Debian packages]]
+
* tag a build either ''testing'' or ''release''
−
|}
+
* move a package from ''testing'' to ''release''
 +
* sigulsign_mwsec.py the tag
 +
* write out the signed RPMs
 +
* update the mash repository
 +
* sync to '''software.nikhef.nl'''
 +
|-
 +
! colspan=2| Debian
 
|-
 
|-
−
! scope="row"| Distribution
 
 
|
 
|
−
* mwsec repository
+
The Debian packages are already signed by the packager. The packages are delivered to the reprepro repository on ref-debian6-64.testbed. This repository is signed with an automatic signing key.
−
* EMI, IGE, UMD and other middleware integrators
+
|
−
* Fedora, EPEL, Debian, Ubuntu
+
* dput the .changes files to the right distribution
−
|
+
* sync to '''software.nikhef.nl'''
−
koji tags, sigul, mash, signed repositories, dput
 
−
* write Release notes and announce on [https://mailman.nikhef.nl/mailman/listinfo/grid-mw-security-announce the announce mailing list].
 
−
 
 
 
|}
 
|}

Latest revision as of 01:30, 18 January 2013

Cheat Sheet

The following table gives an overview of the workflow and procedures involving development and delivery of our software.

resources procedures
Source
  • Before tagging, go through the mandatory checklist:
    • Update configure.ac with the right version
    • make distcheck
    • Update the files BUGS, INSTALL, NEWS, README
    • svn update ; svn log -v > ChangeLog
    • commit to SVN
  • Prior to preparing a release, the software should be tagged.
Release

Releasing the tarball requires interaction with several testbed machines. Make good use of the fact that they share the home directories.

  • svn export svn+ssh://svn@ndpfsvn.nikhef.nl/repos/mwsec/tags/component/tag component_tag
  • Run bootstrap on fc17.testbed, to use the newest possible automake/autoconf
  • Run 'make dist' on mwsecbuild.testbed; this machine has all the required dependencies for a build
Packaging RPM

Generating RPM packages

Debian

To do Debian packaging you need to set up a work environment with the right toolchain. This can be on a local Debian machine, such as a Virtual machine, or through a testbed machine such as debian.testbed.

  • The debian directories are maintained separately from their components in SVN. svn-buildpackage will merge the directory with the source tarball.
  • The central build machine for Debian is ref-debian6-64.testbed. It has a cowbuilder configuration to build for various Debian-style distributions.

See SAC_Debian_packaging.

Generating Debian packages

  • check out the component directory
  • download the original tarball
  • generate an initial source package for the UNRELEASED distribution
  • prepare all required backports
  • start cowpoke for each backport
Distribution

The built packages are collected in repositories for automated download and installation. Besides the local Nikhef repository, the packages are integrated downstream by middleware projects such as

  • EMI,
  • IGE,
  • UMD and
  • OSG.

A subset of the packages is submitted for integration in mainline distributions such as

  • Fedora,
  • EPEL,
  • Debian and
  • Ubuntu

Packages and repositories are GPG signed where possible.

RPM

The koji builds that are ready for release are tagged, so they can be automatically picked up by mash. The signing system sigul.testbed is set up to sign all the RPMs belonging to a release tag.

  • tag a build either testing or release
  • move a package from testing to release
  • sigulsign_mwsec.py the tag
  • write out the signed RPMs
  • update the mash repository
  • sync to software.nikhef.nl
Debian

The Debian packages are already signed by the packager. The packages are delivered to the reprepro repository on ref-debian6-64.testbed. This repository is signed with an automatic signing key.

  • dput the .changes files to the right distribution
  • sync to software.nikhef.nl
Retrieved from "https://wiki.nikhef.nl/grid/index.php?title=User:Dennisvd@nikhef.nl/SoftwareOverview&oldid=8792"