Difference between revisions of "User:Dennisvd@nikhef.nl/SoftwareOverview"

From PDP/Grid Wiki
Jump to navigationJump to search
 
(17 intermediate revisions by the same user not shown)
Line 1: Line 1:
−
== Overview ==
+
== Cheat Sheet ==
  
 
The following table gives an overview of the workflow and procedures involving development and delivery of our software.
 
The following table gives an overview of the workflow and procedures involving development and delivery of our software.
  
−
{|
+
{| class="wikitable"
 
|  
 
|  
−
! procedures
+
! resources
−
! control
+
! procedures
 
|-
 
|-
 
! scope="row"| Source
 
! scope="row"| Source
−
| [[Version Control]]
+
|  
−
| [[tags]]  
+
* the software is maintained in a [https://ndpfsvn.nikhef.nl/viewvc/mwsec/ Subversion repository].
 +
** Access to SVN is configured in the central LDAP database.
 +
* The software packages loosely follow the [http://www.gnu.org/prep/standards/ GNU coding standards], and use automake/autoconf.
 +
|
 +
* Before tagging, go through [[SAC_software_procedures#Mandatory_Check_list|the mandatory checklist]]:
 +
** Update configure.ac with the right version
 +
** make distcheck
 +
** Update the files BUGS, INSTALL, NEWS, README
 +
** <tt>svn update ; svn log -v > ChangeLog</tt>
 +
** commit to SVN
 +
* Prior to preparing a release, the software should be [[SAC_software_procedures#Creating_a_tag_in_Grid_Security_Middleware|tagged]].
 
|-
 
|-
 
! scope="row"| Release
 
! scope="row"| Release
−
| tarballs, checksums, release notes, announcements
+
|
−
| Signoff in e-mail
+
Releasing the tarball requires interaction with several [[Agile_testbed|testbed]] machines. Make good use of the fact that
 +
they share the home directories.
 +
* <tt>svn export svn+ssh://svn@ndpfsvn.nikhef.nl/repos/mwsec/tags/''component''/''tag'' ''component_tag''</tt>
 +
* Run bootstrap on '''fc17.testbed''', to use the newest possible automake/autoconf
 +
* Run 'make dist' on '''mwsecbuild.testbed'''; this machine has all the required dependencies for a build
 +
|
 +
* [[SAC_software_procedures#Release_software_as_tarball|make a tarball]]
 +
* record the sha1 and sha256 checksums
 +
* upload to '''software.nikhef.nl'''
 +
* [mailto:grid-mw-security@nikhef.nl?subject=Release write a signed e-mail] with the name of the tarball and the checksums
 +
|-
 +
! scope="row" rowspan=4| Packaging
 +
! colspan=2| RPM
 
|-
 
|-
−
! scope="row"| Packaging
 
 
|
 
|
−
{|
+
* SPEC files are [https://ndpfsvn.nikhef.nl/viewvc/mwsec/packaging/fedora/trunk maintained in SVN]
−
| RPM
+
** [http://fedoraproject.org/wiki/PackagingGuidelines#Changelogs Update the changelog]
−
| Deb
+
* The local [[Koji_Testbed|koji instance]] automatically [[Koji_Testbed#Triggering_a_build_in_svn|triggers builds on committing a SPEC file]] to a tag
 +
** a SVN commit hook on '''sikkel.nikhef.nl''' will start the build
 +
** Koji has a central hub, '''koji-hub.testbed''' that may be [https://koji-hub.testbed/koji/ visited locally through a proxy] or [https://bleek.nikhef.nl:8443/koji/ externally through another].
 +
** There are two builders, '''koji-builder.testbed''' and '''koji-boulder.testbed''' that run all the builds.
 +
** There are nightly builds for all components based on the latest SVN revision.
 +
|
 +
[[SAC_software_procedures#Generating_RPM_packages_from_the_distributed_tarballs|Generating RPM packages]]
 +
* [[SAC_software_procedures#Update the SPEC file|update the SPEC file]],
 +
* [[SAC_software_procedures#Starting the Koji build by creating a new tag for the SPEC file|Starting the Koji build by creating a new tag for the SPEC file]],
 +
 
 +
|-
 +
! colspan=2| Debian
 
|-
 
|-
−
| spec files, mock, koji
+
|
−
| debian/ directory, cowpoke
+
To do Debian packaging you need to [[SAC_Debian_packaging#Setting_up_a_personal_build_environment|set up a work environment]] with the right toolchain. This can be on a local Debian machine, such as a Virtual machine, or through a testbed machine such as '''debian.testbed'''.
−
|}
+
* The ''debian'' directories are [https://ndpfsvn.nikhef.nl/viewvc/mwsec/packaging/debian/trunk/ maintained separately from their components in SVN]. svn-buildpackage will merge the directory with the source tarball.
 +
* The central build machine for Debian is '''ref-debian6-64.testbed'''. It has a cowbuilder configuration to build for various Debian-style distributions.
 +
See [[SAC_Debian_packaging]].
 
|  
 
|  
 +
[[SAC_software_procedures#Generating_Debian_packages_from_the_distributed_source_tarballs|Generating Debian packages]]
 +
* check out the [https://ndpfsvn.nikhef.nl/viewvc/mwsec/packaging/debian/trunk/ component directory]
 +
* download the original tarball
 +
* generate an initial source package for the UNRELEASED distribution
 +
* prepare all required backports
 +
* start cowpoke for each backport
 
|-
 
|-
−
! scope="row"| Distribution
+
! scope="row" rowspan=5| Distribution
 +
|
 +
The built packages are collected in repositories for automated download and installation. Besides the local Nikhef repository,
 +
the packages are integrated downstream by middleware projects such as
 +
* EMI,
 +
* IGE,
 +
* UMD and
 +
* OSG.
 +
A subset of the packages is submitted for integration in mainline distributions such as
 +
* Fedora,
 +
* EPEL,
 +
* Debian and
 +
* Ubuntu
 +
Packages and repositories are GPG signed where possible.
 +
|
 +
* write Release notes and announce on [https://mailman.nikhef.nl/mailman/listinfo/grid-mw-security-announce the announce mailing list].
 +
|-
 +
! colspan=2| RPM
 +
|-
 +
|
 +
The koji builds that are ready for release are tagged, so they can be automatically picked up by mash.
 +
The signing system '''sigul.testbed''' is set up to sign all the RPMs belonging to a release tag.
 +
|
 +
* tag a build either ''testing'' or ''release''
 +
* move a package from ''testing'' to ''release''
 +
* sigulsign_mwsec.py the tag
 +
* write out the signed RPMs
 +
* update the mash repository
 +
* sync to '''software.nikhef.nl'''
 +
|-
 +
! colspan=2| Debian
 +
|-
 +
|
 +
The Debian packages are already signed by the packager. The packages are delivered to the reprepro repository on ref-debian6-64.testbed. This repository is signed with an automatic signing key.
 
|
 
|
−
* mwsec repository
+
* dput the .changes files to the right distribution
−
* EMI, IGE, UMD and other middleware integrators
+
* sync to '''software.nikhef.nl'''
−
* Fedora, EPEL, Debian, Ubuntu
 
−
| koji tags, sigul, mash, signed repositories, dput
 
 
|}
 
|}

Latest revision as of 01:30, 18 January 2013

Cheat Sheet

The following table gives an overview of the workflow and procedures involving development and delivery of our software.

resources procedures
Source
  • Before tagging, go through the mandatory checklist:
    • Update configure.ac with the right version
    • make distcheck
    • Update the files BUGS, INSTALL, NEWS, README
    • svn update ; svn log -v > ChangeLog
    • commit to SVN
  • Prior to preparing a release, the software should be tagged.
Release

Releasing the tarball requires interaction with several testbed machines. Make good use of the fact that they share the home directories.

  • svn export svn+ssh://svn@ndpfsvn.nikhef.nl/repos/mwsec/tags/component/tag component_tag
  • Run bootstrap on fc17.testbed, to use the newest possible automake/autoconf
  • Run 'make dist' on mwsecbuild.testbed; this machine has all the required dependencies for a build
Packaging RPM

Generating RPM packages

Debian

To do Debian packaging you need to set up a work environment with the right toolchain. This can be on a local Debian machine, such as a Virtual machine, or through a testbed machine such as debian.testbed.

  • The debian directories are maintained separately from their components in SVN. svn-buildpackage will merge the directory with the source tarball.
  • The central build machine for Debian is ref-debian6-64.testbed. It has a cowbuilder configuration to build for various Debian-style distributions.

See SAC_Debian_packaging.

Generating Debian packages

  • check out the component directory
  • download the original tarball
  • generate an initial source package for the UNRELEASED distribution
  • prepare all required backports
  • start cowpoke for each backport
Distribution

The built packages are collected in repositories for automated download and installation. Besides the local Nikhef repository, the packages are integrated downstream by middleware projects such as

  • EMI,
  • IGE,
  • UMD and
  • OSG.

A subset of the packages is submitted for integration in mainline distributions such as

  • Fedora,
  • EPEL,
  • Debian and
  • Ubuntu

Packages and repositories are GPG signed where possible.

RPM

The koji builds that are ready for release are tagged, so they can be automatically picked up by mash. The signing system sigul.testbed is set up to sign all the RPMs belonging to a release tag.

  • tag a build either testing or release
  • move a package from testing to release
  • sigulsign_mwsec.py the tag
  • write out the signed RPMs
  • update the mash repository
  • sync to software.nikhef.nl
Debian

The Debian packages are already signed by the packager. The packages are delivered to the reprepro repository on ref-debian6-64.testbed. This repository is signed with an automatic signing key.

  • dput the .changes files to the right distribution
  • sync to software.nikhef.nl
Retrieved from "https://wiki.nikhef.nl/grid/index.php?title=User:Dennisvd@nikhef.nl/SoftwareOverview&oldid=8792"