Difference between revisions of "GitSetup"

From PDP/Grid Wiki
Jump to navigationJump to search
 
 
(3 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
== Git setup on span ==
 
== Git setup on span ==
  
−
I push siteinfo and firstboot work to git repositories on my home dir on span.nikhef.nl. I installed gitweb to have an easy web frontend for these repositories.
+
The goal is to have a a shared, writeable repository for files controlling the setup of virtual machines, in particular site-info.def and friends, and firstboot scripts.
  
−
I followed the instructions from [http://git.wiki.kernel.org/index.php/Gitweb|the Gitweb wiki].
+
The git version control software has very good branch management and is fully distributed in setup. All actions are committed to a local repository, and can be pushed to (or pulled from) other repositories.
 +
 
 +
=== Gitosis ===
 +
 
 +
To have a shared writable repository nonetheless, gitosis is installed on span.nikhef.nl
 +
 
 +
[http://scie.nti.st/2007/11/14/hosting-git-repositories-the-easy-and-secure-way Instructions for setting up gitosis] are found on-line and these were followed almost literally.
 +
 
 +
git clone git://eagain.net/gitosis.git
 +
cd gitosis/
 +
sudo python setup.py install
 +
sudo adduser -r -d /srv/git -c 'git version control' --shell /bin/sh git
 +
sudo -u git gitosis-init < id-rsa.pub
 +
 
 +
The management of gitosis is through git!
 +
 
 +
git clone git@span:gitosis-admin.git
 +
<manage the config file and keydir>
 +
git commit
 +
git push
 +
 
 +
Now it becomes interesting. After configuring write access to the 'siteinfo' repository, the local siteinfo repository was configured to push to git@span as well:
 +
 
 +
cd ~/src/siteinfo
 +
git remote add shared git@span.nikhef.nl:siteinfo.git
 +
git push shared
 +
 
 +
=== Git-daemon ===
 +
 
 +
It is convenient to have read-only anonymous access to selected repositories as well. First, through git-daemon. This comes with the git distribution so the following init script was installed as /etc/init.d/git-daemon.
 +
 
 +
#!/bin/sh
 +
#
 +
# Startup/shutdown script for Git Daemon
 +
#
 +
#  Linux chkconfig stuff:
 +
#
 +
#  chkconfig: 345 56 10
 +
#  description: Startup/shutdown script for Git Daemon
 +
#
 +
. /etc/init.d/functions
 +
 +
DAEMON=git-daemon
 +
ARGS='--base-path=/srv/git/repositories --detach --user=git --group=git'
 +
 +
prog=git-daemon
 +
 +
start () {
 +
    echo -n $"Starting $prog: "
 +
 +
    # start daemon
 +
    daemon $DAEMON $ARGS
 +
        RETVAL=$?
 +
    echo
 +
    [ $RETVAL = 0 ] && touch /var/lock/git-daemon
 +
    return $RETVAL
 +
}
 +
 +
stop () {
 +
    # stop daemon
 +
    echo -n $"Stopping $prog: "
 +
    killproc $DAEMON
 +
    RETVAL=$?
 +
    echo
 +
    [ $RETVAL = 0 ] && rm -f /var/lock/git-daemon
 +
}
 +
 +
restart() {
 +
    stop
 +
    start
 +
}
 +
 +
case $1 in
 +
    start)
 +
        start
 +
    ;;
 +
    stop)
 +
        stop
 +
    ;;
 +
    restart)
 +
        restart
 +
    ;;
 +
    status)
 +
        status $DAEMON
 +
        RETVAL=$?
 +
    ;;
 +
    *)
 +
 +
    echo $"Usage: $prog {start|stop|restart|status}"
 +
    exit 3
 +
esac
 +
 +
exit $RETVAL
 +
 
 +
 
 +
The git-daemon will only export repositories that have the file named
 +
git-daemon-export-ok
 +
in the root, and gitosis-admin is thereby excluded from public view. This file is not under version control.
 +
 
 +
=== Gitweb ===
 +
instructions were found on [http://git.wiki.kernel.org/index.php/Gitweb|the Gitweb wiki].
  
 
The git installation on span is from DAG:
 
The git installation on span is from DAG:
Line 9: Line 109:
 
  git-1.5.2.1-1.el5.rf
 
  git-1.5.2.1-1.el5.rf
  
−
The web application gitweb.cgi is not included in the rpm, so it was built from the sources in the source rpm.
+
The web application gitweb.cgi is not included in the rpm, so it was built from the sources in the source rpm. The variable GITWEB_EXPORT_OK indicates the file name that flags public visibility for repositories, just like git-daemon.
  
 
  wget http://dag.wieers.com/rpm/packages/git/git-1.5.2.1-1.rf.src.rpm
 
  wget http://dag.wieers.com/rpm/packages/git/git-1.5.2.1-1.rf.src.rpm
Line 16: Line 116:
 
  tar xfj git-1.5.2.1.tar.bz2  
 
  tar xfj git-1.5.2.1.tar.bz2  
 
  cd git-1.5.2.1
 
  cd git-1.5.2.1
−
  make GITWEB_PROJECTROOT=/srv/git prefix=/usr gitweb/gitweb.cgi
+
  make GITWEB_PROJECTROOT=/srv/git/repositories GITWEB_EXPORT_OK=git-daemon-export-ok prefix=/usr gitweb/gitweb.cgi
−
 
+
sudo cp gitweb/gitweb.* /srv/git/gitweb/
−
gitweb is installed in
 
−
 
 
−
/srv/git
 
−
 
 
−
Apache configuration in /etc/httpd/conf.d/git.conf:
 
−
Alias /git /srv/git
 
  
−
  <Directory /srv/git >
+
The apache configuration in /etc/httpd/conf.d/git.conf:
−
        Options Indexes FollowSymLinks ExecCGI  
+
Alias /git /srv/git/gitweb
 +
 +
  <Directory /srv/git/gitweb >
 +
        Options Indexes FollowSymLinks ExecCGI  
 
  AddHandler cgi-script cgi
 
  AddHandler cgi-script cgi
−
        AllowOverride None
+
        AllowOverride None
 
  DirectoryIndex gitweb.cgi
 
  DirectoryIndex gitweb.cgi
 
  RewriteEngine On
 
  RewriteEngine On
Line 37: Line 134:
 
  </Directory>
 
  </Directory>
  
−
The /srv/git directory contains symlinks to the repositories.
+
The repositories can be browsed at http://span.nikhef.nl/git/.
 +
 
 +
== Git repositories for driving certification ==
 +
 
 +
The two main repositories for certification are firstboot.git and siteinfo.git.
 +
 
 +
'''Firstboot''' is a collection of machine-specific scripts that will be run on a new virtual machine, after the installation process; the name of the file must be identical to the virtual machine name. Typically the firstboot scripts will deal with adding users, setting up mounts, repositories, installing and configuring software, etc.
 +
 
 +
'''Siteinfo''' is a tree of site-info.def and friends, to be used by YAIM, the friendly middleware configuration tool.
 +
 
 +
As both of these repositories under gitosis are so-called ''bare'' repositories and therefore have no files, we need to fetch from these repositories in locations where the files should really end up. This is done by adding a post-receive hook that updates the real files whenever new data is received.
 +
 
 +
For firstboot. /srv/git/repositories/firstboot.git/hooks/post-receive:
 +
 
 +
#!/bin/sh
 +
unset GIT_DIR
 +
cd /usr/local/mktestbed/firstboot && git pull
 +
echo "all updated!"
 +
 
 +
for siteinfo, /srv/git/repositories/siteinfo.git/hooks/post-receive:
 +
#!/bin/sh
 +
unset GIT_DIR
 +
/usr/local/mktestbed/siteinfo/site-info-prepare.sh
  
−
$ ll /srv/git
+
The last script updates three separate trees corresponding to the master, condor and mpi branches of site-info:
−
total 352
 
−
lrwxrwxrwx 1 root root    28 Feb  9 23:51 firstboot -> /home/dennisvd/src/firstboot
 
−
-rw-r--r-- 1 root root    164 Feb  9 23:42 git-favicon.png
 
−
-rw-r--r-- 1 root root    208 Feb  9 23:42 git-logo.png
 
−
-rwxr-xr-x 1 root root 153112 Feb  9 23:42 gitweb.cgi
 
−
-rw-r--r-- 1 root root  6946 Feb  9 23:42 gitweb.css
 
−
-rw-r--r-- 1 root root  24142 Feb  9 22:30 gitweb.js
 
−
-rwxr-xr-x 1 root root 153311 Feb  9 23:42 gitweb.perl
 
−
lrwxrwxrwx 1 root root    31 Feb 10 00:13 siteinfo.git -> /home/dennisvd/src/siteinfo.git
 
  
−
You can browse the repositories at http://span.nikhef.nl/git/.
+
#!/bin/sh
 +
 +
cd /usr/local/mktestbed/siteinfo
 +
 +
for i in vanilla strawberry chocolate ; do
 +
( cd $i && git pull )
 +
tar cfzC  $i-new.tar.gz $i .git --exclude .git .
 +
mv $i-new.tar.gz $i.tar.gz
 +
done

Latest revision as of 09:10, 2 March 2010

Git setup on span

The goal is to have a a shared, writeable repository for files controlling the setup of virtual machines, in particular site-info.def and friends, and firstboot scripts.

The git version control software has very good branch management and is fully distributed in setup. All actions are committed to a local repository, and can be pushed to (or pulled from) other repositories.

Gitosis

To have a shared writable repository nonetheless, gitosis is installed on span.nikhef.nl

Instructions for setting up gitosis are found on-line and these were followed almost literally. 

git clone git://eagain.net/gitosis.git
cd gitosis/
sudo python setup.py install
sudo adduser -r -d /srv/git -c 'git version control' --shell /bin/sh git
sudo -u git gitosis-init < id-rsa.pub

The management of gitosis is through git! 

git clone git@span:gitosis-admin.git
<manage the config file and keydir>
git commit
git push

Now it becomes interesting. After configuring write access to the 'siteinfo' repository, the local siteinfo repository was configured to push to git@span as well: 

cd ~/src/siteinfo
git remote add shared git@span.nikhef.nl:siteinfo.git
git push shared

Git-daemon

It is convenient to have read-only anonymous access to selected repositories as well. First, through git-daemon. This comes with the git distribution so the following init script was installed as /etc/init.d/git-daemon. 

#!/bin/sh
#
# Startup/shutdown script for Git Daemon
#
#   Linux chkconfig stuff:
#
#   chkconfig: 345 56 10
#   description: Startup/shutdown script for Git Daemon
#
. /etc/init.d/functions

DAEMON=git-daemon
ARGS='--base-path=/srv/git/repositories --detach --user=git --group=git'

prog=git-daemon

start () {
    echo -n $"Starting $prog: "

    # start daemon
    daemon $DAEMON $ARGS
        RETVAL=$?
    echo
    [ $RETVAL = 0 ] && touch /var/lock/git-daemon
    return $RETVAL
}

stop () {
    # stop daemon
    echo -n $"Stopping $prog: "
    killproc $DAEMON
    RETVAL=$?
    echo
    [ $RETVAL = 0 ] && rm -f /var/lock/git-daemon
}

restart() {
    stop
    start
}

case $1 in
    start)
        start
    ;;
    stop)
        stop
    ;;
    restart)
        restart
    ;;
    status)
        status $DAEMON
        RETVAL=$?
    ;;
    *)

    echo $"Usage: $prog {start|stop|restart|status}"
    exit 3
esac

exit $RETVAL


The git-daemon will only export repositories that have the file named 

git-daemon-export-ok

in the root, and gitosis-admin is thereby excluded from public view. This file is not under version control.

Gitweb

instructions were found on Gitweb wiki.

The git installation on span is from DAG: 

git-1.5.2.1-1.el5.rf

The web application gitweb.cgi is not included in the rpm, so it was built from the sources in the source rpm. The variable GITWEB_EXPORT_OK indicates the file name that flags public visibility for repositories, just like git-daemon. 

wget http://dag.wieers.com/rpm/packages/git/git-1.5.2.1-1.rf.src.rpm
rpm -i git-1.5.2.1-1.rf.src.rpm 
cd rpmbuild/SOURCES/
tar xfj git-1.5.2.1.tar.bz2 
cd git-1.5.2.1
make GITWEB_PROJECTROOT=/srv/git/repositories GITWEB_EXPORT_OK=git-daemon-export-ok prefix=/usr gitweb/gitweb.cgi
sudo cp gitweb/gitweb.* /srv/git/gitweb/

The apache configuration in /etc/httpd/conf.d/git.conf: 

Alias /git /srv/git/gitweb

<Directory /srv/git/gitweb >
       Options Indexes FollowSymLinks ExecCGI 
	AddHandler cgi-script cgi
       AllowOverride None
	DirectoryIndex gitweb.cgi
	RewriteEngine On
	RewriteBase /git
	RewriteCond %{REQUEST_FILENAME} !-f
	RewriteCond %{REQUEST_FILENAME} !-d
	RewriteRule ^.* /gitweb.cgi/$0 [L,PT]
</Directory>

The repositories can be browsed at http://span.nikhef.nl/git/.

Git repositories for driving certification

The two main repositories for certification are firstboot.git and siteinfo.git.

Firstboot is a collection of machine-specific scripts that will be run on a new virtual machine, after the installation process; the name of the file must be identical to the virtual machine name. Typically the firstboot scripts will deal with adding users, setting up mounts, repositories, installing and configuring software, etc.

Siteinfo is a tree of site-info.def and friends, to be used by YAIM, the friendly middleware configuration tool.

As both of these repositories under gitosis are so-called bare repositories and therefore have no files, we need to fetch from these repositories in locations where the files should really end up. This is done by adding a post-receive hook that updates the real files whenever new data is received.

For firstboot. /srv/git/repositories/firstboot.git/hooks/post-receive: 

#!/bin/sh
unset GIT_DIR
cd /usr/local/mktestbed/firstboot && git pull
echo "all updated!"

for siteinfo, /srv/git/repositories/siteinfo.git/hooks/post-receive: 

#!/bin/sh
unset GIT_DIR
/usr/local/mktestbed/siteinfo/site-info-prepare.sh

The last script updates three separate trees corresponding to the master, condor and mpi branches of site-info: 

#!/bin/sh

cd /usr/local/mktestbed/siteinfo

for i in vanilla strawberry chocolate ; do
	( cd $i && git pull )
	tar cfzC  $i-new.tar.gz $i .git --exclude .git .
	mv $i-new.tar.gz $i.tar.gz
done
Retrieved from "https://wiki.nikhef.nl/grid/index.php?title=GitSetup&oldid=7347"