NDPF rsync backup

RSync Indirect Backup Service

Only a few systems in the NDPF are actually linked to the ADSM backup service from SARA: hooimijt and beerput. If other systems have data that should be backed-up, they should rsync that data to beerput so it gets forwarded to tape later. Additionally, the rsync-based backup can serve as a quick-restore for larger data volumes.

The amount of backup storage available on beerput is approximately 450 GByte.

Configuration on beerput

The rsync-backup is initiated from beerput, who connects to rsync-servers on each of the clients. The clients are contacted sequentially, with the backup programme on beerput triggered by cron

10 2 * * * nice -n 10 /project/backupservice/bin/rsync-backup > /var/log/backup.log 2>&1

The server has a configuration directory (/project/backupservice/etc/) with a set of client description files (one file per client), that are named by the hostname and each look like this:

HOSTNAME        hek.nikhef.nl
USERNAME        backup
PASSWORD        27^j8JSds76%lkj
MODULE          fsroot
DEST            hek.nikhef.nl
EXCLUDES        +/project**     +/etc   \
                +/etc/sysconfig**       -*

The password is in plaintext (but typically generated randomly) so the directory itself is readable by root only. A rsync port number can optionally be added to the HOSTNAME directive ("sikkel.nikhef.nl:873").

Each of the clients should have an rsync server started or startable from xinetd, that specifies the rsyncd.conf configuration file to use. An "rsyncd.secrets" file (name specified in the conf file) should hold the password and be mode "0600".

An example client rsync.conf file looks like

uid = root
gid = root
use chroot = yes
max connections = 10
syslog facility = daemon
pid file = /var/run/rsyncd.pid

[fsroot]
       path = /
       comment = hek.nikhef.nl:/
       read only = yes
       list = false
       auth users = backup
       secrets file = /etc/rsyncd.secrets
       hosts allow = 194.171.96.69/32
       hosts deny = 0.0.0.0/0

with an rsyncd.secrets file like:

 backup:27^j8JSds76%lkj

Please remember that firewall rules (on the client and on the router) can influence the connectivity from beerput to the rsyncd daemon.

Archiving

If you want to actual make a persistent archive instead of a backup, you can add

PERSISTENT      yes

to the configuation file. Be sure to set the DEST directive to a special directory for this, since such an archive is painful to use as a backup restore source (it will contain obsolete files by definition). Also, it can only grow, as files will never be deleted from the archive directory (but changed files will be updated).

Suggested name of the dstination directory is "archive-<hostname>", like

DEST            archive-stro.nikhef.nl

Backed-up hosts

bedstee.nikhef.nl (grid database server) dorsvlegel.nikhef.nl (D0 project disks) hek.nikhef.nl (ra.dutchgrid.nl) hooimijt.nikhef.nl (NDPF core file server, perm only) kaasvat.nikhef.nl (ca.dutchgrid.nl) sikkel.nikhef.nl (dist.eugridpma.info)

Notes

Only selected data is copied from the clients, see the configuration files in …/etc/ for details. The syntax is rsync ‘–exclude’ mode (which is different from the ADSM inclexcl.opt file!).