MoniFarm Utilities

MoniFarm Utilities

Monifarm is a script to collect data from a local batch system and from a local CE GRIS (Glue Schema v1.2) and store the results in a Round Robin Database (RRD) using the rrdtool programme from Toby Oetiker.

Examples of what MoniFarm does can be seen at the [NDPF Statistics Pages], which are entirely based on MoniFarm.

Prerequisites

To use monifarm, you need RRDTool from Toby Oetiker, and the OpenLDAP clients (for the GRIS-Views). OpenLDAP is part of your standard OS distribution, and RPM with RRDTool (for RHEL3 and compatible systems) can be obtained from DAG, or here: Media:rrdtool-1.0.48-1.rhel3.dag.i386.rpm.

Download

Download the tar ball and binaries from http://www.nikhef.nl/grid/systuils/dl/monifarm. For an excert directly from CVS, please contact the maintainers.

README.txt

 monifarm (README.txt)
 -----------------------------------------------------------------------------
 
  *** This is a pre-release, more documentation should be forthcoming ***
 
 monifarm is a script to collect data from a local batch system and from
 a local CE GRIS (Glue Schema v1.2) and store the results in a Round Robin
 Database (RRD) using the rrdtool programme from Toby Oetiker.
 
 The configuration is done via a file in perl(1) syntax, located by
 default in /opt/monifarm/etc/monifarm.conf, that has the definition of the
 RRD groups, the local unix accounts (groups and users) that correspond
 to that, and optionally the VOView from the CE GRIS that contains additional
 information like the ERT/WRT per VO, job slots &c.
 This information is stored in two (2) RRD databases for further use.
 
 Please read the comments in the example configuration file for more
 information.
 To use a different configuration file, use the "-c" option.
 
 Creating the database
 ---------------------
 Before first use, you'll have to create the RRD database. Before
 doring so, make sure your configuration file is correct, and at your
 options define a few "superfluous" vodefinitions so that you can
 later add vo groups without having to throuw away your history. Remember
 that you can never change the number of vo definitions later without
 throwing away your complete history!
 
 If you are sure that your configuration is correct, create the
 rrd database with
 
       monifarm -c <configuration_file> --create
 
 You can check the integrity of the lrms database later using the
 "--verify" option (to make sure that the configuration file and the
 database still match).
 
 
 Setting up the server
 ---------------------
 make sure your LRMS server allows unrestricted queries from the
 user (and host) where the monitoring script runs. 
 For PBS/Torque, this user must be an "operator" if you have set
 "query_other_jobs" to False.
 
 
 Running the monitoring script
 -----------------------------
 The monitoring script is best run from cron(1), every minute:
 
   * * * *     /opt/monifarm/bin/monifarm -c /etc/monifarm.conf
 
 
 
 Plotting
 --------
 The data from the RRD database can be plotted in many different ways. The
 best is to write your own display commands, and make beatiful graphs that
 are tuned to the VOs you serve. You can use rrdtool's CDEFs to add VOs 
 together in the graphs, normalise them etcetera.
 
 For a series of quick plots, a "plotfarm" command is provided, that reads 
 the original configuration file from monifarm and generates a series
 of standard plots.
 For each VO definition in the configuation file, those groups that have
 a "colour" definitions will appear in the graphs.
 
 A template web page is provided that displays a few of the generated graphs.
 By default, the output is sent to /var/www/html. Copy the web page templates 
 from /opt/monifarm/share/doc/monifarm-0.1/www/ there for immediate viewing.
 Of course you'll have had to run the plotfarm command at least once.
 
 Running the plot script
 -----------------------
 The plot script is best run from cron(1), every 5 minutes:
 
   */5 * * *   /opt/monifarm/bin/monifarm -c /etc/monifarm.conf
 
 
 Comments
 --------
 Comments are welcome. Send them to David Groep, NIKHEF, <davidg@nikhef.nl>