Maui reservations

Creating recurring reservations with Maui is tricky. So far all attempts creating a standing reservation failed to reserve what we wanted. It did reserve "something else", though. Using another approach, based on the setres command shows better results.

The following was tested with Maui version 3.3.4.

The concrete challenge: create a reservation for 6 cores for jobs submitted to the short queue on working days (Mon-Fri) from 9:30 to 18:30.

Standing Reservation

Using a standing reservation in Maui, the following fragment should be added to the configuration file /var/spool/maui/maui.cfg, followed by a restart of the maui daemon.

SRCFG[short] STARTTIME=9:30:00 ENDTIME=18:30:00
SRCFG[short] PERIOD=DAY DEPTH=3
SRCFG[short] DAYS=Mon,Tue,Wed,Thu,Fri
SRCFG[short] TASKCOUNT=6 RESOURCES=PROCS:1
SRCFG[short] CLASSLIST=short

Maui creates a reservation:

# showres
Reservations

ReservationID       Type S       Start         End    Duration    N/P    StartTime

short.0.0           User -    00:00:00     7:01:46     7:01:46    2/8    Fri Jun 21 11:28:14

1 reservation located

The 'Start' is always 00:00:00. The numbers of reserved nodes and processors are 2 and 8, respectively (why? we asked for 6 processors, right?). Anyway, submitting enough jobs to fill the entire cluster via another queue (i.e., outside the reservation) results in N-2 running jobs. So apparently, Maui reserved only 2 cores for the short queue....

Reservation via setres

Another shot, this time using the setres command (e.g. from a cron job). Again, we try to reserve 6 slots as 2 slots per node on 3 explicitly named nodes:

# /usr/bin/setres -n short  -r "PROCS=2" -s 09:30:00 -e 18:30:00 -c short 'tbn(08|09|17).nikhef.nl'

Maui has the following reservation:

# showres short.0
Reservations

ReservationID       Type S       Start         End    Duration    N/P    StartTime
 
short.0             User -    -01:53:06     7:06:54     9:00:00    3/6    Fri Jun 21 09:30:00

1 reservation located

It is interesting to see that the start time is in the past and that the numbers of nodes and processors now match what we requested.

Again submitting enough jobs to fill the entire cluster to another queue, results in N-6 running jobs. That is exactly what was expected.

System reservations

A system reservation is intended for system maintenance; no jobs are allowed to run during the reservation. A good use case for system reservations is planning maintenaince a few days in advance. As time towards the start of the reservation is running out, only short deadline jobs may still be scheduled.

Setting a system reservation is done just like before:

# setres -s 9:00:00_05/07 -e 17:00:00_05/07 ALL

In this case for all nodes.

If the reservation cannot be made, check the RESDEPTH parameter in maui.conf and try to increase that. We've run into this issue when we introduced machines with 80 (virtual) cores.