Bieten

How to manage and configure the Suiker Bieten

First some specs about the cluster:

• 4 x Dell R710
  • 2 x Intel X5667 quadcore
  • 96GB DDR3 ram
  • 2 x 600GB 15K RPM SAS disks in hardware raid 1
  • Intel dual 10Ge X520-DA2 network card
  • iDRAC6 enterprise management card
  • 2 x SAS 6Gbps External Controller
  • Onboard 1Ge network cards are disabled in the BIOS

Each Dell R710 has one Dell MD3200 connected with 2 SAS connections. Each Dell MD3200 has 2 controllers and there are 4 Dell MD1200 connected redundante to the Dell MD3200 controllers.

• 4 x Dell MD3200
  • 2 raid controllers
  • 12 x 2TB NearLine SAS 6Gbps 7.2k 3.5" harddrives
  • 4 x Dell MD1200
    • 12 x 2TB NearLine SAS 6Gbps 7.2k 3.5" harddrives

There is also one machine for the management of the storage systems:

• Dell R210II
  • Intel Core i3-2100
  • 4GB DDR3 ram
  • 250GB 7.2K RPM SATA disk
  • 2 x 1Gbit network card
  • iDRAC6 enterprise management card

This machine is connected with a Dell PowerConnect 3524 switch to the Dell MD3200 storage controllers.

Hostnames of the management machine:

• biet-mngt.ipmi.nikhef.nl is to reach the machine itself.
• biet-mngt-oobm.ipmi.nikhef.nl to reach the ipmi/drac/oobm interface of the machine.

To access and configure the storage

Log onto the biet-mngt machine as root. Make sure that X forwarding is configured and working.

Start the following command: /opt/dell/mdstoragesoftware/mdstoragemanager/client/SMclient

This will start the Dell storage management software.

If you'll want to reconfigure one of the storage blocks from scratch, do the following:

!!!WARNING!!! THE FOLLOWING ACTIONS WILL DESTROY ALL DATA ON THE SYSTEM !!!WARNING!!!

• Start the Dell storage management software.
• Click once on the storage device you want to reconfigure.
• Click on Tools > Load Storage Array Configuration
• You'll get a pop-up that you're sure that you want to do that, click on Yes
• You'll get a screen where you can see a couple of *.cfg files, choose the one that has the same name as the storage device.
• Click on Execute and wait when the script has finished configuring the storage device.
• The storage device is now back in the state that we normally use for production. (It can take 24 hours before the storage is done with configuring all the drives and the raid-sets)

How to setup the Suiker Bieten

First off install the machines (for example with a base EL5 Quattor install). The steps below must be executed per machine (Dell server).

After the machine is installed, you'll see a lot of /dev/sd* devices. Just ignore these. The system make use of the device-mapper-multipath software and this will create, /dev/dm-* devices. We will use these devices for the creation of our storage system.

Do the following commands to create the Raid0 and the LVM:

mdadm --create /dev/md0 --level=stripe --chunk=4096 --raid-devices=6 /dev/dm-4 /dev/dm-5 /dev/dm-6 /dev/dm-7 /dev/dm-8 /dev/dm-9
pvcreate /dev/md0
vgcreate data /dev/md0

To create a volume:

lvcreate --name test --size 10G data
mkfs.xfs -f -ssize=4k -isize=2k /dev/data/test
mkdir -p /export/data/test
mount /dev/data/test /export/data/test