Difference between revisions of "Hooikanon storage details"
(Created page with "== Hooikanon Storage Details == The hooikanons consist of 3 Grid dCache storage units and 1 local (stoomboot) dCache storage unit. Each server has 2 drives in RAID1. Attache...") |
(→NTP) |
||
| (3 intermediate revisions by 2 users not shown) | |||
| Line 7: | Line 7: | ||
=== Servers === | === Servers === | ||
IBM POWER SYSTEM LC922 (16-CORE 2.9GHZ POWER9) | IBM POWER SYSTEM LC922 (16-CORE 2.9GHZ POWER9) | ||
| − | Mellanox MCX415A-CCAT ConnectX-4 EN 100GbE single-port QSFP28 PCIe3.0 x16 LP | + | * Mellanox MCX415A-CCAT ConnectX-4 EN 100GbE single-port QSFP28 PCIe3.0 x16 LP |
| − | 2-Socket 2U 12 LFF/SFF 4 NVMe Direct Attach Fab Assembly | + | * 2-Socket 2U 12 LFF/SFF 4 NVMe Direct Attach Fab Assembly |
| − | 2 TB 3.5-inch 7200 RPM SAS 12Gb/s HDD 512e NONSED WrtCache | + | * 2 TB 3.5-inch 7200 RPM SAS 12Gb/s HDD 512e NONSED WrtCache |
| − | Broadcom (LSI) 9305-16E SAS-3 HBA PCIe3.0 x8 LP | + | * Broadcom (LSI) 9305-16E SAS-3 HBA PCIe3.0 x8 LP |
| − | 16 GB DDR4 memory | + | * 16 GB DDR4 memory |
| − | 16-core 2.9 GHz POWER9 Processor | + | * 16-core 2.9 GHz POWER9 Processor |
=== Storage Arrays === | === Storage Arrays === | ||
| − | E5700A, 64GB Cntrlr, No HIC, 16Gb FC, 2-pt | + | NetApp E5700A |
| − | Controller: 2x 16Gb/10GbE optical onboard ports. | + | * E5700A, 64GB Cntrlr, No HIC, 16Gb FC, 2-pt |
| − | HIC, E5700/EF570,12Gb SAS,4-ports | + | * Controller: 2x 16Gb/10GbE optical onboard ports. |
| − | 4U, 60 disk drives, 12TB, 7.2k, 12Gb, Non-FDE | + | * HIC, E5700/EF570,12Gb SAS,4-ports |
| + | * 4U, 60 disk drives, 12TB, 7.2k, 12Gb, Non-FDE | ||
| Line 28: | Line 29: | ||
https://ndpfgit.nikhef.nl/ndpf/flight-rules/blob/master/flight-rules/netapp-storage-setup/netapp-storage-setup.md | https://ndpfgit.nikhef.nl/ndpf/flight-rules/blob/master/flight-rules/netapp-storage-setup/netapp-storage-setup.md | ||
| + | |||
| + | === Controllers === | ||
| + | |||
| + | ==== NTP ==== | ||
| + | The controllers log events, but as their clocks can drift out of sync with the server it can be difficult to correlate the logs with other events. It is best to set up NTP. | ||
| + | |||
| + | The configuration is found here: https://ndpfgit.nikhef.nl/ndpf/flight-rules/-/blob/master/flight-rules/hooikanon-controller-ntp-setup.md | ||
Latest revision as of 10:24, 6 September 2021
Hooikanon Storage Details
The hooikanons consist of 3 Grid dCache storage units and 1 local (stoomboot) dCache storage unit.
Each server has 2 drives in RAID1. Attached to each server is a storage array with 60 drives/device. These are set up with 4 LUNs in RAID6.
Servers
IBM POWER SYSTEM LC922 (16-CORE 2.9GHZ POWER9)
- Mellanox MCX415A-CCAT ConnectX-4 EN 100GbE single-port QSFP28 PCIe3.0 x16 LP
- 2-Socket 2U 12 LFF/SFF 4 NVMe Direct Attach Fab Assembly
- 2 TB 3.5-inch 7200 RPM SAS 12Gb/s HDD 512e NONSED WrtCache
- Broadcom (LSI) 9305-16E SAS-3 HBA PCIe3.0 x8 LP
- 16 GB DDR4 memory
- 16-core 2.9 GHz POWER9 Processor
Storage Arrays
NetApp E5700A
- E5700A, 64GB Cntrlr, No HIC, 16Gb FC, 2-pt
- Controller: 2x 16Gb/10GbE optical onboard ports.
- HIC, E5700/EF570,12Gb SAS,4-ports
- 4U, 60 disk drives, 12TB, 7.2k, 12Gb, Non-FDE
Information on how to set up the ppc64le systems can be found here:
https://wiki.nikhef.nl/grid/Hooikanon_server_set_up_(ppc64le)
Storage system setup is here (ndpfgit access only):
Controllers
NTP
The controllers log events, but as their clocks can drift out of sync with the server it can be difficult to correlate the logs with other events. It is best to set up NTP.
The configuration is found here: https://ndpfgit.nikhef.nl/ndpf/flight-rules/-/blob/master/flight-rules/hooikanon-controller-ntp-setup.md
