New ZFS Server

I bought a new server to replace my Dell PowerEdge R710. The new server is an HP DL180 G6. Equipped with E5620 CPU and 48GB of RAM.
IMG_0604

The server also came with 12 300GB 15,000 RPM SAS drives. These drives will be split between the server backplane and a Dell PowerVault MD1000 to increase throughput.
IMG_0605

The new server will also act as a backup server, so I’ll add 3 3TB drives in RAIDZ-1 config.

First order of business is to replace the stock HP P400 RAID card with a non-raid HBA. For this, I’ve chosen the popular IBM M1015 SAS card. It’s a great 6Gbps PCIe 2.0 card that can be easily reflashed to plain HBA mode.
IMG_0606

I’ve also added a temporary Intel PRO/1000 Quad port NIC for Round-Robin iSCSI. In the next few weeks this will be replaced with 40Gbps Infiniband card. But that’s not ready yet.
IMG_0607

The stock RAID card has the SFF-8087 connector in the back of the card, the M1015 has the connectors on top.
IMG_0608

Fortunately HP provided enough slack on the cable to reach to the top of the new card.
IMG_0609

Before the M1015 can be used with ZFS (NexentaStor in my case). I need to reflash the card to “IT” firmware. The process is relatively simple.

Download the IT firmware here – This should be compatible with any LSI SAS2008 based card.

1. Extract the contents of the RAR file to a DOS bootable USB Flash drive.
2. Boot the server using the flash drive. Make sure only one card is connected.

3. Clear the firmware from the card

> megarec -writesbr 0 empty.bin

4. Erase the flash

> megarec -cleanflash 0

5. Reboot the box.
6. Flash the new IT Firmware to the card

> sas2flsh -o -f 2108it.bin -b mptsas2.rom

7. Enable the card’s IT mode. “500605bxxxxxxxxx” SAS address from sticker on the card without dash or quotes.

> sas2flsh -o -sasadd 500605bxxxxxxxxx

8. Final reboot and the card is good to go.

Last step is to rack up the server, populate the drives and install NexentaStor.
IMG_0613

Once NexentaStor is installed. Create the proper DataSets and we’re done. In current config running 14x300GB drives in Mirrored groups giving me about 2TB of usable, high performance storage.
nexenta

Leave a Reply