Tumbleweed Rants Stefano's World http://tumbleweed.org.za/node/58/atom/feed 2008-01-02T19:55:10+00:00 Horrific performance with 3ware RAID http://tumbleweed.org.za/2007/02/16/horrific-performance-with-3ware-raid 2007-02-16T08:01:54+00:00 2008-01-02T19:55:10+00:00 tumbleweed I’ve been enjoying our server at UK2.net. It’s a pretty speedy machine (although a little light on RAM - I suspect that they don’t want people running Xen), and it’s connected to a fat pipe. But I’ve been experiencing a lot of bad lockups.

I traced the problem to postmaping the uceprotect.net RBL file. They recommend that you rsync this file from them, and then postmap it into a fast lookup database for postfix, rather than using their DNSRBL service. But running the postmap was taking my box 40 mins. The same operation, on a loaded, lower-spec, 2 year old server took 2 mins (yes this server also has RAID1 on the volume concerned). On my UK2 box, while the postmap was running, the machine became totally unresponsive, and it could take a minute or two to log in, serve a web page, or even execute a basic command like ps.

Clearly something wasn’t right. And it was something in the IO system. The only answer is the 3ware RAID controller. (It’s a 8006-2, doing RAID-1) I know these controllers have a big buffer, so I looked up the 3ware website, for tuning guidance. I followed it to the letter, and things didn’t really improve. I tried the deadline scheduler, and tweaking the buffers, but it only got marginally better.

Personally, I’ve always used software RAID, even for RAID-5, and I’ve never had bad performance like that. And having the RAID in a portable format has really helped with recovery in the past. I understand that Windows monkeys have to use hardware RAID (because their software RAID sucks so much), but is this kind of performance normal?

I’ve asked UK2 to chuck my controller and give me software RAID :-)

Update

I’ve now got software RAID 1, and postmap runs in 25 seconds. That’s what I call a 60x speed improvement :-)

Oh, and the system is totally responsive while the postmap runs.

]]> I’ve been enjoying our server at UK2.net. It’s a pretty speedy machine (although a little light on RAM - I suspect that they don’t want people running Xen), and it’s connected to a fat pipe. But I’ve been experiencing a lot of bad lockups.

I traced the problem to postmaping the uceprotect.net RBL file. They recommend that you rsync this file from them, and then postmap it into a fast lookup database for postfix, rather than using their DNSRBL service. But running the postmap was taking my box 40 mins. The same operation, on a loaded, lower-spec, 2 year old server took 2 mins (yes this server also has RAID1 on the volume concerned). On my UK2 box, while the postmap was running, the machine became totally unresponsive, and it could take a minute or two to log in, serve a web page, or even execute a basic command like ps.

Clearly something wasn’t right. And it was something in the IO system. The only answer is the 3ware RAID controller. (It’s a 8006-2, doing RAID-1) I know these controllers have a big buffer, so I looked up the 3ware website, for tuning guidance. I followed it to the letter, and things didn’t really improve. I tried the deadline scheduler, and tweaking the buffers, but it only got marginally better.

Personally, I’ve always used software RAID, even for RAID-5, and I’ve never had bad performance like that. And having the RAID in a portable format has really helped with recovery in the past. I understand that Windows monkeys have to use hardware RAID (because their software RAID sucks so much), but is this kind of performance normal?

I’ve asked UK2 to chuck my controller and give me software RAID :-)

Update

I’ve now got software RAID 1, and postmap runs in 25 seconds. That’s what I call a 60x speed improvement :-)

Oh, and the system is totally responsive while the postmap runs.

]]>