Geek question: RAID array performance

[andygates]

We've got a big data store. It's currently built as a single max-size 2Tb RAID 5 array over 11 disks. It performs like a three-legged dog and each of the 300Gb disks has about 75Gb unused (because that would take it over the 2Tb SCSI RAID limit).

We need to rebuild it. We've been advised that volume sizes over 1.5Tb are dogs under SCSI RAID so we're keen not to do that. Our options are:

* RAID 10, 10 disks+hotswap, total volume space 1.5Tb, good write performance.

* RAID 5, 7 disks+hotswap, total volume 1.6Tb, but it feels wasteful of all these lovely disks.

* Two smaller RAID 5 volumes, 4+hotswap and 5+hotswap, giving us .9 and 1.2 Tb respectively. The volumes should behave better, because they're smaller and on fewer disks, but will this just move the bottleneck up to the servers' RAID controller?

Gurus, your wisdom is much appreciated!

Comments

[gedhrel.livejournal.com]

Incidentally the raid-5 performance issue comes down to the fatness of your reads and writes. For large reads, having lots of disks is great because you can stream data off all of them in a go. But for file service you need stripe-sized writes for this to work well, because otherwise a small write means reading from the disks that're touched plus parity* as well as the corresponding writes.

So you want your stripe size down. That means (1) think about using fewer disks per raid-5; (2) look at the average write sizes - think your fileserver should be able to tell you this - and see if you can get your stripe size to match this.

* there's a simple XOR trick that means you don't need to read the whole stripe in this case.

[andygates.livejournal.com]

Guys, you're both golden, and I'll wave this at my coworker in the morning.