Re: [MLUG] Reasonable hard disk throughput numbers?

[Phillip Kelchen <EMAIL:PROTECTED>]

On Tuesday 04 April 2006 04:20, Mark Rages wrote:
> Hi,
>
> I'm building a system that needs to record a high bitrate stream to hard
> disk.
>
> I've got two drives in a RAID0 configuration.  My test script is to
> 'dd' 10G of /dev/zero to a file.  I divide the time required to do
> this by 10*1024, which gives me the throughput in MB/sec.
>
> [EMAIL:PROTECTED doc]$ time dd if=/dev/zero of=~/largefile
> bs=$(( 2**12 )) count=$(( 10*2**18 ))
> 2621440+0 records in
> 2621440+0 records out
>
> real    2m12.032s
> user    0m0.676s
> sys     0m39.966s
> [EMAIL:PROTECTED doc]$ python
>
> >>> (10*2**(30-20))/(2*60+12.032)
>
> 77.55695588948133
>
> On my RAID0 setup I am seeing 77.56 MB/s (as seen above)
> On my RAID1 setup I am seeing 77.63 MB/s. Essentially the same bandwidth.
>
> Does this seem like a reasonable bandwidth?  I naively assumed that
> RAID0 write performance would be greater than RAID1, because twice as
> much data is written with RAID1.
>
> Hardware information:  The disks are Hitachi Deskstar 7K80 7200 RPM
> SATA disks, going to an Nvidia-chipset motherboard with a dual-core
> AMD64.  OS is Fedora Core 4.  RAID is Linux native.  Filesystem is
> ext3.
>
> A little research finds the drives' datasheet, which calls out the
> sustained data rate as "61.1 - 29.6 MB/s (zone 0 - 29)".  I suppose
> this factor-of-two variation means the bandwidth is higher at the
> outer edge of the disk and smaller towards the center. ( datasheet is
> at
> http://www.hitachigst.com/tech/techlib.nsf/techdocs/5AB985BA152021C686256CE
>800718F09 )
>
> Incidentally, it would appear that this spec (which happend to be
> *the* most important one for multimedia applications) is omitted
> entirely on most drives' datasheets.  The closest you can hope for is
> "Media transfer rate (max)", which is specified in megabit, because
> that makes a bigger number, and padded by a unknown factor for error
> correction and the like.  Instead, the manufacturers like to tout the
> 3 Gb/s transfer rate dictated by the SATA standard.  Edward Tufte
> might say the data-to-ink ratio is pretty low in these "data"sheets.
>
> Regards,
> Mark
> EMAIL:PROTECTED

RAID is better known for read than write speeds. I don't know how fast you 
need to have the data written, but if it's really fast, you may do better to 
have a fast 15K SCSI drives. If I remember correctly, peak write speeds on a 
147GB 15,000 RPM Maxtor Atlas were near 100MB/sec and the slowest was about 
60-65MB/sec. If you need faster than that, you don't have much of an option 
other than putting several of those fast HDDs in an array, and so would 
getting a 3rd-party RAID controller that sits on a high-bandwidth bus like a 
133MHz PCI-X or PCI-express x4 or x8. But that's some serious money right 
there and the gear is server-grade and not consumer-grade. 

A cheaper option would be to use 500GB SATA hard drives or the 74GB or 150GB 
WD Raptor 10k SATA drives, depending on how much data you need to store. 
These are a lot more affordable and will perform between your 80GB setup and 
the 15k SCSI setup but cost $150-300 per drive instead of at least twice that 
just for the HDDs and then you need at least an SCSI controller to do SCSI on 
your board and those are expensive too. Again, how fast do you need to go?

Also, XFS is supposed to be the best "big file throughput" file system. Its 
only downside is that it needs to be cleanly umounted or it will become 
corrupted. So you would need a UPS or something to ensure that you can 
cleanly umount the disk if the power goes off. 

I hope this helps.

Phillip 

_______________________________________________
members mailing list
EMAIL:PROTECTED
http://mlug.missouri.edu/mailman/listinfo/members