Right now I am trying to spec out a storage area network (SAN) which will handle the data generated by our 454 instrument and potentially other sequencing instruments down the line. A recurring theme when dealing with the major vendors is the major discrepancy between the unit cost of hard drives – a 1TB drive can be had for as little as £54 these days – versus the cost of deploying enterprise-grade storage. Having gathered quotes from Dell, HP and IBM I have been surprised to find that you don’t get much change from £10,000 for a 12TB SAN populated with disks. That works out at a whopping £1000/TB! Even scaled up to 60TB, the HP MSA solution doesn’t manage to get much below £500/TB. That’s why I was quite interested to read Petabytes on a budget on the Backblaze blog where they demonstrate how they build custom rack-mountable storage devices, each capable of storing 67TB and manage to get the cost down to $117 (£72)/TB – a very small premium on the cost of disk. Now, I dare say most bioinformatics readers won’t fancy building this themselves at work, it would take some time to source the components and build it all from scratch. Plus you won’t get any fancy disk configuration utilities and you don’t get any block-level high I/O access via fibre-channel or iSCSI, only networked access over HTTP. But these may be luxuries we can’t afford when Solexa machines are routinely chucking out 90 gigabases of nucleotide read data per run (probably 200 gigabytes uncompressed on disk with quality scores). The article also helps answer the commonly posed question “so why don’t we just store our data in the cloud?” – well, the provided graph goes some ways as to explaining why Amazon S3 hasn’t taken over just yet.
Hi Nick,
I thought your blog was very interesting… The one thing I would be a
bit careful of when going for the blackfire system (if that’s what
someone was going to do) would be the quality of hard disks. I would
expect that from HP/IBM/Sun you would get a better grade of disk drive
than if you bought the components from somewhere like overclockers.co.uk
and built it yourself. Also, you’d miss out on a lot of disk management
systems (software and hardware) that are easier to integrate into a
computer network solution.
Bhima
Bhima, yes, definitely good points (although disk management software often incurs additional license fees above that which I’ve quoted). About the hard drive issue, I’ve always wondered what the real difference is between “enterprise-grade” hard drives which have a significant mark-up associated and consumer grade disks. I believe the major manufacturers do spend time doing soak tests and QA on them but presumably the underlying electronics and manufacturing process is identical. Is it just a scam? Either way, no disk can be relied on to function and you would need at least RAID protection (RAID-1 or RAID-6 would give some comfort, although you are still exposed to a single disk failure associated with unrecoverable read errors on the other volume). Given that no hard drive can be relied on to keep working, perhaps it is better to deal with things that way.
There is a detailed critique of Backblaze’s units in this Cloud Computing newsgroup thread: http://groups.google.com/group/cloud-computing/browse_thread/thread/9dcdec496423cfa1 (see the message by by Robert Peglar).