Thoughts on Drive Pooling and RAID

As an IT systems engineer with decades of experience, I've seen nearly every scheme imaginable to increase the performance of computer storage and reduce the chance of data loss to an acceptable level for an individual or company.  Nearly everyone will have a unique requirement and situation, whether that is constrained by budget, industry regulations, technical support requirements, or specific business needs.  With such a complex set of requirements, there is a rich field of technology dedicated solely to the science of mitigating against the inevitable failure of storage components.  Today, there are many more solutions than I will mention, so instead of typing a research paper, I will share with you some of my experiences and the preferences I have come to adopt when recommending solutions for these challenging and very important issues.

Introduction to RAID

First, a little background.  RAID stands for Redundant Array of Independent Disks and has been the primary technique used to aggregate the storage of multiple disks and present them to the Operating System as a single volume with certain characteristics.  RAID can be optimized for the highest level of performance, and it can be optimized for higher and higher levels of redundancy.  RAID volumes require a RAID controller to coordinate the data access to and from the array of disks, and these controllers vary in quality and expense vastly.  Most mid to high end motherboards for the past 5-10 years have included a built-in RAID capability of some sort or another.  These are good enough for home and small business use just as long as adequate planning and forethought goes into the decision to choose the cheapest option and recovery process that would be required should something fail.  Most people are concerned about the failure of a hard drive, but I have seen the motherboard fail as well and this is certainly an event that could be disastrous without backups available.  Even high end RAID cards in expensive servers can fail, and when that happens you can't always go out and buy a replacement and expect it to work.  A competent IT provider would test failure scenarios beforehand on replicas of production equipment to validate and help guarantee a successful business continuation plan.  That plan could be a recovery from a backup, or it could be the successful rebuild of the RAID without any data loss.  Every failure happens a little differently and some can be easily recovered, others not so much.

Here is a list of the most prevalent RAID levels:

RAID 0 - mirroring.  This involves two drives and a strict mirroring of all reads and writes.  There is no performance loss or gain but you are limited to two drives and lose one full drive to the mirror. This is very easy to recover as either drive will work independently should the other fail.

RAID 1 - performance striping. Data is written across all the drives in the array sequentially, so the first block goes to the first drive, the second block to the second drive and so on.  This is called data striping and it increases the read and write performance in a linear fashion up to the maximum throughput of the RAID controller itself.  Any number of drives can be used in a RAID 1 performance stripe, but the failure of any one of the drives will corrupt the data across the entire array making it useless and virtually unrecoverable.  This can be great for a caching drive or a scratch drive that is just used for holding data while it is being processed, but the more drives you assign, the greater the chance of a total failure.

RAID 10 - Striped Mirrors.  This RAID level adds performance and scalability characteristics of RAID 1 to the reliability of RAID 0 by taking an even number of drives and mirroring each of them, then striping the mirrors to create a single large volume.  Each member of the stripe can tolerate a single drive failure, so the volume could lose up to half of its drives and still function and recover, just as long as each mirror still has a good drive available to feed the overall RAID 1 performance stripe.  The downside is you lose half of your drive space to the mirrors. The upside is you gain half the performance of a RAID 1 stripe.  8 drives would give you the performance of a 4 drive stripe.  For this reason, RAID 10 is generally recommended for databases, big data, AI, machine learning, and other IO-intensive applications.

RAID 5 - Distributed Single Parity.  This RAID level has a long and sordid history mostly due to the false sense of security it offers on paper. The lack of understanding has given RAID a bad name and even today, most serious admins will not touch RAID 5.  That said, the logic is sound.  RAID 5 takes an array of disks and then reserves a small amount on each disk to store parity information about the other volumes.  This parity can be used to reconstruct a failed drive, since the loss of any single drive can be completely rebuilt by the collective parity data distributed across the volume.  This is done at the expense of some performance, and here is where you really get the value out of an expensive standalone RAID card that can mitigate the performance loss due to calculating the parity information and processing extra writes to the array.  The best part is, you only lose about one disk's worth of data to the parity, regardless of the size of the array.  If you have 8 drives, you still get to keep the storage of 7 of those drives, making it a much better deal than RAID 10, even though it is slower.  The nasty secret RAID 5 holds is that rebuilding a lost drive from parity data can take days of 100% utilization to finish.  There are so many horror stories of this processes triggering a second drive failure due to the taxing nature of the rebuild process, and that can be catastrophic. Data recovery companies earn their thousands of dollars per event cover charge reassembling data from corrupt RAID 5 arrays every day of the year.  A popular way to mitigate this rebuild time is to use a hot spare to sync the parity data is quickly as possible. This does cost another drive, but it does not incur the performance hit that RAID 6 has to deal with, which I'll mention next.

RAID 6 - Distributed Double Parity.  This is just like RAID 5, except the parity information is stored twice or different drives. The overhead this introduces is significant, and practically requires you to use a standalone RAID card, which is why you don't ever see it on built-in RAID solutions on motherboards.  RAID 6 is tolerant of a double drive failure, making it perfect for the large hard drives used today and the inherent problem of rebuilding after a failure.  The second layer of parity ensures that the RAID will keep going even if a second drive fails during the rebuild. The storage cost is about two drives from the array, so an array of 8 drives would yield about 6 drives worth of usable storage. RAID 6 is generally used in archival servers where speed is secondary to the size of the array, and technically it is slightly more fault tolerant than RAID 10 since any two disks in RAID 6 can fail, where a perfect storm could take down a RAID 10 array with the loss of two specific drives.

RAID 50 and 60 - Striped RAID 5 and 6.  These complex schemes can be difficult to wrap your head around at first, but the effect is increased speed and reliability, just like RAID 10.  These are used in large servers with dozens of drives.  Say you had 24 drives in a RAID 50.  You could decide to have three RAID 5 arrays of 8 drives each striped together, giving you the performance gain of a 3 volume stripe and slightly increased resistance to data loss, since each RAID 5 could tolerate a single drive failure. This means you could lose a single drive out of each RAID 5 array and the overall RAID 0 stripe would remain healthy.  Lose a second drive out of any RAID 5 array however, and you would lose the whole thing.  RAID 60 gives you the second parity drive at the cost of more storage and an additional performance hit.  To counter this, a 24 drive RAID 60 might be split into two RAID 6 volumes of 12 drives striped together, sacrificing a total of four drives to parity.  RAID 50 is generally used in large pools of SSDs, since the rebuild times are much faster and the resiliency of an SSD is orders of magnitude higher than that of a conventional hard drive.  Both of these schemes are extremely taxing and for this reason generally available only on high-end RAID controllers with dozens of ports.

FreeNAS with ZFS

Software RAID - Some of the more popular schemes include ZFS and Btrfs.  Long ago, software RAID was considered to be inferior to  hardware RAID due to the performance loss due to running the RAID at the OS level.  Any benefit of a specialized RAID controller card is lost, and all computations consume some CPU resources instead.  On older single-core computers, this was a big hit, especially with the slow hard drive connectivity compounding the problem.  On a modern system, however, a software RAID makes a lot more sense and allows for advanced error checking, data caching, and improved reliability provided the administrator is knowledgeable and prepared when it comes to recovery from drive or system failure.  Software RAID recovery is notoriously more challenging when a drive fails or a data volume needs to be rebuilt.  Most of the time it works fine, but there are always edge cases that tend to scare away many professionals without a deep understanding of the inner workings of the software RAID scheme.  Btrfs was originally designed by Oracle and shows a lot of promise as well, but it is not as well established in the industry.  ZFS has its roots in BSD Unix and is considered a mature standard by comparison with very successful projects like FreeNAS under their belt. Now that ZFS has been ported to Linux, Btrfs may take a back seat unless it really proves it has something better to offer.  Keep in mind, neither of these schemes are compatible with Windows at this time, and the built-in software RAID that Microsoft provides for Windows is widely regarded as a poor performing and technically risky proposition versus hardware RAID.
In short, a ZFS software RAID will be faster, cheaper, and carry more guarantees of healthy data over the long term versus hardware RAID, provided there is adequate technical support in place to handle problems. 

StableBit DrivePool for Windows

Drive Pooling - Also called unRAID, this refers to an alternate approach to software RAID which works at the file system level instead of the block level of the hard drives.  Specialized drivers work at the same level as the OS kernel and file system to prevent file locks and provide levels of caching to improve performance.  The overall effect is the ability to assign multiple drives of various sizes and performance characteristics to a unified pool of storage, allowing the drive pooling software to intelligently place files on faster volumes like SSDs as needed to boost performance.  It also allows files or directories to be tagged for redundancy, so that data will automatically be stored on multiple drives to protect the data in the event of a failure.  Many people prefer drive pooling for the transparency into the scheme as it is easy to see under the hood and optimize by adding storage on the fly.  Unlike a software or hardware RAID, a volume can be expanded simply by adding additional drives to the pool.  A true RAID needs to be backed up and reformatted in most cases when additional storage needs to be added, making it a weekend project to properly perform an upgrade.  Windows has a built-in feature called Storage Spaces which provides a very simple drive pool, with no options for redundancy.  You would use RAID to get the redundancy you need, and each RAID volume can be added to a drive pool, just like individual drives.  DrivePool by StableBit is an excellent and easy to use drive pooling software with all the features I described above including redundancy and performance optimization.  They offer a free trial and a full license for $30 which I highly recommend and have implemented quite successfully many times before for home users and businesses alike.  Oh, one more thing about DrivePool.  The files are stored in a standard file system, so data recovery is a breeze if you ever needed to recover data from individual drives without rebuilding the pool.  This translates to peace of mind when recommending a solution to someone without a great deal of technical support at their disposal.