Double Disk Failure - Follow up

Yesterday, Dave wrote about double disk failure in his blog (See Expect Double Disk Failures with ATA Drives). I agree with Dave's comment "As drives grow, I suspect that it (double protecting RAID) will become a requirement even for Fibre Channel drives." In my opinion, as the rebuild time goes beyond 12 hours, it will become requirement for all drives.

Couple of weeks ago, I also wrote about double disk failure event I encountered (See Data loss risk during RAID rebuild). And my concerns about increased exposure to data loss as RAID rebuild time is getting longer with higher disk capacity in RAID groups (See Happy New Year & Food for Your Brain). I prefer to have a Plan B if the RAID5 rebuild time is going to exceed 6 to 8 hours.

In last couple of weeks, my readers and colleagues referred me to several good articles on RAID protection. Here are two that I found really interesting and useful.

Daniel Feenberg, Things we wish we'd known about NAS devices and Linux Raid

… the majority of single drive failures are followed by a second drive failure before redundancy is established.

The fault lies with the Linux md driver, which stops rebuilding parity after a drive failure at the first point it encounters a uncorrectable read error on the remaining "good" drives.

A single unreadable sector isn't unusual among the tens of millions of sectors on a modern drive. If the sector has never been written to, there is no occasion for the drive electronics or the OS to even know it is bad. If the OS tried to write to it, the drive would automatically remap the sector and no damage would be done - not even a log entry. But that one bad sector will render the entire array unrecoverable no matter where on the disk it is if one other drive has already been failed.

We don't know what the reconstruction policy is for other raid controllers, drivers or NAS devices. None of the boxes we bought acknowledged this "gotcha" but none promised to avoid it either.

K.K. Rao, James L. Hafner and Richard A. Golding, Reliability for Networked Storage Nodes (pdf)

By using drive bandwidth more efficiently through the use of larger rebuild block sizes, we see significant improvements in reliability. In fact, the rebuild block size is a controllable parameter with the most significant impact on reliability.

No right or wrong answers to EQ tests

Do you score high on EQ (Entrepreneurial Quotient) tests like the one
posted by Guy Kawasaki? The high score doesn't necessarily mean that you will make a better entrepreneur. But, you may have a future as a writer who may become successful writing business books.

I took Kawasaki's test but with a twist. I answered eleven questions based on what popular entrepreneurship culture preaches and eleven questions based on my actual experiences. I got three times as many wrong answers when I answered based on my experiences instead of what entrepreneurship books preach.

I used to be a typical engineering guy and had no background in business. So I read a lot of books on entrepreneurship, joined associations and attended events. And only thing I can say that the preachings of popular entrepreneurship culture will not make you successful. The key to success is flexibility and considering all available alternatives.

In my opinion, there is no right or wrong answer to EQ tests, all answers are valid alternatives until one works for you.

Only if this could be an expense-paid trip to Cancun!

Today, I received this message for a Symantec event in Cancun, Mexico.

Hi Anil,

Whoever said you can't mix business with pleasure must not have heard of Symantec's new disaster recovery strategies seminar in Cancun, Mexico. The seminar will held May 14th – 19th at the Playacar Palace near Cancun, Mexico and will cover various topics including Business Continuity Plan vs. Disaster Recovery Plan, Replication vs. Backup, Data Protection Strategies, and Storage Virtualization.

This could go a long way towards easing the burden of IT professionals.:) Instead of sitting behind a cubicle listening to a Webinar or being herded like cattle into a stuffy conference room, why not get out of the office, make a trip south of the border and get the best of both worlds--training from the best on strategic planning for disaster recovery and then the reward for your hard work and efforts in IT.

Designed for IT managers, IT staff, and self-employed IT professionals, the seminar will present material with a focus on concepts and solutions rather than product-specific features.

If you or the readers of your blog are interested in more information let me know, or visit http://www.techgetaway.com.

Thanks,
Nadja

On one end, it reminds me of free heavenly trips to Hawaii, doctors get for listening to recent development in medical fields, paid for by pharmaceutical companies! On the other end, it reminds me of free hellish week at timeshare properties in return for listening to sales pitch of timeshare company.

Now, only if we could get someone to foot the bill for this trip without acting like timeshare company and treat IT professionals like doctors?