Physician heal thyself

You’d think I’d learned my lesson about backing up critical data, but nope, not me. In an extreme case of overconfidence and hubris, I let a Mac mini that I had set up in my house as a web server for a genealogically oriented family association go without a full backup … for almost seven years! Yeah, yeah… I had planned on doing something when I had a little free time on my hands, but that time never came.

This server was running several specialized web applications, including the genealogy of the Grinnell family, configured in such a fashion that it is very difficult to start over from scratch. Oh, and did I mention the main membership database? Yup. Also gone. At least for that one, I think I’ve got a backup of that file somewhere, and because it’s membership renewal time, the membership chair had just made printouts of critical member information. Not that this should in anyway excuse what has happened.

Last Thursday, I got an email from the association president (I am the secretary and webmaster) noting the difficulty they were having in accessing the website. When I got home from work, I confirmed the problem and decided to start with the usual method of fixing things like this — reboot the server. Well, that’s where things got REALLY interesting/bad.

When I rebooted, I got a blank screen for about three minutes, followed by an alternating blank folder icon and a Mac OS X icon. I tried inserting a diagnostic CD, and it wouldn’t mount. Now, it won’t even eject!

That may actually be good news. If there’s a hardware problem (something on the computer itself and not the hard disk drive), I may be able to put the drive into another similar vintage 1.25GHz G4 Mac mini (many to choose from on eBay) and resume operations as if nothing went wrong.

If it’s the drive, my only hope is that I can pull the drive from the current Mac mini, connect it to a special adapter cable, and attempt to run some very sophisticated data recovery tools (Prosoft Data Rescue) to get the data onto another drive.

So, in preparation, I made a trip to my friendly neighborhood CompUSA store and picked up a new 80GB IDE hard drive (very inexpensive for one of these older model Macs), and a USB-to-hard-drive adapter cable. The goal is to try to recover directly to this new drive, which, as I said, would replace the drive currently installed inside this Mac mini.

Next, if I can indeed recover the data, I will pull one of my older external hard drives out of its current service, and do a full copy of this recovered data to the external drive using a utility like Carbon Copy Cloner or SuperDuper, both of which are free, and which create bootable backups. This means that if the internal drive fails again, I can immediately switch over to the external, boot from it, and only lose whatever data may have been added since the previous backup.

I won’t know how much damage has occurred for a few more days, but suffice it to say, I have let down the organization I have been supporting with my technical expertise for over 20 years, and if things have gone as bad as I think they may have, I may have caused the loss of 6-9 months of data entry work performed by our genealogy committee chairperson in preparation for my task of writing and editing the next edition of the Grinnell family genealogy, a tome estimated to be over 2,000 pages, not to mention the entire website content, and the membership database. Here’s hoping it won’t come to that.

So, what have we learned here? First and foremost, make sure your data is backed up regularly, and if it absolutely, positively has to be available 24/7, make sure you have created at least one, and maybe even two bootable backups of your work. Next, if the data is that critical, get it off your little computer at home, and if you are using software that permits it, move it to a hosted service, though even there, no matter whether they do regular backups or not, you must do your own backup of your critical data often. Ultimately, if you are the caretaker of the data, it’s your responsibility to ensure that it remains available to those who need it, when they need it.

This is has been a very painful exercise in poor judgment that could easily have been avoided, and it could have all been done automatically with a few simple scripts.

Excuse me while I flog myself a little more before bedtime.

=================================================

That was the story as of the early part of February, soon after the disaster began to unfold. Next, my dear friends, is the rest of the story:

My tech friend (president of the local Mac user group), known as “JG” (short for a very long German name), an interesting fellow who used to own several Apple stores in the Washington DC area (he sold his business just before Apple built their first Apple Store, directly across the street from his store, at Tysons Corners, near McLean, VA), helped me out quite a bit. I found an equivalent Mac mini for under $200 on eBay, which had an identical 40GB Seagate drive. The problem, however, was much, much worse, as he discovered that the drive in my Mac mini had suffered a terminal head crash. “JG” took the drive apart and found deep gouges on the media surface. Oh, did I mention I didn’t have a backup?

Anyhow, the good news (yes, Virginia, there was at least some good news) is that I was able to locate about 80% of the content in various forms and formats on other computers around the house, and so I have set up an account with a webhost (hostmonster.com) and am in the process of building a Joomla!-based website for grinnellfamily.org. The person responsible for keeping up the online genealogy database (40,000 names, 15,000 families, etc.) kept a second copy of the database in Family Tree Maker (finally a Mac version after an almost 20 year absence!!!!!), so I should be able to recover all of that, put it back into The Next Generation of Genealogy Software (an interactive, web-based genealogy database), and tie it into Joomla! with single sign-on integration (one login works for both elements–premium content on the website and select access to the genealogy). The membership database, in FileMaker Pro, after we recover information on the most recent 30 or so members, is going to a FileMaker Pro host for 20 bucks a month.

Boy, did I learn a big, painful lesson! Backups are good. No backups are bad.

Oh, and for the rest of my computers…

I’m putting together a killer media server system, centered around a  2.53 GHz Core2 Duo Mac mini and a Synology DS-509+ 7.5TB NAS (using Plex as the engine behind it all). Then I thought to myself, what happens if the RAID5 goes really bad and I lose two disk drives? So, I bought the Synology DX5, a secondary 5 drive box which holds five 2 TB drives in a RAID5 configuration. The DX5 connects to the DS-509+ with an eSATA cable, so transfer speeds are pretty snappy.

I’ve already ripped…er…backed up over 300 movies and hundreds of TV shows and am still in the process of ripping and saving the TV shows as H.264 MP4 files that Plex requires if I want to use their really neat TV show features that scrape content from TVDB to make the whole media server experience that much more enjoyable…but I digress.

I now have somewhere around 4TB of content converted and there’s more to come. I now have a nightly backup to the DX5, but because I planned it with more capacity (10TB on 5 two-terabyte drives in a RAID5 configuration works out to be roughly 7.5TB of usable storage, I’ve got a fair amount of reserve capacity, as the 7.5TB NAS really has about 5.5TB of usable space), I can back up my laptop and my desktop to it, using a copy of NTI Shadow Backup that I reviewed for MyMac.com a few years ago, and another copy I bought for myself. It works great. The first backup between box one and box two took about 36 hours. The backup of the G5 (about 400GB) over my wireless network took about three days….

The last few weeks have been, at the very least, harrowing, frightening, sad, terrifying, humbling, humiliating, and more. I can’t think of a more perfect time to increase my medication.

(Portions reposted from Palm Beach Business.com with their kind permission)