Tom Northenscold Photography | Data Storage And Backup For The Enthusiast Photographer

Subscribe to my mailing list

Email Format

Subscribe To RSS Feed
January February March April (2) May (1) June July August September October November December
January (1) February (1) March (1) April May June July (1) August (2) September (3) October November December
January February March April (4) May (2) June (1) July August September (1) October November December
January February March April May (1) June July August September October November December
January February March April May June July August September October November December

Data Storage And Backup For The Enthusiast Photographer

October 11, 2014  •  Leave a Comment

You may have heard this many times, but it bears repeating--if you don't have your data stored in three places, with one of them offsite, it's not protected. The trick is how to have this happen without you having to think about it. It ought to be set it and forget it. There is one proviso to that objective though, it's important to test your recovery capability once in awhile.

It's real easy to get deep into the technical weeds on this, which is not my intent here. Others have done that before me. What I do think is important is putting forward a simple solution that works for everyday enthusiast photographers like myself. One disclosure right up front, I am a Mac user, so the first two products mentioned are Mac centric, although the architecture of my solution can certainly be replicated on other platforms.

Local Storage:

If you have a larger photo library like me (e.g. > 600 GB), then you may have this data stored on an external drive or drives. I use a Pegasus R6 RAID array from Promise Technology. A RAID array can provide some protection for a disk drive failure, depending on how it is configured. The way the Pegasus R6 RAID array is configured, it provides protection in case one of the six drive fails at any point in time. This is called RAID5. Because this connects to my MacBook Pro via Thunderbolt, it is extremely fast. Tests have shown it to be faster than an internal solid state drive. The way my Pegasus R6 is set up, I have 5 GB of usable storage out of the 6 GB total in the unit. Basically, one of the drives is consumed providing the protection for a drive failure. As it is, I'm only using about 1.7 TB of the 5 TB available.

You could certainly just get yourself a large external hard drive that connects via either USB 3 or Thunderbolt. Because I have had drives fail on me, I decided to get a solution that has some redundancy built in. I felt nervous having all my eggs in one big basket. True, if my backups are working I could recover from a failure, but having to do so is a royal pain. I know because I've had to do it. I've used the Pegasus unit for over a year now and it has been rock solid for me. The folks at Promise Technology have come out with a gen 2 version of the Pegasus R6, but the photo I'm showing is the version I own (the one on the right).

Pegasus R6 RAID ArrayPromise Technology, Inc.

Local Backup

Your local backup solution ought to happen behind the scenes without you ever knowing that it's happening. For my needs, Time Machine running on an Apple Time Capsule is the perfect solution. Time Machine backs up every hour, so at most my backup is only an hour out of date. I purchased the 3 TB version of the Time Capsule, which backs up all the computers in our household over the WiFi.

Time CapsuleApple, Inc.

Offsite Storage (Cloud):

Now comes the tricky part, how do you get a backup of everything offsite? For me the solution is Backblaze cloud backup. For under $50 a year I get unlimited back up of all the data from my MacBook Pro. This includes all the data stored on the Pegasus R6. The backups happen in the background. Depending on the upload bandwidth at your home, the initial backup can take some time, in my case, a couple weeks. Right now I've got 1.2 TB stored at Backblaze. I have tested recovering files from Backblaze and it couldn't be easier. You can recover individual files or folders via a ZIP file that gets emailed to you. You can also have your entire backup sent to you via hard drive.

Whether cloud backup works for you is largely dependent on how much data you have to back up, how much new data you are generating and how much upload bandwidth you have to the Internet. I delete upwards of 80% of the images I take, which helps me hold down the size of the backup. When I started using Backblaze with my MacBook Pro about a year ago, the initial backup was around 1 TB. My current backup is around 1.2 TB, so I've added about 200 GB over the past 12 months. The bandwidth at our house to Backblaze allows me to backup around 75 GB per day. If I were generating more than 75 GB per day then this solution wouldn't work. That's an awful lot of data though. I'd have to be shooting a couple thousand images a day on my D800 and not deleting any of them. I have never generated that much data in a day even on my most active shooting days. I would argue that for the enthusiast photographer, a solution like Backblaze is perfect.

If you're a little nervous about not having an offsite backup during the initial backup time period you could always buy an inexpensive external drive and back up your photo library on your computer and then keep that drive somewhere else until the initial cloud backup is done. That's what I did.

The last little bit in my strategy has to do with when I reformat the camera memory cards. If I've just uploaded some important files, such as a senior portrait shoot for our niece, I don't reformat the cards until I know that they are backed up out at Backblaze and on Time Machine. I will actually go our to the Backblaze website to verify that the files are out there. Then I feel comfortable reformatting the cards. If it's a less important shoot, I may wait until I know that Time Machine has backed up the images, then I will reformat.

One of the beauties of Time Machine and Backblaze is that they offer journaling, which means that I can recover older versions of files that I may have deleted inadvertently or may have become corrupt. I find this to actually be my most common recovery need, just me being dumb and accidentally deleting a file or messing it up in some way.

This three-part solution has worked really well for me. Of course now that I've written it up for God and everyone to see, I've probably jinxed myself and will have some major storage failure happen to me. I'm keeping my fingers crossed.


No comments posted.