Drobo and Amazon S3, possibly the best backup partners!

Many months ago I decided that I wanted a decent backup system for my massive photograph library. I’ve been using Apple Aperture to manage my photos, and the library has been growing and growing. Our main computer is my work station and the family computer, so the wife has been adding to the iPhoto library as well.

This year I upgraded my camera from the 6 megapixel, Nikon D40 to the 12.3 megapixel, D300. That’s over double the amount of megapixels in each photo now. This converts into much larger file sizes too!

I had the iPhoto library local on the Mac Mini back then. The Aperture library was being managed on an external LaCie harddrive. That system worked, but had no backups. If I lost a harddrive, I’d have been sunk. So I committed to reorganizing my file management system, so that I’d have an increased likelihood of not loosing data. The Drobo was my solution. With its built-in redundancy, I was able to also be free to upgrade, at minimal cost. Earlier this year, I got the basic Drobo, and two 500 Gb drives. This didn’t give me an outrageous amount of space, but it got me started. I moved my iTunes library, iPhoto library, and my complete Aperture photo library to the Drobo, freeing up space locally and a harddrive. I re-tasked the harddrive for Time Machine backups, and was generally more pleased with myself!

As time went on, I began to desire an off-site backup. What if my house burned down? In fact, what does everyone want to get, when you see the hills of California on fire every year? The people are frantic about getting into their homes, to retrieve their photo albums. That’s really all that the insurance company can’t replace.

We stopped shooting with film, and getting regular prints made years ago. So, the entirety of my 4 kids’ childhoods is contained, as photographs, on that stupid Drobo! If I lost that, I’d freak out!

It didn’t take long to convince myself that getting my important content backed up, off-site, was vital. In fact, my advice would be to get your important data off-site, before you set up a local backup system! If I lost everything, I could simply log into my account, via FTP, and one very long download later I’d have it all. This kind of off-site backup is really best described as a “cloud-based”. The regular 0ff-site backups are generally more expensive.

I looked at a wide variety of options. I host this site with Dreamhost, and thought of them first. Naw, they don’t like you using your storage for personal stuff like that. You can do it, but it’ll cost you if you’re caught. I really don’t want to do it like that. Then I looked at some of the others. None really compare to Dropbox, so I stopped looking. I signed up, and ran a test.

It’s pretty cool that you can backup as easy as just dropping your stuff on a folder. That’s really easy! Most people will get what they need with Dropbox. However, I really wanted to setup my backups through FTP, allowing me to setup an Automator action to do it regularly. Yeah there’s other ways, but that was what I wanted.

From the beginning, I had decided that I was willing to pay for this service. DropBox is cool because the first 2 Gb is free. Most will be fine with that. Me, however, I’ll be needing much, much more. The next step with DropBox is $9.99 for 50 Gb, then $19.99 for 100 Gb. Neither of those are enough either.

Poking around, I found an article that made me realize that most of these cloud-based backup solutions are using Amazon S3 for their data. They’re just writing cool apps to make it easy for regular folk. Well guess what?! I already have a sweet app for accessing Amazon S3! It’s called Transmit, by Panic Software. I’ve been using it for all my FTP needs for several years. I love it! It didn’t take a genius to find a simple article telling me how to set up Transmit to access my Amazon S3 account. Oh yeah, I had to set that up too. Being an Amazon customer, from their massive store, I already had an account, which made the whole thing much easier.

Amazon S3 is:

Amazon S3 provides a simple web services interface that can be used to store and retrieve any amount of data, at any time, from anywhere on the web.

Basically, Amazon.com has built one of the most robust websites in the world. When have you seen Amazon.com going down? It’s even older than Google! All of their data is spread all over the world, so if a server goes down in Bakersfield, and I was viewing content on that server, they just reroute my viewing experience to another server, somewhere else.

So now they’re just providing that kind of data storage to me! Here’s the catch…wait for it…IT’S CHEAP! Really really cheap! The first day, I uploaded one folder, with just a small amount of data; 2 Gbs worth of photos and some other stuff. I can’t remember what the total amount of data was, but is wasn’t much more. They charge for the upload and then the storage. My total cost for that month was $2.57. That’s it!

So, on December 1st, I decided to jump in with both feet. I set up the whole back up! Right now, I’ve backed up my entire photo library, iPhoto and Aperture. I’ve also included all my RAW files. It’s all there! The first big back up took like 3 days, non-stop! That’s mostly because my ISP is Comcast, and they suck. It would drop me occasionally, and Transmit would be forced to re-log in, and pick up where it left off.

Then on December 7th, they announced a sweet price drop, and they’re waiving the upload fee, until June of 2010! SWEET! Now is the best time to jump on that price drop! So I can upload all sorts of stuff, and I’m only paying for the storage right now. That’s 15¢ per GB! They drop the price at several tiers, the first one at 50 TB. That’s $154/mo. for 1 terabyte of storage!

Since most people won’t be backing up a whole terabyte of data, here’s a more normal look; 10 GB = $1.50/mo.

So, where am I now? I’ve set up the account. I’ve chosen the folders I want backed up. Oh, I forgot to tell you that Transmit can easily synchronize a folder, without rewriting content that’s already there. Maybe that’s normal for all FTP apps, I don’t know. It’s cool for me! This way I don’t have to worry about eating up bandwidth, when Amazon starts charging for that again.

I set up a few “Favorites” in Transmit, and then I set up a simple Automator Action. I saved the action as an .app. Lastly, I set up an event in iCal, calling my new application in the alarm.

Now I’m good to go! I have it syncing each Saturday evening, and I don’t have to think about it anymore. Below are a few screenshots of the whole process…

In Transmit, input all the important information about your new Amazon S3 account.

In Amazon S3, top most folders are called "buckets". I'm naming my main backup folder, "trevor-photos"

I chose to "Upload" my data and to have it "Update" it. This will keep my bandwidth down.

Here's the Automator Action. It's pretty self-explanatory. Make sure to save it as an Application.

I have incited a riot

Yesterday I was in a special mood. I don’t know why, but I was grouchy for a bit. However, as usual, I wander the internets reading blog posts and articles.

Apparently, I stumbled upon one of my dailies, and posted a comment. Well, I guess my comment was a bit off topic, or something. I was also critical. I know, I know, that’s out of the ordinary for me. Usually I’m such an amicable kind of guy.

Well, the blog post in question was over at Scott Kelby’s blog. The article is titled, Camera Raw, Bridge, or Lightroom? Basically, Scott had a question about the confusion over using the RAW converter in Photoshop, Bridge, and Lightroom. Rather than answer the question directly to the asker, he blogged it. Makes sense. I’d do the same. In my special mood, I read the article and was curious about why Scott didn’t mention anything about Apple’s Aperture.

Photoshop Insider » My Apple Aperture Conspiracy
Uploaded with plasq‘s Skitch!

Now, let me be clear. I was a bit harsh in my comment. I was being sarcastic. I guess it was enough for Scott to reply with a blog post today.

However, I am still curious about Scott deciding to take a specific question, from one fan, and answer it generally for the public. Now, if the original question happened to come from a Windows user, then a Windows based answer is appropriate. But, to bring the discussion to a more open forum, and attempt to become more general, why not include a wider set of variables to the question?

Why not consider the RAW converters in most of the industry standards for photographers? I’m sure if you included Apple Aperture, Nikon NX, and maybe an open solution, he’d have generated a much more curious dialogue. I expect a more narrow conversation from lower level bloggers, like myself. But from the top bloggers, I expect more.

So, I apologize for my attitude. And I wish I had posed my question differently. However, I am still curious. I understand that Scott has made his brand loyalty choice. That’s fine. However, when reading his awesome book, The Digital Photography Book, you don’t have him telling you all about his Nikon cameras. Yes, he prefers Nikon. But, his book looks more generally at the issues facing all photographers. Scott is a great writer and instructor. Why can’t I expect that a blog at ScottKelby.com would provide me with the same high level of fairness in evaluation?

On the road to some good photo editing and management

Yesterday I had the great opportunity to shoot another architectural photo shoot. I shot two residences and earned enough to acquire a copy of Apple Aperture 2.0 and step a bit closer to my first real strobe kit. I’ve been shooting with my one speedlight and renting more when I need them. Yesterday, I rented a pair of genuine dyna-lite strobes, from Samy’s. I’m very grateful to have been able to shoot with those nice lights. I’m sold on strobes!

DSC_9475DSC_9446DSC_9473

Additionally, I tested out Aperture 2.0, with their 30 day trial, over the last month. My trial expired a week back, and I was actually sad when it did. Not that I don’t have decent alternative asset management  and editing. I have been using a combo Adobe Bridge and Photoshop system. It has worked well for almost a year. Unfortunately, upgrading to Adobe’s Lightroom is really the professional step they want you to make. I trialed Lightroom late last year, and I didn’t like it. I didn’t like the modular system. I didn’t like how it performed on my slower Mac Mini. I simply didn’t like how it flowed.

I had played with Aperture in version 1.0, and quickly didn’t like it. But I think the biggest killer was that it was dog slow. Then came version 2.0.

The most important change to 2.0 was the speed enhancements. It’s now quite smooth, even on my slower Mac Mini. I can’t afford to upgrade the Mac right now, so a software solution that can effectively operate on it, is more than likely going to get my vote. In the next week or so, I’ll review what I do with Aperture, and show you my workflow. I have noticed that many more people use Aperture than I originally thought. So I’d like to help increase Aperture’s awareness just a bit.

So yesterday, I got it installed and processed the photo shoot with it. Lovely.

Today I woke up to see a big announcement from Apple about Aperture 2.1. What?! No problem, a free upgrade for me. It’s not loaded with a huge number of improvements. However, it is loaded with one massive feature addition. PLUGINS!

Now third party developers can design plugins to assist and enhance the editing within Aperture. Over time, I may not need to load Photoshop at all. I don’t have a problem with Photoshop, but how sweet will it be to only need one clean solution for my photo processing needs. I’ve heard great things about the solutions from Nik Software. Maybe they can be enticed to produce their plugins for Aperture too.

To show us how nice this new plugin system is, Apple included one nice plugin with the update.

Dodge & Burn Plug-in. A sample plug-in is pre-installed, taking advantage of the Edit API introduced in Aperture 2.0. The plug-in adds brush-based tools for Dodge (Lighten), Burn (Darken), Contrast, Saturation, Sharpen and Blur.”

I was actually poking around the manual last night, looking for dodging and burning. Sweet! I’m real excited about all these new tools and how I’ll get to use them.

****edit***

StudioLighting.net is reporting that several top notch plugin developers have announced…

Expected additions to Aperture plug-ins include:

• Nik Software’s Viveza plug-in, powered by U Point technology, which provides a powerful, precise and easy way for photographers to selectively control and adjust color and light in their digital images;

• PictureCode’s Noise Ninja plug-in that delivers advanced high ISO noise analysis and reduction;

• Digital Film Tools’ Power Stroke plug-in that features a simple, stroke-based interface to quickly mask and intuitively perform targeted adjustments;

• The Tiffen Company’s Dfx plug-in that provides an expansive suite of creative filters and effects;

• dvGarage’s dpMatte plug-in, which is a high performance chroma key tool for creating seamless composites, and the HDRtoner plug-in that enables the selection of multiple photos to create a single high dynamic range (HDR) image; and

• Image Trends’ plug-ins that include Fisheye-Hemi to quickly and effortlessly correct fisheye lens distortion, ShineOff which automatically removes shine from faces and PearlyWhites that automatically whitens and brightens teeth.

Sweet! I shot over to Nik Software’s page and discovered that they’re already announcing compatibility with Aperture 2.1 in May 2008. (less than 2 months away)