Relocate Your Masters Online With Amazon S3

by Micah Walter

One of my favorite features in Aperture 1.5 is the ability to store my master image files anywhere I want. By using the Relocate Masters and Manage Referenced Files functions in Aperture's file menu I can easily take care of the task of spreading my RAW images files across as many external hard drives as I wish. Referenced files are really the core of how my workflow works, allowing me to quickly import images into Aperture as Managed files on my laptop’s hard drive and then later relocating the masters for backup and storage to larger disks. So far it has really worked out great.

reloacte.png

The one part of the equation that I have been trying to solve for some time now is what to do with all of the older files. These are the files that I want to keep, but that I also don’t really want to worry about constantly backing up and making sure they are secure. I also would like to have reasonable access to these files, so the idea of moving them to DVD or a separate hard drive and putting them on the shelf somewhere kind of frustrates me. Call me lazy, but I really don’t like having to search through piles of DVDs just to find a single image.

Enter Amazon’s Simple Storage Service (S3). For those of you who haven’t already heard, Amazon offers a scalable and fairly cost effective online storage solution. It was designed with the web developer in mind, offering businesses a quick and easy way to set up a redundant server quickly and easily utilizing Amazon’s extensive network of servers throughout the world. But now, with the aid of an application like JungleDisk, just about anyone can sign up and take advantage of the service.

finder2.png

For my own situation I immediately recognized S3 as being a great way for me to manage my image archive. So here is what I do.

First of all, I don’t recommend using S3 for al of your master images. You have to consider the amount of time it will take to upload your files and the fact that once they are uploaded you may wish to have faster access to some of your more recent work. However, for those files that you shot months or years ago, the ones that you still want to hang on to but don’t necessarily need on hand at any given moment, S3 is the way to go.

To get things set up you first need to create an Amazon Web Services account. It is pretty easy to do. Just go over to Amazon’s website and scroll down the page until you see the link in the left hand side-bar under Amazon Services.

Step two is to download JungleDisk. JungleDisk is a pretty cool application that interfaces with S3. It is easy to set up and comes with all the instructions you will need. In the end, what you get is the JungleDisk monitor application that allows you to connect to your server space, and monitor upload progress. The application also creates a server icon on your desktop that looks like any other server you might connect to. This gives you the ease and freedom to interact with your S3 server in the Finder window. In the JungleDisk preferences you can name the volume whatever you want (I chose Amazon), and you can also choose to encrypt all of your S3 data.

Now before we go on here, let me just add that you can use this for any type of data, not just images. I have already uploaded plenty of files from my Documents folder that I just don’t need on my computer anymore. In fact, to make my life really simple I made a single folder on my Amazon volume called Archives, and in that folder I made a sub-folder called Micah, and beneath that I mirrored the standard folder structure that OS-X creates for each user account. So I have a folder called Documents and one called Pictures and so on.

manage.png

Back in Aperture you have a couple of choices about how you go about uploading your master images. The easiest way to do it is to use the Relocate Masters function found in the File menu. Just select the Project you are interested in and then select all of the images within that project that you would like to relocate and click Relocate Masters.

Then choose your JungleDisk volume and select a location. I like to put things in a folder beneath my Pictures folder called PhotoWork. By selecting the Day/Month/Year sub-folder format option Aperture will create a really nice and easy to navigate sub-folder hierarchy for me, and if I continue to use the same option for pictures in the future, Aperture will continue to add to the hierarchy as needed.

Now it is important to note here how JungleDisk works. When Aperture relocates the master image files, it may seem to happen fairly quickly. In fact what is happening is the location you are actually moving the image masters to is a cache location on your local hard drive. Once the files have hit the cache location, JungleDisk will begin uploading the files to Amazon in the background.

You can monitor JungleDisks’s progress and what is really nice is that JungleDisk seems to handle pausing really nicely. I can close JungleDisk at anytime, or close my computer, and JungleDisk will pick up where it left off the next time you open the application. You can also set JungleDisks cache size to whatever you want.

The second way to move your master up to S3 is to move them manually. If your images are already Referenced on an external hard drive, you can simply copy them to your JungleDisk volume. Once they have finished uploading, you can use Aperture’s Manage Referenced Files feature to reconnect to your images that are now stored on S3. An advantage of this method is that you can keep two copies of your files. One on Amazon and one on your external drive. This may make it more convenient to access current projects, using Amazon as your online backup, and then later when you decide it’s time to put the project to bed, you can do the Manage Referenced Files trick to switch the reference over to your files stored up on S3.

invoice.png

Oh, and let’s talk about prices for a moment. Amazon has some interesting ways of chargning for their services. They charge for the total amount of space used each month, as well as the transfer bandwidth you take up. They also charge for GET and PUT operations, but it is still fairly unclear to me how that all works when using JungleDisk. The end result is still pretty cheap, relatively speaking, and I can rest easy knowing my data is redundantly backed up both physically and geographically.

No, Amazon’s S3 is obviously not a replacement for services like Digital Railroad and Photoshelter, but is sure makes a nice compliment.

3 Comments

Dudley Warner
2007-09-16 18:25:20
This sounds like a good way to store images safely. After doing the math for storage of 10-15 GB of images + the cost of the software, JungleDiskMonitor, wouldn't .Mac be a potentially better alternative?
dgp
2007-09-17 09:20:15
If you do the math for 2 years of just 100GB or archived photos, it comes to around $406. That could easily buy you a nice external network enclosure with 2 x 250GB of mirrored storage vs. 100GB on Amazon S3, and it would be much faster than S3 (usually 100mbit/sec).
The Amazon S3 storage differentiates itself by the fact it is online and always available from the internet. Perhaps this would be justified if you had a large sharing application for a whole group of photographers, but there are better and cheaper alternatives.


Heck, this site gives me 750GB of storage and and 5TB of data transfer per month for just $166 per 2 years:
http://www.webhostingbuzz.com



The Math assuming archival usage:
100GB * $0.15 * 24 months ($360)
100GB * $0.10 data uploaded ($10)
15GB downloaded per month * $0.10 * 24 months ($36)

Micah
2007-09-17 10:12:26
Well, I agree, there are certainly cheaper alternatives, but consider the fact that a) those hard drives you paid for will probably (WILL) fail at some point. But, more importantly although they may be set up as a mirrored raid, giving you redundancy for a potential hard drive crash, you aren't protected if your house burns down! With Amazon, and other online storage plans, you get the geographic redundancy...


As for the.Mac question. I am a .Mac subscriber and I use it quite a bit, but there are a few things I dont like about it, and I think something like Amazon may be more flexible.


-m