Creating a Master Archive in Aperture and Lightroom - Part 1

by Micah Walter

After spending a week comparing Aperture and Lightroom I realized something; I need to get my archive organized. Ever since I started using Aperture, about a year ago, I have been paying special attention to how I organize my work. However, everything leading up to that point is still largely unorganized. As I have mentioned in previous posts, before Aperture came along I was mostly using a combination of PhotoMechanic, and iView Media Pro to edit and archive my work. Well, let me clarify a few things here. I was basically only scratching the surface of digital asset management back then. I used PhotoMechanic to import, caption, and edit all my images after a shoot, sending selects to Photoshop CS2 for image adjustments, and I was using iView later on (sometimes much later) to keep track of where I stored all my files. Well, that is where my organization ended.

I rarely added my own keywords, unless a client requested it, and I lost most of my tagged images (a PhotoMechanic way of filtering your selects) during the import to iView. So, needless to say, finding images from long ago is always a challenge. I mostly have to rely on my memory and figure out "when" I shot something in order to find it.

With Aperture those days are over. Not only am I adding keywords and star ratings to images as I edit them, but I am also keeping similar images together in stacks, as well as all of my various versions of the same master. Life is good.

So, I have decided to investigate a little further and try and make some more comparisons regarding Aperture and Lightroom. To do this, I plan to create side by side image libraries of my entire archive. This archive consists of about 80,000 original and edited images. The images span the years 2002 through the present and were shot on a variety of digital cameras in both Jpeg and RAW formats. The end result will hopefully be a pretty neatly organized and useful library of my work in both Aperture and Lightroom. So in a nutshell, this is going to be a close examination of the Library portion of both applications.

To start things off I will once again disclose some key information. Number one, this isn't going to be a benchmark type of testing scenario. I may time things here and there, but I make no claims that anything I do here will be very scientific. The equipment is the same as before, my 15" MacBook Pro CoreDuo, with 2Gig Ram, a 7200RPM 100Gig drive, and the upgraded video card.

I am storing all the image files as referenced masters on a USB LaCie 500Gig external drive, but Aperture's and Lightroom's library files will reside on the laptop's hard drive. Hopefully I won't run out of disk space!

lightroom1.png

To be fair, I am going to start things off by simply importing all the master image files into both applications. Since I have been using Aperture for the last year or so, I will be ignoring all the versions, and metadtata that I have accumulated up until now. At the end of this little experiment, I will try and merge the old Aperture library with this new one, bringing all that Aperture data along in the process.

In my archive of master image files the only added metadata that exists is any IPTC information that I wrote using PhotoMechanic. I have never used XMP sidecar files, so they won't be an issue. I think there may be a small set of PSD files in the heap, as well as a shoot I once did in the DNG format.

The images on my LaCie hard drive are filed into folders according to their EXIF date. In fact this is the normal way I store my master images, with the folder structure built by Aperture. For your information, I use the Year-->Month-->Day folder structure to organize the pictures. These all sit under a top level folder I named PhotoWork.

To get things going, this past weekend I spent some time importing my archive with Lightroom. Because Lightroom takes a pretty simple approach to referencing images, telling it to import 80,000 images, which live on an external drive was a snap. I just pointed to the top level folder and about seven hours later, the job was done. In Lightroom's Folder panel I am now able to see the folder hierarchy and can easily find my way down to a days shoot.

During the seven hour import, I tried to do a few things in Lightroom. As my images began to appear in Lightroom's workspace, I tried to scroll through thumbnails and open images in Loupe View. In Lightroom importing images takes place in the background, so I am free to peruse the new files. On my MacBook Pro, I was able to browse the thumbnails with ease, although on occasion the program had to pause to catch up. Displaying files in Loupe View was another story. While importing, Lightroom had a hard time displaying images full size. I wasn't too shocked here. The application is doing quite a bit of work during import. Things seemed to get a little worse a few hours into the import, and I eventually gave up, allowing Lightroom to complete the task at hand.

The only other applications I had running during the import were Safari, iChat, Skype and Mail, so I will try and keep this the standard from now on.

At the end of the import (somewhere around 12:30am on a Saturday night!) Lightroom reported a list of images that could not be imported. I know for a fact that there is a good number of images in that archive which are corrupt due to a hard drive crash I experienced about a year ago, but I will be interested to see how Aperture handles these non-importable files.

In the end, Lightroom successfully imported 72,566 photos. The Lightroom folder (which includes Lightroom's databse file and separate preview file) on my laptop is currently weighing in at a modest 3.10Gig, and I will continue to monitor this number as time goes on.

The next step will be to use Lightroom's Collections to start organizing all of my shoots. I plan to use the Collections to sort of mimic Aperture's system of Folders, Projects and Albums.

Next week I will talk about my experiences in importing the same archive into a fresh Aperture library. Hopefully by the time of my next posting I will have both applications essentially running neck and neck.

11 Comments

etherfarm
2007-03-13 09:24:03
Micah, this is exactly what I'm in the middle of doing myself--working through my existing iView library in both Lightroom and Aperture as my trial periods on both programs dwindle to zero :(. I have only about half the images you do, and I'm working through them one project at a time, often with both programs running simultaneously (like you, I'm not that interested in benchmarking, and it helps that I have a lot of RAM :)).


I've realized a few things I value about both programs during this process:
-collections are _not_ smart folders, they must be managed manually, which is a bit of a pain. The closest "smart" grouping would be a to use a keyword, which Lightroom can then use as a filter very quickly, but it's a single-level filter.
-I much prefer the staging area of Aperture's import than Lightroom's. Though Lightroom has a preview "extension" on its import dialog which allows one to select (or more importantly, deselect) images, I find being able to stack, list (which doesn't appear anywhere in Lightroom), and establish other settings (such as timezone--I travel a lot) much more intuitive before import rather than after. True, this isn't practical for a mass import such as yours, but unless you're very, very prolific or away from your computer for a long time, a mass import is a pretty rare event.
-I also find Aperture's visual continuity between your library structure and the import process (the way the import window extends from the import panel and points back towards a folder of your choice) a really smart UI move on Apple's part. I'm moving images into my library--why would I not want to see them in the context of that library? Of course, if you want your library structure and the folder structure to be identical, this probably isn't a big deal.
-Lightroom's ability to convert to DNG on the import process is brilliant.
-Lightroom's option to apply develop presets to images on import is also brilliant (though I have other problems with presets)
-I have to jump through hoops to get Aperture to recognize my camera (Leica M8), so with out-of-the-box support for my main rig, Lightroom scores big points. Luckily most of the images in my existing library were taken with cameras Apple supports.
-Lightroom is a bit smarter about managing metadata presets on import. I like this because even though all of this can be handled post-import in both programs, I don't like how Aperture requires you to have the presets ready beforehand (I also find it bizarre that Aperture doesn't allow you to manage metadata presets where all of Aperture's other presets are managed, File->Presets, only the metadata inspector.)


I really look forward to your next few installments.

Rene
2007-03-13 18:04:08
I look forward to reading more about this experiment. That's a really neat project to see how both programs handle such a large number of images.
Michael Ball
2007-03-13 19:30:02
Be careful with Aperture and corrupt files. It will import them, well in my experience w/ one corrupt file it imported it. Well in combination w/ my memory card not working properly, my entire project, which contained an additional 433 images from the first soccer game that morning, got messed up. I tried to delete the entire project, so I could get to the files in the trash. Nope the files weren't there and I lost only the first 430. I could get the 600+ from the second game, and 3 from the first, but the 430 are gone. :-( I just received my new portable drive, a G-Tech C-Drive mini 40gb from Apple. The sad thing is that at 1 point I could have had all my files, but I guess I was lazy, I dunno.
clear_blue_skies
2007-03-14 01:33:22
I wish Aperture had a view that mirrored the file structure. I would just like to add multiple directories as roots, and have the program watch those directories (and subdirectories) for the addition of new images. (Multiple root directories are necessary because my images fill up multiple hard drives.) No photo/DAM app seems to get this right.


For Lightroom, I don't like that Lightroom modifies original files. Blecccch. Another thing that bothers me about Lightroom (besides general immaturity) is that its way too easy to adjust the star ratings on images. iView has this problem too. If you have several hundred thousand images, clicking on an image wrongly and clearing a star rating by accident is almost the equivalent of losing a photo forever! Meta-data should not be changed by an accidental sneeze.


Both Aperture and Lightroom are very sluggish as image browsers, even for jpegs. With ACDSee I can zip through several images per *second*. I've never found a photo browser than can go through images as fast as ACDSee.

Daveed V.
2007-03-14 12:03:25
clear_blue_skies: For me Lightroom 1.0 can keep up with me "arrowing through" both JPEGs and Canon CR2 RAW files (probably about 4-5 images per second). This is on a dual-core 2.3 GHz PowerPC PowerMac.


Also, when does Lightroom modify originals?

clear_blue_skies
2007-03-14 12:46:29
I must admit that my Lightroom testing has been on a Pentium 4 machine. I've never tried it on my Core2 Duo system, but I can't imagine it being that much faster.


The Aperture testing that I've done has been on current Mac Pro towers, and it feels very sluggish.


> Also, when does Lightroom modify originals?


Lightroom modifies the image metadata (EXIF/XMP data) in jpeg and DNG files. You can set the file as read-only through the operating system, and Lightroom will respect that though.

Gio
2007-03-15 06:09:00
So what if it does alter the metadata in documented file formats when you tell it to do so?
Micah
2007-03-15 06:17:01
John Nack on his blog has a pretty good explanation here...


http://blogs.adobe.com/jnack/2007/02/nondestructive.html

Mark
2007-03-15 09:25:35
Here you go with a project that may make or break my decisions just when I had finally decided that perhaps Aperture was the program for me! I'm quite looking forward to this experiment.


After your last series, I was sold on the idea that Aperture had better file organization. But the more I use Aperture, the more I miss LR and its develop module. I really don't want to use both, so if this review can show me how to make LR organize somewhat well, I may be sold. Then again...


Anyway, can't wait to read more!

clear_blue_skies
2007-03-15 23:18:36
> So what if it does alter the metadata in documented file formats when you tell it to do so?


Lightroom alters the metadeta whether you want it to or not. You can at least set the files read-only via the operating system, and it will respect that though.


In addition to the issue of possible data corruption, writing the data to the orginal file has other undesirable properties such as:
1. It causes externally stored checksum/CRC checks to become useless.
2. Is highly undesirable in backup situations with normal backup tools because if even 1 bit in an image is changed, the entire image must be backed up (causing lots of unneccessary disk/network activity to accomplish backups).

Rory Sinclair
2007-11-28 08:12:16
so.. uh... what happened to part two?