Syncing Your Aperture Library via Dropbox
The timing on this article goes along nicely with Apple's WWDC announcements and suggested direction of a cloud-based library.
EDITOR'S NOTE: The following post discusses how to do something that's frankly dangerous. You will read this disclaimer about a dozen more times throughout this post. If it works for you, it can be incredibly useful. If however you're not the exceptionally diligent time, you may want to give this a skip…
-Joseph @ApertureExpert
Have you ever considered using Dropbox to sync Aperture libraries, but been too afraid to put your precious library in harm’s way to see how it works? No need any longer—just read below to learn more.
Although there are no other tips on this subject to date, I will start out by referencing a very relevant conversation here that took place in the Forums and has some great insights. The consensus there was that using Dropbox is more trouble than it’s worth and could really screw up your library.
All of that may be true, but, as someone who has used Dropbox with my Aperture library for years, I can also say that it can be a valuable tool.
It is possible to mess up your library, so as a big disclaimer up front: this isn't a workflow that is recommended by either Apple or Dropbox. It works for me, and provides some solid benefits, but yours may vary, so be very careful.
Caveats: Look Before You Leap
Before you delve further into this article, I will start by stating the obvious: before you do this (or anything) with your Aperture library, be sure to back it up! You’re likely making regular backups already. If not, there are plenty of resources on how to backup your library (here is a good place to start).
The fact of the matter is that you really can screw up your library if you aren't careful. On top of that, you will not find any Apple-designed safeguards to prevent it from happening, so keep that in mind before moving any further. If you backup regularly, then nothing should be irreparable, but either way: you’ve been warned.
Why bother?
After listing all those caveats, you may be asking yourself why I bother. Do the benefits really outweigh the potential heartburn?
That all depends on how you use Aperture as a tool. Personally, there are several factors that make use of Dropbox worthwhile to me. Those include:
- Referenced masters
- Multiple computers
- One “main” computer for majority of work
- “Secondary” computers occasionally
- My computers are on the same network
A library of referenced masters is very relevant here because a managed library will be a huge file, likely into the hundreds of gigabytes (or even larger) depending how much you shoot. Referenced libraries are generally small(er), as they only contain the the changes you have made to your images, rather than the images themselves. Larger libraries not only take longer to sync, but will cost more, given that Dropbox’s cost tiers are linked to storage space.
If you use multiple computers, especially if you do the vast majority of your Aperture work on one machine, then this is likely useful. I am almost always using my laptop for my day-to-day Aperture work, but also like the option of using another machine for doing book layouts or other time-intensive projects, and I want all that work in the same place without taking many extra steps. Using the computers over the same network allows the changes to sync quickly via LAN rather than waiting hours over broadband.
Using this setup, I organize my library and sync those changes across computers. Even if images are offline, I can sort them into whatever organization scheme makes sense for me. I can also use large previews of offline images to layout books and do other work that doesn't require the originals to be online. Finally, I store my referenced files on an external hard drive, so it's easy to work with the referenced images from any machine (I just plug the external into whatever computer I'm using and the masters are immediately online).
That said, I’m not constantly going between machines. If you are, then this probably isn’t the option for you, given that you are at greater risk for creating conflicts by accidentally simultaneously opening the same library on multiple Macs.
Making it Happen
The trick to using your Aperture library with Dropbox is that you never want both libraries open and syncing at the same time. If you do, the one will try to overwrite the other. Given that a library is actually a package file with many many smaller files, this will cause a headache… but isn’t the end of the world.
In fact, like most stupid things, I’ve done it once, and the results aren’t terrible. It took me about an hour to figure out the issue, grab a recent backup, then get back up and running. It was not ideal, but using Dropbox has saved me many hours of exporting and importing various projects, books, and changes between computers, so I consider an hour of recovery work a bargain.
Given that I do not frequently open Aperture on my secondary computers, my risk for these mistakes is very low. This is particularly true given that my computers live on the same wireless network, using the LAN sync feature on Dropbox, which allows changes to be synced locally, many times faster than they would over broadband.
Some users have voiced a concern they may hit throughput limits on Dropbox by having it constantly syncing in the background whenever Aperture is open. I have never run into this issue. I always keep Dropbox running, and will edit my library, sometimes for hours on end. Never had an issue.
Wrapping up
One important point: for most users, you will need a paid Dropbox account to make this happen. So far, it’s been worth the cost to me, especially given it also serves as a backup, but it is an expense I’m often weighing. I am also not trying to push one service over another: I'm sure many similar services would work as well, but cannot confirm it personally. That said, I’ve used Dropbox long enough to trust it: it’s never botched my library.
What other methods are users employing to achieve the same results? Are you syncing your library some other way? Are there any readers who have experimented with other sync services (like SugarSync, or OmniPresence)? Are there any questions I didn’t answer? Please let us know in the comments!
Comments
on June 10, 2014 - 2:30pm
Aperture is essentially a single-user relational database; all of the normal restrictions apply. I use Filemaker Pro for a research project, and I use it from several different locations, the file being hosted at copy.com with no problems whatsoever mostly because nobody else ever opens the file but me. My FileMaker Pro file has referenced photos as well. No problems at all. And everything backed up on the mothership, of course.
Wouldn’t the use of Aperture be similar?
www.msadesign.com
on June 11, 2014 - 1:53am
Yes, it should be similar. The problem is simply opening a database on another Mac either simultaneously, or before the database has synced from one mac to the other (via dropbox or not). And a damaged Aperture database could result in an un-openable database, which isn't good for anyone, much less an inexperienced user.
@PhotoJoseph
— Have you signed up for the mailing list?
on June 11, 2014 - 6:41pm
Joseph writes: ”or before the database has synced from one mac to the other”
For this single reason, and on my reflection, using Dropbox is really questionable for anyone not seriously familiar with how DB and Aperture work: how they handle conflicts, how they handle syncs, when they sync, etc.
And I think my comment about Aperture being essentially a relational database, while correct in the main, is incorrect in the particulars: FileMaker Pro knows how to handle conflicts, while Aperture doesn’t.
Overall, Bad Idea, ApertureExpert!
www.msadesign.com
on June 11, 2014 - 6:45pm
Hence the repeated warnings, msadesign :-) This is not for the weak of heart or the casual user. As the author and I wrote repeatedly, this has dangers, and only works if you are exceptionally diligent and have regular backups. It works for Matthew because he handles it like a porcupine.
@PhotoJoseph
— Have you signed up for the mailing list?
on June 11, 2014 - 7:19pm
Echoing Joseph's reply about the warnings/notices in the post.
That said, I think it's a simple cost vs. benefit consideration. For me, the benefits are worth the extra pain and risk, but for a lot of folks it won't be & I completely respect that!
Either way, I'm sure I'm not the first person to consider it and, believe it or not, there's virtually no overview anywhere (at least easily searchable through Google) about putting Aperture on Dropbox, which is why I thought it'd be valuable to have a resource like this post.
I also think it highlights how many hoops someone is willing to jump through to get albums and non-destructive edits synced between multiple devices – which at its core are some extremely powerful features. Most importantly, Apple seems to have realized this as well & seems to be moving in the direction of supporting that natively, but I'll delve into that more in a future post. :)
on June 11, 2014 - 4:30pm
Thanks for your comment!
I'd add to Joseph's comment by saying that based on the combination of Aperture Libraries being packages of many files and the way that Dropbox handles conflicts, it can lead to a headache because it duplicates a file and creates an alternate version in case of an active conflict (e.g. two files being updated at once). That can blow up your library (both in size and functionality).
That said, and not to underestimate the number of things that can go wrong, if you're religious about backups, then a recent backup can always save the day even in that case (not that you should put all your eggs in that basket).
on July 21, 2014 - 8:15am
I was looking for way to sync Aperture library, because I often do some edits on my Macbook and do import/export/auto-enhance (in other words, all things that takes a lot of time) on my mac mini server.
I tried your solution, but faced with problem: its too slow and takes a lot of performance. I think that Dropbox is bad in syncing thousands of files and Aperture library (even referenced) contains thousands of thumbs, previews, versions… So when I start syncing, fan of my mac speeds up to 6000, Dropbox becomes the most performance consuming app and it takes too much time to sync it. My referenced libraries are about 5Gb (I’ve done everything to make them smaller: masters are referenced, previews deleted, thumbs shrunk by JpegMini) and Dropbox syncs them slower than if I’m just copy-paste library via local network. The most disappointing thing that a lot of files changes even if I just open-close library and Dropbox starts sync again… I tried also Bittorrent sync, same problems.
Maybe something wrong with my libraries? Does your referenced libraries also contains about 200k files (I mean internal database files, like thumbs, system files, etc)?
Also, Dropbox recognizes Aperture library as a folder, which makes Versioning feature useless (because you can recover previous versions only for files, not folders).
The only thing that sync files without taking lot of performance and time is rsync utility, but with it I always have to remember where (on macbook or mac mini) I have latest library version, because rsync support only one way syncing. Also I have to start it manually.
on July 22, 2014 - 4:58pm
Thanks for the feedback, and interesting to hear…
Have you enabled “LAN Wi-Fi Sync” on both your Dropbox machines (and ensured they're on the same network)? If that, that should resolve your issue.
Another option I've followed before is simply copying and pasting your Aperture library onto your second computer before you start syncing (doing either through an external or target disk mode, or maybe even Airdrop although haven't tried that one). That way both machines are basically starting in the same place and it's just a matter of getting your library into the cloud, although it sounds like your setup may have already been this way when you started.
Otherwise, it sounds like you're doing everything you can. With that said, many of the files within your Aperture library aren't actually updated when you change your library, so each time you do an update you don't need to push all 200k files or whatever into the cloud – there are several, but I think the number for me usually hovers around 100-200 (although don't take that for gospel, I haven't checked the number in awhile), so in other words most of that is a one time push, then after it's in Dropbox, you'll only need to sync a few files as you go.
As for the versioning feature, I can't speak to this unfortunately, as I don't use it myself. That said, even though the folder isn't versioned, it's presumably versioning all the files within your folder, so if you're given the option to restore from a backup of a specific date, you could delete your Aperture library (eek! after saving a backup of course!) then just restore from whatever date. Again, I don't use the feature, so I'm not sure if this is a workflow option.
on July 23, 2014 - 9:10am
Yep, I have LAN sync enabled. And I already done first sync/uploading to cloud. So start point is: there are identical libraries on both Macs, everything is already synced and uploaded, both Macs are in same network and LAN sync is enabled.
Then I open library on one of the Macs and do some changes (for example edit 2-3 photos, export some of them, etc). Idea is that Dropbox should sync only changes for those 2-3 photos (updated previews, updated thumbs, some system data), but in my case Dropbox showing 25-30k of updating files. And its not first push, it happens each time I open library.
I’m starting to think that it’s something wrong with libraries…
Here is example of my Commercial library, where I keep photos from weddings:
- it’s referenced
- all previews are deleted
- all thumbs are shrunk by JpegMini
- size of library is ~4Gb
- number of photos in library is ~16k
Now I examine library (by right click -> Show package contents) and here is what I see:
- folder “Thumbnails” has size about ~2.5Gb and contains ~37k images. Also it contains folder “Segments” ~500Mb, containing files like “Thumb64Segment_0.data”
- folder “Database” has size ~1.5Gb. In this folder, most “heavy” folders are: “apdb” (680Mb), “history” (480Mb, contains 18 files like “0000000001.plist”), “versions” (296Mb, 37k of files like “Version-0.apversion. Also here is a lot of folders and subfolders, containing this files).
When Dropbox syncs, seems like most of this 25-30k files it syncs are .apversion (maybe I’m wrong, but I think that syncing them takes most of the time).
So… 16k photos and 37k of .apversion files… More than 2x… Maybe I have some trash files, which should not be here? I have this library for 3 years, so it could be a bug..
Could you please tell me, if you have library with similar number of photos (~15k), how many .apversion files and thumb files does it have?
They could be easily counted by custom search:
- Show package contents of aperture library
- navigate to “Database”/”Thumbnails” folder
- search in folder (not whole Mac) with options: name is .apversion (or kind is “image” in case of thumbs), “show system files” is turned on.
on July 23, 2014 - 9:56am
Do some tests. Even if I just open and close Aperture library (without editing and even opening any photo) “Updating library” message pops-up (see attached picture) and about 1k .apversion files changes. Why? What’s Aperture updating? Its not sharing previews (I turned them off), not previews (they are also turned off). Maybe here is a way fix it?
on July 23, 2014 - 7:10pm
Yes, it certainly shouldn't be taking that long, but I will say there are files that are updated every time you open/close Aperture. For example, you may notice that it remembers what folder or project you have open or closed (or if you have multiple folders showing or hiding contents, etc.). OS X itself does something similar with the .DS_Store files that are in each folder (although usually hidden to most users). It's basically storing your display properties and current file & folder view.
You can see this in action if you open a library in Aperture, close folders or change folders, close Aperture, then reopen the same library in iPhoto. It'll pick up exactly where you left off, so that's stored in some preference files somewhere and will need to sync every time.
Mine doesn't take long though… I'll noodle it and see if I can figure out what's going on on your end that's causing a delay there.
on July 30, 2014 - 7:54am
Partly solved problem by rebuilding library. It still needs to sync about 1-2k of files even if I just open/close library, but all files are small and it doesn’t take a lot of time.
Also I think that BitTorrent Sync app is a better solution, because it syncs directly from one Mac to another, without cloud. I already have backup of my library on Time Capsule and in Crash Plan cloud, so I don’t really need another one copy in Dropbox cloud. Also as I said before, Dropbox doesn’t allow to restore previous version of folder, only of file, and as soon as Aperture library contains thousands of files, restore them to previous version manually is… you know)) So, as a backup-in-cloud solution, Dropbox is useless (I mean for Aperture library). Thats why I think BitTorrentSync is better.
Anyway, thanks a lot for your article and help, now I have my libraries well organized and synced and don’t worry when I am somewhere with my Macbook Air only because I always can connect via VNC to my home Mac and start import/export/auto enhance tasks remotely, knowing that all changes/choices/new keywords will be synced (and I use same scheme for my FCPX libraries too).
The only thing disappointing me is that all these improvements made with software that in fact dead) Hope Apple will improve their new Photos app (as they killed Final Cut and now recovering it, adding features update by update). I make most of color correction in Nik Software plugins, but I can’t live without Aperture import/organize/export features. If Photos will (after several updates) offer same power in bulk editing/renaming/exporting and will have good integration with third-party plugins, than I think it could replace Aperture. If not, well, seems like its time time to look on Lightroom…