I’m looking for advice, using Lightroom, on where to host my main photo folder.
I think I’m finally understanding how Lightroom works and some of the ease of use that it will add to my workflow and power to basic editing, so much so that my use of Photos may be limited to less than what it was originally going to be. I’ll still use it but in a different way now.
I have photos all over the place… on cards, on various older computer drives, in folders, and in various iPhoto Libraries. I think I understand all the nuances of transferring all of the photos to the point of getting them into my Lightroom workflow, including the merge to Aperture for export process.
I’m now understanding that Lightroom does not actually store the files, but instead there is a catalog of adjustments and previews that are used. This keeps the original RAW files I’m shooting untouched. I’m also getting the hang of the catalog system lr6 uses although I’m a relative newb at that concept. Several articles talk about archiving older files to external drives etc which makes sense to me too.
So here is what my new startup DAM strategy might look like.
- take photos from all sources listed above
- place all photos - culled or not - in one folder (location “x” not yet determined - see question below)
- in lr6 (tending more towards the standalone version as opposed to the Cloud version for lack of hispeed reasons) do my sorting, star ratings and keywords thing, creating sub folders and categories all the while developing my naming strategy for import (which will probably look similar to Joseph’s original Aperture strategy of using dates etc)
- do some basic editing in lr6… white balance, sharpening etc
- external optional editing in ON1’s Perfect Photo Suite (which has a handshake with lr6)
- intergrated backup of all files to location “y” and location “z” (one on site and one off site)
- done
- well, not quite, as I still think that I will make use of Photos… my intent is to export to Photos all the files I want to have available, whether they be Personal for friends and family and social contacts, or business for… well business contacts! these I can use the power and ease of Photos for producing quick slideshows, books etc and because they will be a selected number photos only will not exceed the 20gb iCloud limit I have. Nor will it blow up my limited bandwidth. The beauty of this part to the DAM strategy is that these select group of photos will be available from all of my machines, mobile and workstation alike.
- now I think it’s done, but I’d be happy to hear from anyone on any glitches or issues you see arising from this workflow strategy
So here is the real problem I am trying to solve now. What I haven’t figured out yet, and here is where I’m hoping for some advice from some lightroom users, is where I will be hosting the main photo folder.
My main hardware consists of:
- imac 27”, 2 TB HD, 16Gb Ram (approx. 6 years old) presently running mavericks but soon to run Yosemite
- MacBook Pro 13 Retina, 500gb SSD, 8Gb Ram, brand new… I now do almost all of my work on this little beauty
- 2Tb external Time Machine backup HD
- ipad2
- iphone 4
- Airport Extreme base station (2nd one available, neither with internal storage)
- HiSpeed was brutal up until a loaner from Bell and which I will probably be moving towards soon… and will offer 60Gb bw and speeds of between 5 and 8 mbps… which is better than the 3 gb and .2 mbps I’ve been getting with my older Rogers hub
I now need to come up with a storage strategy for both original and backups.
The backup drive I can put anywhere on my network… probably just a HD connected to either my iMac or an Airport Extreme.
But… and so here is my conundrum, and (finally) my question. Where can I host my main photo folder on my network that will make it accessible from all my machines. I know its not going to be on my MBPro as it will quickly fill my internal HD. I know I won’t have a HD hooked up to that same MBP as that would negate the portability of the laptop.
I’m presently thinking of either an external HD connected to either 1) my iMac or 2) to the Airport Extreme. I’m getting conflicting reports of which one will work best. Some lead to arrays as the best bet but in my case that would be overkill… and too costly.
What would anyone suggest, given the hardware i have as a solution for the location of the HD that will host my original photo folder given that catalog info file and previews will probably be located on my MBPro?
… sorry about the length of this but I’m sure there are others who are also in the same boat, with similar equipment and are wondering the same thing.
I just have a 15inch rMBP so it’s obviously not similar to your situation. But I have a 6TB (two mirrored 3TB) LaCie external HD on which there is one main folder for all of my Lr photos. This includes all of my former Aperture photos (which was a combined old iPhotos library and Aperture library). Since the beginning of 2015, all of my new photos are purely Lr. So they are all in one folder on my external HD. When I travel with my rMBP and I’m taking pictures, say on vacation, I’ll create a separate Lr Library and do all of my post processing in it with a smaller 1TB external HD as backup and then when I get home merge the travel library with my main library and move the photos into my master folder. Julieanne Kost has a video on this. The interesting thing about the video is that it shows how you can integrate several different computers with just one master file at about the 16th minute mark of the video. If you attach the external HD to your iMac, I do not believe, that Time Machine will back up files on the EHD. You’d have to have an off site cloud based back up system (I use Backblaze) to make sure you can access a source should your iMac crash, get stolen or damaged. I use another external HD to physically copy my master Lr folder to and keep that off site as well. Joseph ingrained that paranoia of multiple back ups into my brain a long time ago. :-) A network system might be the easiest with regards to using several pieces of hardware together, but I know very little about networks, so maybe someone who does will chime in. Hope this helps.
Florian
Florian Cortese
www.fotosbyflorian.com
Thanks for the quick reply Florian…. and yes it was very useful. I suspect your LaCie drives are 7200 rpm? I have been told anything slower and they are too slow as external working drives. 6TB would also be plenty of storage for my purposes… I would think, anyway.
So, is this HD attached to your MBP all the time or do you work off of the lrcat and lrdata files that would be hosted on your MBP? Or, is your setup for these different?
Thanks for the link to Julienne Kost’s vid… it was exceptional and I somehow think exactly what I will be doing on our next trip. I’m also thinking about signing up for her “Revitalize your lightroom workflow” course. She is a very good teacher and between her and Joseph most of my needs should be met!
I’m thinking that if I host the lrcat and lrdata files on my MBP (as that is my main machine now) and my main photos folder on a HD attached to one of my Airport Extremes I should be ok… hoping to get some advice on that soon. Else I can connect the EHD to my 27” and share it across my network. dunno!? I could I suppose hook up an old mirrored Tower Mac with some larger HD in them and just tuck it out of sight… but it might not be quick enough or be able to run current enough system software… dunno again!?
If you don't have a solution you don't have a problem!
Pondball, My external HD is really two 3TB HDs mirrored so that if one crashes the other will be available. So I really only have 3TB of storage which for my usage is just fine for now. It does run at 7200 and is attached to my rMBP which sits on my desk 95+% of the time. Only when I go out of town for several days or more mainly on travel/vacation/business, do I take the rMBP with me and I already have a separate lrcat for travel set up on my rMBP HD and a 1TB external HD for travel which I already had mentioned. My rMBP is were I house my main lrcat file, as well. So the photos are on the EHD and Lr is on my rMBP. I’m anxiously waiting for Joseph’s Lr for Aperture User training video. I’ve set something up which is close to what I had in my Aperture Library with a slightly different variation for Collections. The process probably could be a little smoother so I can’t wait to see how Joseph approaches Lr, Glad I was able to provide some help.
Florian
Florian Cortese
www.fotosbyflorian.com
you bet… any indication when Joseph’s LR6 series will be available… I’m looking forward to it as well. I took his Photos course and it cleared up a lot about Photos, so much so that I almost decided to go with it… but eventually succumbed to the lure of LR. I hope Joseph devotes some of his series to the HD setup options as, to me, this has been a real stumbling block to actually starting up the process. I want to get it right the first time and don’t want to make a newbie error that will cause me to start all over. I don’t have either the time nor inclination for that.
Thus, I really appreciate your responses as they do indeed help out!
cheers, dan
If you don't have a solution you don't have a problem!
Dan, In the Tips section, Joseph mentions that he will begin working on the Lr for Ap users in August.
Florian
Florian Cortese
www.fotosbyflorian.com
Just some thoughts. I’d do organization in Finder, not LR. While I’ve found LR stable and capable, it’s not as convenient as positioning and moving Finder windows on your desktop. If you’re as dispersed as you indicate, you’ll be spending a lot of time in LR just locating images.
If your images are already rated and keyworded, you want to either create side cars or, use Aperture to suck them all up in a referenced library. Then import that Aperture library into LR. I moved from an Aperture referenced library to LR. It went well. Start with some test libraries and import them to see what LR is doing. Look in the Finder, understand what you’re looking at. I did not care to have LR create a separate, apart from my organization, folder for all edited Aperture images. I exported tif’s of all Aperture rated images higher than 3 or 4, forget which. Exported to the same folders the masters were in. Then set LR to not create the Aperture Previews (or whatever they call them) upon import. If you go with the Adobe approach you may very well end up with a very disorganized mess of edited images you will never put back in their appropriate folders. I edit a lot but I hand out stars very frugally. For me, my approach worked much better. I can give you my hardware layout. It’s not the right answer as there is no right answer. But it works for me and I have no complaints. The Mac is dedicated to photography and streaming media throughout the house. I use an iPad for just about everything these days. rMBP + 25” Dell + 2 OWC dual Thunderbolt enclosures. 1 TB internal, all LR catalogued masters located here. Dualie #1, 1 TB ssd for all other data and non-catalogued images, 2 TB partitioned spinner to backup each of the former. Dualie #2 is nothing but media including some duplicates of images in the former drives, 2 4 TB drives, nothing backed up. 2 usb3 2.5” bus driven drives, clones of the backup in Dualie #1, stored in a safe deposit box. Convenient. I’m not ready to embrace cloud storage. More from a convenience pov than security. If I have a calamity that takes down primary and backup drives, 1), recovery probably won’t be my number 1 priority and 2), it’s a simple process to pick up the drives and both use them and clone them back when the primaries are replaced.
In the above scenario, odds are there won’t be any internet to do me any good with cloud storage. I live in a hurricane zone. House is on the water.
Sorry about the lack of paragraphs but that’s what happened when I edited a few words.
Hi Ray… thanks for the detailed response… and no worries about the paragraphing… works for me as it was the content that mattered anyway.
Am I reading this correctly that you are using your rMBP more as a standalone workstation connected permanently to a 25” monitor and several ExtHD’s than as a portable device?
Are you using the standalone LR? It is the only Adobe product I would be using as I now have both On1 Perfect Photo Suite and/or Affinity Photo that I will be using for any external editing I can’t do within LR itself.
I still have a (2008 or 2009) 27” iMac with 16GB ram that I will be moving to a cooler location and I was now thinking of using that to host my masters. Would that mean I would also have to host my lrcat files on the older 27” iMac or can I still have the cat files on my much faster rMBP?
If I thought I could host the masters on a network HD (raid or otherwise) connected to one of my Airport Extreme units I would do that but not sure what that would do to performance.
I have just switched from my old ISP that was giving me a whopping .2 to .5 mbps “hi”speed to another that now provides 4 x’s the bandwidth and average between 5 and 8 mbps of hispeed. I was thinking of connecting (via cat5) one of my AE’s to that so that I can also have one of my printers shared across my home network (the other is in our home business area and is wireless area) but if I could also connect my master photo files to that AE I’m wondering if that would be better than connecting them to my iMac that will be no “hardwired” to the network.???
my rMBP has a 500 GB SSD so will be used for travel, editing and processing only, not storing of archives or masters. I have an older iPad but will be using that for showing relatives photos in Photos when they start showing me their photos… ;)
btw… what is a Doulie?
I like your idea about not using iCloud for any major photo storage… I’ll only be using for storing some coaching photos and trip slideshows tat I don’t want taking up room on my iPad or rMBP… not much else… will look into size of our safety deposit box and see what I can find… much cheaper than bumped up cloud storage… and more secure IMO!
… forgot to add… I really don’t have enough photos already in Aperture as I had just purchased it about a month before it was canned by Apple :(… about the only thing I will be using it for is to merge several salvaged iPhoto libraries I’m trying to pry out of older iMacs so I can import them into LR from Aperture. Basically I’m starting my DAM from scratch. Have been playing around with a test version of LR (5 days left) and seem to be getting on ok although between learning LR and trying out On1’s PPS and Affinity Photo there is a lot on my plate. I’m bringing any current photos in via the card reader and directly to the finder at this point… but so far the experiment is all being done on my rMBP until I sort out the master storage solution and where it will be hosted
If you don't have a solution you don't have a problem!
We have 2 homes and having desktops in both places was a pain, maintenance and sync. So, yes, I now have only the rMBP and displays at both ends. Chucked a half dozen enclosures and run OWC enclosures that hold 2 3.5” or 2.5” drives each (dualies). Been about a year now and delighted with the setup. Both from a travel pov and everything in one place. When we’re at home #2, 2 2TB 2.5” bare drives go with me and I have an empty usb3 dualie they are mounted in. We tend to travel a lot when we’re there so this year I haven’t even bothered using the external display. It’s not pleasant but I’m using the 13” display on the rMBP.
I am using standalone LR 6.1.
With LR’s smart previews, you can pretty much decide where you want your primary image storage to be. Say it’s on your iMac. Then for those files you want to edit on the MBP you create smart previews. You would copy the LR catalog (and perhaps a Smart Previews folder – don’t know as I don’t use them) and work on those on the MBP. I believe it’s a good feature but you would have to brush up on it as I don’t have any experience with it. There are all sorts of ways to deal with multiple machines. Just need to find those with experience and their thoughts. But yes, the catalog would be on both machines. Either the same catalog or parts of it.
I usually run pretty new AE’s. I have never found file access even remotely acceptable to me. Wake times are slow, access times are slow. Have not tried it in years but I’d test before you get very far into this approach. It may not be cutting edge but attached storage is cheap, fast and a lot more reliable. 7,200 rpm used to be important. With today’s large drives, platter density can get you more speed than rpm’s. My 5,400 rpm 2TB Spinpoints on usb3 can outrun my 7,200 rpm 2TB Hitachi on Thunderbolt. Unless you’re using old drives – which should be replaced anyway – I would not worry about rpm’s. Drives that are 75% or less full and fast I/O, usb3 or Thunderbolt, have changed the old metrics.
You’re running into the same issues I dealt with for years. I looked at how I and other people handled things and came down to one machine is simply a better solution for me. I’m not a pro and don’t need a 16 core Mac Pro with multiple externals, etc to get the job done. You might want to give some thought to just using your MBP. I use an Arc stand and it’s a snap unplugging one USB cable, one Thunderbolt cable and the power adapter. I have a 30,000 image library, all originals are on the internal, I limit 2880 previews to 20gb and am using 504 GB. Music, when we travel is a downsized iTunes library on a 128GB Jet drive (short sad card).
If you’re not already vested in a lot of Keywording and you have not used Aperture all that much, you’re not gaining much by importing an Aperture library into LR. If you already have tif’s or jpeg’s for your Aperture edits, you may want to: 1), get your organization done in Finder. Import into Lightroom. The intermediate step into Aperture is not saving you much time. If you are using Managed libraries in Aperture, iPhoto, Photo you would want to first switch them to referenced. Give timing some thought before you start. Once you import part or all into LR, you want to stop using that/those libraries.
If it’s of any assistance I use a modified approach to organization. 80% of my “events” are in date order. Then I have a series of folders for subject specific categories. (family and friends, interests, etc). I also don’t catalog all photos. For those that are very subject specific and access very infrequently, I keep them in folders on the external ssd. I access them with a variety of browsers (currently Graphic Converter). I like to keep as little as necessary in Lightroom.
thanks again Ray… that was once again very helpful and very clearly stated… especially to this DAM Newb!
And you’re right… its the redundancy that I am trying to avoid… as you seemed to have very well!
my rMBP is a very new 13” as well… and I walk around with it as if its an iPad sometimes! so not sure that I’d want to tote ext HD’s with me. At this point I’m just getting back into photography after about 25 years off, but took about 3000+ shots during a European river cruise/Paris trip last summer and am hoping to get organized (in my mind and digitally) before starting this whole DAM thing.
I’ll probably have more questions but your tips have helped immensely!
If you don't have a solution you don't have a problem!
You know your data requirements better than I do. When we travel, my 1TB rMBP and my wife’s 256 GB Air travel with us. DropBox is used extensively for anything shared, iCloud for the usual iOS syncing for mail, calendars and contacts.
I tote along a very small OWC Express enclosure with a 2.5” 2TB Samsung Spinpoint. It’s partitioned for the rMBP (750 GB) and Air (200 GB) backup clones (SuperDuper) and a 1.1 partition that contains my external SSD and some extra space. It’s a very small unit and more than worth the extra space. I’ve been using SpinPoints and the Express enclosures for years and they have been 100% reliable.
We travel an easy 2 to 3 months every year. One home is in the USA, the other in Europe. The above works for me.
I keep my photos on a network drive. The NAS is reliable and redundant, and I move the files off the cards onto the “images” share for safe keeping and permanent storage.
I use gigabit LAN, which is the norm these days—just make sure your switches are gigabit and your cables are good enough quality.
It is not too slow. Initial cataloging is slower than I would want, but working in it with develop settings, metadata, etc. is no problem.
I back up the file server, or the “images” share, onto a bare HDD. I don’t need a fancy name-brand enclosure; just drop it into a “dock” and pull it out when I’m done. The NAS is redundant against drive failure and uses ZFS snapshots to protect against human error: image files don’t change once stored, so a snapshot doesn’t consume any storage and I set it to hourly snapshots when I was worried about messing something up in a Premere project when my desktop seemed to be having RAM problems. Since it adds trivially to the storage I’ve left it that way ever since, and I’m “safe” if I wait an hour after dumping the cards, against accidentally deleting the file when curating them.
you ask, “Where can I host my main photo folder on my network that will make it accessible from all my machines.” If I understand that correctly, you want a file server. Either a dedicated file server machine on the network (like I have), or some other machine that’s always on and doesn’t move, that can also be the file server even if you’re using it for something else.
There are enclosures sold for that purpose now: you plug it into a network switch rather than into any single machine. They are overpriced and the RAID features are not as robust as you would hope. An old machine can be turned over to the task, just leaving it on, or downloading special (free) NAS operating systems for it.
If you have an old machine, left over after an upgrade, just park that in a closet. Attach the drives to that. Or, scrounge around for one from family and friends. A too-old slow laptop with too-little RAM for the lastest OS upgrade and apps can run a lean mean Linux image just great and find new life with a new purpose.
The tricky part, really, is the RAID stuff. You want to set up a drive-level redundant system, or a system to automatically replicate the files onto other drives. An old Mac has Time Machine! So problem solved there, with a main and a backup drive. If you have a place to plug in more drives… no “internal” connections for SATA drives there. A hand-me-down PC motherboard with (say) 6 on-board SATA connectors and the firmware booting from a USB flash drive would give you lots of capability.
To recap,
More thoughts on salvage:
Note on sourcing drives:
sorry if that’s long, but was following your example!
“sorry if that’s long, but was following your example!” LOL
Thanks John… and I appreciate your detailed description. Some of it I will run by my son when he’s home next as he is the one who is more up to date on networks than I am right now… been about 7 or 8 years since I did any network admin and although getting back on the bike is supposed to be an easy thing to do sometimes the wobbles are there at the start. Having said that I understood your very clear description nearly 100%. The rest I will look up!
I have several old machines hanging around besides the 27” iMac (2009). It has only USB 2 and FW800 and I don’t believe there is any way of upgrading that to USB 3 or Thunderbolt. I usually have that machine on at all times so could very well set it up as a file server. I don’t use it as a workstation nearly as much as I used to as I use my newer 13” MBPro… much faster and yes, more portable than the 27” ;)…
I also have an old PC (built by my son for his 3D and Gaming needs about 6 or 7 years ago. He has since outgrown it but it is there nevertheless. I also have an older Mirrored Drive PowerMac (not sure what vintage as it hasn’t been used for awhile) that might also be used as a file server. Do you think either of those would work to store my working files on the network.
It also sounds like I may have to upgrade my network routers. My Airport Extremes may not be GB as they are about 4 years old… will have to check. Only the one is connected directly via cat5e to the internet modem. The rest of my network in my home is wireless. Will this become an issue if the server is not in close proximity to the connected Airport Extreme (AE)? Would it be better if I used my extra AE as a bridge router/switch and connected whatever i would use as my file server to that via cat5e?
If you don't have a solution you don't have a problem!
I shopped around and found (reportedly) good quality gigabit routers that are a generic import brand but contain the same chip as the expensive ones. With one switch and one user at a time, it probably doesn’t make much of a difference. Easy enough to test a file copy job and see if you are getting the fastest you expect.
iMac: no upgradability that I know of, unless you can take it apart to the extent of replacing the internal HDD. Insert a “SATA port extender” and drive several disks from one SATA connection. I know of inexpensive 4 or 5 port expanders that are even cheaper now since they are (only) SATA II. I have one as a cheap hardware RAID-5 controller and am happy with it. So you *could* build a separate enclosure with power supply for a few drives and the port expander, and cable that home-built “DAS RAID” to the old imac or laptop. IF you had the power supply hanging around already (free salvage), that would be a cheap solution ($62 when I wrote about it).
For USB/FireWire ports, you need an external enclosure with power for the drive and the right kind of communication. Most USB 2 enclosures have a hard limit of a lousy 20 MB/s. I’ve seen one, using a more expensive Intel chipset IIRC, that was faster. They always list the raw USB/Firewire specs on the box but that has nothing to do with the product’s capabilities!! I don’t know how fast Firewire-800 connections are in real (inexpensive) products, but I think you’ll find that they were designed with old drive speeds and capacities in mind. Oh yea, USB/FW/whatever (except eSATA) enclosures often run into limits like 2TB or whatever, smaller than today’s newer drives.
Old gamer PC: Jackpot! look inside for the motherboard model number, and to see how you can easily mount more HDDs in the chassis. SATA II came out in 2004, so a 2008 mainboard should have at least 4 of those, possibly more. Hmm, you might run into a 2TB ceiling, but the free NAS firmware might work-around that automatically.
You might yank the power-hungry graphics card and use the built-in graphics (if present) or a cheap VGA card instead.
I don’t know anything about Apple’s branded wireless access point/switch/storage products. Only that they are expensive :)
The older PowerMac might be similar to the PC: it’s an expandable motherboard, not a laptop-like fixed solution. Look inside and see what connectors are present, both SATA and daughter-card slots. They’ve used the same PCI cards as PCs for a long time.
I upgraded my home-office LAN, and my wife’s machine, to Gigabit LAN in 2008 after I got a DSLR and started shooting RAW photos. The files were too big for her old surfing machine to handle. High-end PC motherboards were already featuring gigabit a few years before that, and in 2008 even the cheap motherboard had (1) gigabit port. Anyway, a new switch will set you back $18 at Target or Newegg.
Thanks again John… from the sounds of it I probably have a workable solution sitting here in one of the older machines… I’ll check with my sons to see if they know what they put into them at the time… they tend to build high end cuz they were avid gamers and one is a graphic artist. They upgraded and I got the “hand-me-down” which quite truthfully I haven’t used cuz of my familiarity with the mac environment. Up until 8 years ago I ran the server and network at our school but I’m sure both the software and hardware have changed since then. Looks like Ive got some brushing up to do!
If you don't have a solution you don't have a problem!
Take the cover off the the gaming machine and see what’s in it, for sure. You can find the number on the motherboard and look up the specs for details of HDD support and network speed. See how many SATA connectors are on the board, and whether there are video connectors directly on the backplane so you can pull the old power-hog graphics card.
Show your sons my my (previous) build: http://www.overclock.net/t/1155744/build-in-progress-mercury :) It’s striking to see parts like “Drain: ball valve, T fitting, G1/4 coupler.” on a PC.
You might consider doing a guest post (or several) on my Blog, as I have others in the set already. You might notice Bob Hyatt also had different solutions from mine. You also began with clear requirements in your initial post here, so it fits well with the theme, and it’s another take on the problem I have not covered in a first-hand account yet. I can host photos of the build, too.