Photography Backup Workflow

Data loss is a very painful experience that unfortunately many of us go through at some point of our lives. I spend quite a bit of time advocating the need for a well-established workflow that incorporates solid backup strategies to prevent data loss. And during this process, I came across many different backup routines practiced by other photographers, some of which I found to be downright scary. You have probably heard of horror stories of professional photographers losing their life’s work, or wedding photographers losing images of weddings that they were not able to deliver to their clients yet. It sure happens, and it usually happens at the worst possible time too! It is one thing when you lose your personal data / photos and totally another when you are dealing with a client who paid you money. I cannot imagine how one could even handle a situation with lost photos, as it would most often be impossible to recreate those precious moments. Sadly, for many of us, it seems like data loss has to take place in order for us to seriously consider a solid backup strategy and workflow. But it does not have to!
 

Any time you deal with data, there is always a potential for partial or complete loss and it is not a matter of “if”, it is a matter of “when”. You might think your data sits safe on your computer and your external drive, but what if you lose both? Have you thought of a possibility of burglary, fire, flood or other natural disasters? What do you have in place today to protect your data from such catastrophes? If you have never thought of such questions, it might be a good time to revisit your workflow and make necessary adjustments, before it is too late.

1. Start with a Solid Workflow

Before you think about a backup strategy, it is a good idea to revisit your current workflow process. Where do you import your images? Where do you store your images and how? Is there a single place where your data resides, or is it scattered in different locations? These are all very important questions to ask and if you do not have the answers or you are confused about the process, this is where you need to start. Personally, I like taking a simplistic approach to file management. I always import my images to a single folder and store them there, which makes it easy to locate, backup and potentially restore images.

If you have not read my article on "What is your Photography Workflow?", now is the time to do it, as you need to understand what workflow stands for and what it is comprised of.

Here are my recommendations on setting up a workflow process:

1.   Do not reinvent the wheel – use best practices

2.   Do not oversimplify

3.   Do not overcomplicate

4.   Do not dismiss important workflow components

5.   Take time to establish a good, solid workflow

6.   Once solid workflow is established, stick to the same process for consistency and efficiency

7.   If new technology is available or there is a faster/better way to do something, make your workflow flexible to potentially      modify it as needed

8.   Look at all recommended options before committing to a certain workflow process (again refer to point #1)

2. File Management and Organization

A major part of your workflow starts with the transfer of images from your camera to your computer. Whether you copy images, or import them via software, you need to decide where and how you will be storing those images. And this is one of the most critical steps, because it will define your backup strategy going forward. While some photographers prefer importing images first and organizing them later, I strongly oppose such practice, because it can lead to inconsistencies and other problems in the long run. My preference is to organize images at the time of import and if I never have a chance to revisit my files, it is not a big deal, as images were properly named and stored in the first place. This leads us to the importance of file management and organization – a topic that we have written about a number of times in the past due to its importance.

Another article that you should give a read is on "How to Organize Photos in Lightroom". Whether you use Lightroom or any other software, I recommend that you check this article out, as I discuss the process of organizing and storing images in detail. In short, I store all images in a single folder called “Photos”, which is located in the root folder of my storage drive, and I group photos in folders by event name. A very simple and yet effective method of storing images that works for myself and thousands of other photographers. But how do you actually do it? Let’s take a look at two different methods – manual file management and file management via software.

2.1 Manual File Management

A manual file management is the most simplistic approach to file management and organization, since everything is done manually and it is something that you are already familiar with. You start by copying images to a designated folder, then you decide on the folder structure and where those files will reside in the file system and finally, you name the files to your liking. You do this on the file system level and the process is really no different than the process of transferring other files from external media.

Manual file management is simple, because it is easy to understand. Those who do not use Lightroom, Bridge or other software with file management capabilities often end up using this process and sometimes even quite effectively. Since you would know exactly where files are going, it also simplifies the process of backing up images.

However, there are many disadvantages to manual file management. First, it is quite inefficient, since traditional file systems are not designed and optimized for easy file management and indexing. Your computer might have the capability to preview your RAW images, but it surely has no way of reading the proprietary EXIF data / metadata in order to be able to filter through specific criteria, like the type of gear that was used, or the date when images were captured. Your operating system cannot go through this data and index it, to make it easily searchable for you. In addition, there is a big potential for file duplication, as your operating system has no clue about what photos you are copying – it just does what you want it to do. Even if you do efficiently manage files on the file system level, how would you be able to locate a specific photo without going through tens, hundreds or even thousands of images? That’s when file management via imaging software comes into play.

2.2 File Management via Software

Whether you use Lightroom or any other file management software, the advantages of software that is capable of going through images and extracting information are numerous. First, being a specialized tool for photography needs, such software can be very efficient in scanning through and gathering of valuable metadata, then indexing it for searching. Software can reduce and potentially even eliminate duplication, which can save space and improve the efficiency of the backup process. Once indexed, images can be easily searchable in many different ways. For example, you can use Lightroom’s library filter to see what lenses you use the most, which is really neat. You could filter through specific metadata such as lens used, camera used and even find images captured at specific apertures and shutter speeds.

But just like any other process, file management with software certainly does have its own disadvantages. It is more complex to understand. It requires the creation and potential maintenance of a database catalog, which adds more steps to the backup process. If done improperly, it can result in files residing in different partitions / drives, folders, etc, making it difficult to identify file locations. And lastly, it is often impossible to share data and access it from multiple computers, making such process potentially difficult to implement in workgroup environments.

2.3 Hybrid File Management

But what if you were to combine the best of the two worlds to create the most efficient way to organize your photos? I came up with a name for such a process and I called it “hybrid file management”. I personally use this method and after tens of thousands of images stored over a period of 8+ years, with very positive feedback from many other photographers, I would call this the ultimate way to organize images. In short, you get all the advantages of both manual file management and file management via software. If executed right, you end up with very few disadvantages to worry about. The reason why hybrid file management works the best, is because you organize images both on file system level and software level. If you ever lose your software catalog / database, you have enough information on file level to be able to easily rebuild that database from scratch. Since files are already properly named and sorted, you don’t have to worry about doing it again via software. You also do not lock yourself into using one particular software package – you could easily move to other software in the future with this approach.

So how does hybrid file management work? Below are some of the steps, which you can extract from the same article I pointed out above on organizing images in Lightroom:

1.  First, decide on a master folder to store your photos, such as “Photos” or “Pictures”. Ideally, put it in a root folder of your storage drive, as previously explained.

2.  Come up with a proper and consistent way to name files, so that you have enough information on the file system level to be able to quickly locate files. Ideally, each of your file names should contain the date (year, month and day) and the name of the event or sub-event, followed by a sequence number.

3.  Import images using software such as Lightroom and create folder structure during the import process.

4.  Do not heavily rely on keywords and other software-based indexing criteria to reduce software dependency. If you need to reorganize or rename images, do that through software to avoid potential mismatches between the database and the file system. Software such as Lightroom performs moving and renaming of images on the file system level and automatically remaps images in the database after the completion of the process. Use the provided tools to mass-rename images when necessary.

5.   After completing the editing process, force the software to write changes to each image (this will write changes to DNG files or force the creation of sidecar / XMP files), so that you do not lose your changes if your database gets corrupted or lost.

Once you come up with a solid workflow and you organize your images, it is time to think of a backup strategy.

3. The Simple Approach: Hobbyists

What is the “bare minimum”, low-cost approach for performing backups if you are a hobbyist? Beginner photographers cringe when looking at complex backup strategies of enthusiasts and pros, because the process can be over-the-head complex and very expensive. So if you are just starting out or if photography is your small hobby at the moment, I recommend a different, simpler and more cost effective approach for backups. No, I won’t suggest keeping your images on your memory card as a backup – that is not good enough even for a beginner. You want to make sure that you are protected against two main potential problems: hard drive failure and loss / theft / natural disaster. Always keep two backups at the minimum in two separate physical locations. The second part is very important. Do not just rely on keeping one single backup or multiple backups in a single location. This is something you should not skip on and it should become a good habit in the future. Hard drives fail and having another backup will certainly give you a peace of mind. But it won’t protect you from theft, flood, fire or other natural disasters. So while buying that external drive for backups, make it two in order to have two copies of the same data. Establish an offsite backup routine, depending on how often you shoot and how important your pictures are for you – have a weekly, monthly or quarterly process of updating your offsite backups to keep that data somewhat fresh.

The good news is, hard drives are very cheap nowadays. You can buy a small, portable, large capacity hard drive for less than $100. Just buy two of those and store the second drive somewhere safe, preferably away from where you live. If you do not want to pay for storage services, perhaps you could ask your parents or your relatives to store that drive for you. If it is too much of a hassle, just get a storage box in your local post office or bank and make it a routine to drop off the updated backups.

As for the backup process itself, you can do it manually at zero cost, which would require copying and overwriting files each time, or you could pay $20-50 to buy backup software that will backup your photos automatically. There are many different software and hardware solutions out there to accomplish automated file backups. Apple offers Time Capsue and there are literally hundreds, if not thousands of backup software suites. I personally used to love Acronis software for backups and used it for many years. You can backup and restore not only individual files, but also entire volumes / partitions, if needed.

So here is the quick summary:

1.   At the minimum, buy two external drives for backups

2.   Backup to one drive on a continuous basis either using automated software tools, or manual file copy

3.   Backup to the second drive as often as you will be taking it offsite. You can rotate the two drives, but you have to make sure  that you are not just doing incremental / differential backups – you have to have full backups for rotation

If you follow this process, you will have your images stored in three places: your computer, an external drive connected to your computer and another external drive that is sitting in an offsite location.

What about Cloud Storage?
With cloud storage getting incredibly cheap and potentially even free, you might wonder if it is worth keeping your backups on the cloud. Companies like Dropbox, Copy, Amazon, Google and many others have been offering free limited storage or practically unlimited storage if you subscribe to their services. For example, if you are an Amazon Prime member, you will get unlimited storage for your photos without extra cost. And you can upload not only JPEG images, but also RAW files as well (provided that RAW files are from Nikon, Canon or Sony cameras. Sounds great, doesn’t it? Why wouldn’t you skip on the second external drive, since the “Cloud” can be considered an offsite backup? Well, it all depends on how much you shoot and what your upload bandwidth is like. Unless you live in an area where providers can supply you with a boatload of upload bandwidth, trying to upload your entire collection of photos might take a very long time. I would start by estimating the total storage that your photos are currently consuming. If it is reasonable and your bandwidth can keep up without angering your ISP, then go right ahead. It will save you the hassle of going offsite for backups.

In case your computer hard drive dies, you have a backup of your files on an external drive. If both fail, you still either have the offsite backup, or a backup in the cloud depending on which route you take. With this method, you won’t risk losing everything you have, as the likelihood of all three failing or disappearing on you is extremely small.

4. Solid Backup Strategy: Serious Amateurs and Pros

If you are a serious amateur or a pro with a large collection of photos, creating a solid backup strategy is your responsibility, so you should take it very seriously. If you are a pro, it is a part of your job, it is what you are paid to do, even if you think you are not paid enough. I cannot imagine what this photographer went through, trying to explain to 20 different customers that her photos were stolen from her car:

Leaving 20 photo shoots in a single location like a car is not something one should do. This is a serious problem in the workflow process and if the photographer followed best practices, she would have had copies of every shoot on her home computer or a single external drive at the minimum. People can be nice and understanding, but some might start a lawsuit and bring your business and you down in no time! I can’t blame clients suing photographers for such poor practices. You cannot repeat a wedding. You cannot bring back a deceased person. Those memories would be lost forever all thanks to a poor workflow and no backups. Don’t make the same mistake – re-evaluate your current workflow processes and make sure that you back up after every single photo shoot.

Obviously, for a serious amateur, semi-pro or a pro, a simple manual backup process with two external drives, or one external drive + cloud storage would not work for a number of reasons:

1.   A single, cheap external drive is not only unreliable, but also too slow to handle thousands of images from every shoot

2.   External drives are often limited in capacity, which limits how much data can be backed up to them

3.   One should seriously evaluate cloud storage reliability, bandwidth capacity and simulate failure to see how much time a  restore would take to complete. For critical shoots and time-sensitive jobs / clients, cloud storage might not be an option

A more serious approach should be taken with establishing a good backup strategy, which involves fast, reliable storage and images stored in multiple locations. You want to follow the 3-2-1 backup rule, which states that three copies of data should be kept (1 primary and 2 backups), with files kept on two different media types and one backup being offsite. So far that’s the only option that I talked about for a reason – it is the best backup strategy and it really works.

Now if you don’t have a lot of content and you have a lot of time on your hands, you could adopt the above-mentioned workflow for hobbyists and still be fine. However, if you cannot afford to be down and your business heavily depends on your ability to quickly process images, you should look into more serious storage solutions.

First, start off with your editing machine. Does it have enough redundancy built-in to keep you going if a primary drive fails? While a lot of photographers prefer editing from a laptop with a single hard drive, I would explore the possibility of adding a secondary drive in a mirror / RAID 1 configuration. This way, no matter where you are, you won’t have to worry about dealing with repairs and restoring your work – the data is duplicated on two drives. If one of the drives fails, you can continue working and you can replace the failed drive at your convenience. However, this does not mean that you should think of the second drive as a backup, it is NOT (see more on this below)!

If you have a dedicated workstation, ideally, you should have a separate volume dedicated for photos, which is hosted on two drives in a mirror configuration (protected with RAID 1). In such configuration, if one of the drives fail, you will have the exact copy of your photos on the other drive. So if you experience a failure, you just replace the failed drive and the rebuild process will be very fast. In comparison, a failed RAID 5 volume might take several days to complete and during this process, everything will slow down to a crawl.

RAID is not a form of Backup!
Whether you run RAID 1 (mirroring), RAID 5 (striping) or any other type of redundant RAID, never make an assumption that your data is backed up and you do not need to back it up elsewhere. RAID does not replace backup. When RAID volumes need to be rebuilt, especially if it is a RAID 5 volume, there is always a chance that another drive might fail during the rebuild process. In fact, most people buy drives for a RAID volume in the same time-frame, which increases the chance of more than one drive dying at the same time. RAID is great to keep your storage running and keep you working, but you should always have at least two more backups, as explained above.

What about external storage? Remember, for a 3-2-1 backup, you will need at least two more backups on different media, with one being an off-site backup. Which means that aside from your working machine, you will need a primary backup device and a secondary backup device that will be taken off-site. I recommend that you size your primary backup device with a good potential for future growth. I personally have over 3 TB of just images alone and growing rapidly. So for me, a small storage box with a couple of drives simply won’t cut it. After going through a number of different storage solutions, I ended up choosing the Synology DS1515+ for my current needs. It is a very powerful device that hosts a total of 5 drives (4 TB each) and gives me roughly 20 TB of total storage. I put everything I have on the DS1515+ and it still has plenty of storage left for me to use.

The way I would size your storage needs is by looking at each year of images. How fast is your data growing each year? If you have a lot of data and you don’t want to invest in a big storage box, one solution would be to start archiving your work. By using this method  – you archive your old images to external drives and mirror the data. Then keep two mirrored drives in a closet and another drive in an off-site location. You would do this for each year. To mount and dismount drives when you need access to them, you would need an accessory device which is connected via USB 3.0 to your machine. I personally prefer all of my data to reside in one place, because I find myself accessing older images often and I do not want to store so many hard drives. There is no right or wrong – you can go with either way, as long as you are not forgetting about 3-2-1.

As for off-site backup, there are plenty of different options, including options for cloud storage. I have been backing up to CrashPlan as my cloud storage solution.  They do offer sending the initial backup through to them on a hard drive, if you are not looking forward to uploading massive amounts of data.

To recap, here is my recommended backup process for serious amateurs, semi-pros and pros:

1.   Dedicate two separate hard drives for images on your editing machine. Put them in RAID 1 configuration for redundancy.

2.   Get an external storage array with 4+ drives to make it your primary backup volume.

3.   Either use another external storage for off-site backups, or subscribe to a solid cloud solution tailored for photographers.

4.   Use automated software for backups to replicate data in order to improve speed and efficiency of backups.

5. Summary

The thought of potentially losing data is very scary, especially when your work depends on it. If you have been slacking on establishing a solid workflow process with a working and proven backup strategy, don’t wait until you lose data – act now and save yourself a lot of time and money in the future. It might require a bit of effort and some investment, but it is totally worth it!

Please keep in mind that the above-described processes work for me and many others, but they might not be ideal in every case. Take time to evaluate your backup needs and come up with a solution that you can actually simulate for a potential disaster. You do not want to end up with something that does not work, or you are too lazy to properly execute. Make it a strict and a routine practice to not only back up your photos, but also check on those backups, making sure that they are taking place successfully.