PetaPixel

A Cautionary Tale: How a Bug in Dropbox Permanently Deleted 8,000 of My Photos

dropbox3

TL;DR: If you are using Dropbox as a sole backup of your files, think again. Without making a mistake, you might lose your files.

I started using Dropbox back in 2009 and have always loved the service. Over time, I kept moving more and more files to my Dropbox folder and eventually had to upgrade to the Pro plan to keep up with the space requirements. In particular, I moved there all of my photos in order to be able to view/share them online and also to have them backed up.

In April of this year, a hard drive in my laptop was running low on space so I decided to use the Dropbox’s Selective Sync feature to unsync some large directories from the laptop. Because there was never any problem with the service and also because it’s already the year 2014, I thought it might be about time that one can trust a cloud-based storage service and use them as a sole backup of their files. Boy, I was wrong.

On April 29th 2014, I opened the Selective Sync dialog, unchecked directories called 2003, 2004, …, 2014 from the Photos folder (and few other directories) and clicked the ‘Update’ button. After that, the Dropbox client froze and didn’t show any sign of life for a couple of minutes, so I decided to kill it and restart it again. These directories are large and they might be too big of a bite for Dropbox, I thought, and unsynced them one by one instead.

Everything worked well, the directories disappeared from the local hard drive, but they were still available on Dropbox’s website. All good.

dropbox1

About two months later when I was preparing for a defence of my PhD thesis, I was looking for an old presentation but couldn’t find it. The directory was there but it was empty. I would have never deleted these files, something must have gone wrong.

I contacted Dropbox support, who then broke the news to me: there was a delete event of 8343 files from 2014-04-29 at 14:57:30 GMT (UTC). Looking at the log record from this event, I realized most of the missing files were my photos! All the directories were still in place but many of them were empty, as if Dropbox randomly deleted some files and left some others intact. I was devastated. All those memories and the effort with collecting and organizing the photos… gone.

dropbox2

From all this information it seems that Dropbox client first deletes files locally before it informs the server about the new selective sync settings. Consequently, if the client crashes or is killed before the server is contacted, the files remain deleted without any trace. After the client restarts again, it only sees there are some files missing and syncs this new state with the server.

Unfortunately for me, Dropbox only keeps a copy of deleted files for 30 days (unless you pay $40/year for some Packrat feature) and I found out about this event after two months.

For me it’s most likely over, there is nothing I can do now, maybe try to get some photos back from my friends who went on the same trips… Anyway, I decided to share this story publicly for three reasons:

  1. A bit of publicity might convince Dropbox to put some extra effort and maybe find a way to restore my files.
  2. To let other Dropbox users know about this issue so that this doesn’t happen to them. Basically after doing any major changes in Dropbox settings, one should check the Events page on Dropbox website to make sure no files got deleted.
  3. Dropbox team can use information from this story to fix the problem. Here are few suggestions what can be done:
    • Make the application of the selective sync settings transactional, either is succeeds completely or fails gracefully.
    • Enable the Packrat feature by default for every paying customer, without any extra cost. Looking at the storage plan pricing for Google Drive, there surely is a space for you to do that. Or, at least, keep the delete history as long as the deleted files fit into user’s quota, it’s a space users already paid for.
    • Use machine learning to detect a strange activity on user accounts and notify users by email if it happens. For example, most of my activity over the years consists of adding large number of new files and changing/deleting a small number files from time to time. A sudden deletion of 8343 files is surely a strange activity in this context..

Update (7/30/14): Yesterday night I got an email from Dropbox saying that they worked with their engineering team and were able to restore 1463 files. Also I received some credit for my future use of the service. Better than nothing.


PS: At least I defended my PhD thesis even without that lost presentation, but the photos are gone nonetheless.


PPS: For the sake of completeness, you can read a complete record of my communication with the Dropbox support by clicking here and scrolling to the bottom.


About the author: Jan Čurn is a Ph.D. student in the Distributed Systems Group of the Department of Computer Science at Trinity College Dublin and a CTO at VirtualRig Studio, an application for creating realistic motion blur for professional car photography. You can learn more about Čurn by heading over to his website or following him on Twitter. This article originally appeared here.


 
  • pgb0517

    But you didn’t learn to maintain a local backup?

  • Paul-Simon

    Jottacloud + local backup is my way of doing it.
    Never keep only one copy of your photographs. You should’ve known this by now.

  • Scott Squires

    Yes, there are issues with Dropbox. One of the other major problems has been if you share your Dropbox (not just provide a link) then anyone you share can delete anything shared. And because people don’t think about the sync aspect they tend to delete unneeded folders to them.

    If you have a project and a number of other co-contribuotrs to the project (i.e. film project, work, sharing photos with friends or family from an important event (wedding, etc)etc) and later someone finishes their work on it or leaves the group they will likely just trash the folder on their drive thinking they don’t need this x MB of stuff anymore taking up space. When they do that they not only delete it from their drive, they delete it from everyone else’s drive and from Dropbox itself. Sicne there’s little to no notification it’s likely the deletion won’t be noticed until someone tries to get to the files. As pointed out if for some reason that doesn’t happen within 30 days you’re out of luck.

    Unlike most file systems there are no permissions so anyone who shares can read/write and delete files at will. Since people tend to think of it as begin on their machine they will likely delete without a second thought. I think Dropbox is finally implementing some basic permissions for Busienss account but that’s only if you pay for a business account and I don’t know how much control you actually have.

    So don’t share anything unless you make it abundantly clear to those you’re sharing with that they can not delete. Even then you can’t think of Dropbox as a backup of files and certainly should not be the only location of files. That means you end up having to make duplicates of everything at times and duplicating the amount of space on your machine or at least another hard drive.

    One of the other limits of Dropbox that’s not clear until you use it is you can only share the amount that people have bought. If you pay for Dropbox and have 100GB storage and want to share a 1 GB folder, unless the person sharing it has 1 GB of their own storage available, they will not be able to access the folder. So if they have a free account (5GB) then they may have to delete their own stuff before they can access your shared folder. So the common denominator of how much you share is not based on what you paid for but how much available space those you want to share with have available. Evidently Dropbox doesn’t want people to sign up for a number of free account and being able to access much. But it’s only when you bump into that limit that you find out about it.

  • Dan Tauro

    Like everyone here says keep a copy of your beloved photos in several places. They cannot be lost. I ship a hard disk with photos to my relatives place semiannually and keep photos locally backed up on different drives. Just common sense. Drives are not that expensive.

  • Tobias W.

    The most embarrassing thing about this confession is that a Ph.D. student in freaking computer science doesn’t know what a proper backup is. Having only one copy of a file, regardless where it’s stored and regardless whether it’s the year 2014, is NOT a backup. The moment you trust any one single system with your files, the blame is on you because you didn’t have a backup. It’s that simple.

    I have a copy of my library on my computer, on an additional connected hard drive, another complete copy on network attached storage at home that is switched off unless I need to update that backup or retrieve files. I share files in JPG via Flickr where I have unlimited storage. I bet my backup routine cost less than what he dished out to Dropbox and I have 3 copies of all of my files as well as a fourth copy of the most memorable stuff on Flickr.

    Seriously, the most efficient and comfortable backup strategy is a network attached storage product. You could even get a second one and store it in a separate building, like at your workplace or something to insure against burglary, fire etc.

    I’m sorry about the authors images, but a Ph.D. student in computer science should know better how to organize a backup.

  • http://hellomatt.net Matt Jensen

    Also, could they not have used a hard drive recovery application on their computer? Surely an app like Recuva could have recovered those files. Nothing is truly deleted unless it has been written over by other files in that sector.

  • Mojo

    Well said, sir. I keep at least four backups on various media formats in various locations at all times for this very reason.

  • Andrew Hollywood

    2 is 1 and 1 is none.

  • Ali

    Get a NAS for the home and then keep an external at a friends house. Then you are protected against drive failure, house fire and accidental deletion.

  • Marcus Dashoff

    Computer science is not information systems – though people seem to think they’re synonymous.

  • karel moonen

    WawaWa! dumbass, back up on drives keep in a safe.

  • Jason Yuen

    That’s a handy saying but it’s not entirely safe. What you want is at least one more off site backup. I like to have 1 copy on my computer, 1 on an archival hard drive, another copy on a hard drive off site at someone else’s house and finally one more online storage for non sensitive data. It doesn’t matter if you have 10 copies on 10 hard drives. All it takes is a fire, flood, or act of god to render them all equally dead.

  • Chris Petersen

    Dropbox is a sync and file sharing service not an archival backup service. It’s great for getting the same file between a bunch of different computers, but not a substitute for something like CrashPlan, BackBlaze, TimeMachine, Carbonite, or one of the other myriad of similar services. Or even just keeping an external HDD around with a second copy of the files.

  • Locke42

    The PackRat service is WELL WORTH IT.

    Twice I’ve had hard drives with my photos die on me. PackRat saved my ass both times by allowing me to undelete all of my photos. I end up with a lot of duplicates, but an MD5 scan quickly reveals them. These days, I make sure all of my photos are uploaded to Dropbox first before moving them off my local computer and onto network storage.

    Sometimes I’m tempted to switch to Google Drive or SkyDrive for their much better cost-per-gig ratio, but unless they have a similar sort of unlimited undelete feature, I’m sticking with Dropbox.

  • Girovago Settantasette

    Hello,
    1 – I always backup Dropbox folder to another 2 drives on both PC and MAC, my pictures are too important !!! so I use the command robocopy on PC and rsync on MAC/LINUX ( I never include the “mirror” option of the folder so that the backup “destination” has all the data at all time regardless if anything get deleted from dropbox folder “origin”). This process is scheduled to happen every night on PC using Task Manager and on MAC/LINUX using CRON JOB.

    2 – The external Drive that I use for this backup have being formatted with Encryption so that I can safely take them anywhere since I’m the only one that know the very large and difficult password to access them.

    3 – I Always use External drive of 2.5 inch size, never use the 3.5 external because they are “not reliable” since you need to plug them into the power, you never know when the power line will kill your Hard Drive … instead the 2.5 inch use USB for Power and it’s more reliable.

    4 – I have multiple External Hard Drive and I always bring a full copy to my Parents house and swap it every 6 month with the most recent backup. (this is good for disaster recovery)

    I HAVE NEVER, EVER, EVER Lost any of my 2TB of Data that goes back since 1998.

    I don’t have a college degree…

  • http://www.joachimdyndale.com/ Joe Dyndale

    First: Sorry about the lost photos, I know that sucks big time :(

    Dropbox is not a backup service per se. It’s a file syncing and sharing service. Sure, it can be used for backing stuff up, but it is far from robust for that purpose.

    I use, and highly recommend, Spideroak. They offer the most secure backups on the market (don’t just take my word for it, even Snowden says so), and they never permanently delete your deleted files unless you explicitly do it yourself in the app; almost all other backup and syncing/sharing services have the same policy as Dropbox and delete your deleted files after 30 days. Spideroak has a feature called “Hive”, which is basically what Dropbox offers, but they offer other ways to share files too (like creating a “share room” that requires a password). Their app can also be set up to automatically keep a local (any drive accessible from your machine) backup of everything it sends to their servers. It’s a bit more complex software than Dropbox, but still very easy to use. I’ve been using them for a few years now.

  • Taylor Huston

    Dropbox is for quick and easy syncing of files between multiple computers. But if you want a real actual ‘backup’ solution there are services that are intended for that. Carbonite is a backup service. Dropbox is not.

  • Michal Rosa

    “TL;DR: If you are using Dropbox as a sole backup of your files, think again. Without making a mistake, you might lose your files.” – seriously? Single back-up for important files? Geez, what a shocking lesson indeed. I’m so not sorry for people like you.

  • Tobias W.

    Either way. Someone from that field – computer science or information systems – should know what a backup is.

  • Edgar Allan Bro

    The main issue with using Dropbox for offsite backup is that Google Cloud is ten bucks a month for a terabyte. A freakin terabyte for the cost of a pint of ale.

    Local raid + media server raid + gcloud + a HDD in a safe deposit box. I’m sorted for backups from now until Mad Max time.

  • markz

    one should never trust an external service provider for primary storage if they have no legal obligation to provide practical recoverable backups (usually user available or from their first tier service provider)

    even talking about backups, a single backup is no backup once the original is lost (given the amount of my clients who just started working off their backup drive when their master failed (only to have that fail) this is a big conceptual problem for even some highly intelligent and professional folk) I price my “data rescue” rates at a level to help drive out that misconception.

    Both personally and professionally I always insist on a three tier backup as a minimum for critical (or highly sentimental) data.

    1st is your “working backup” or “online backup” such as your TimeMachine or equivalent – which is a device on the network or connected to the computer that runs regular incremental backups so you, theoretically, never have more than a few hours lost work and can supply earlier copies if you need to “go back”.

    2nd is your “father” or near line copy, usually a copy of the online backup (or the online swapped out) or the working drive created at regular intervals (daily, weekly or at most monthly).
    If this is an online backup it is preferably stored at physically different location or, if offline, locked in a fire safe above flood level if held on the premises.
    This protects you from theft of your desktop equipment or destruction your physical workplace by low level building or block events such as a lower level fire or flooding.

    3rd is a grandfather or offline copy, always held at a separate, physically remote, location, done weekly, monthly, quarterly, annual depending on volume or type of business, to protect you against more disastrous events such as local flooding or fire or a disgruntled employee with server room access and an 8 pound sledge hammer (yep, been there seen that).

  • Ringokarma

    Backing up is like flossing. Any dentist would tell you to
    only floss the teeth you would like to keep. In my circle it’s accepted that
    having one copy of a photo is like not having the photo at all. I download from
    my camera to folders on my desktop, edit images and place the edits in a
    separate folder inside the folder they are from – same folder name with edits
    added to the name. That’s step one, Then I drag the desktop folder to a 4T
    Western Digital Studio desktop HD, and drag the same folder to a second 4T WD
    Studio desktop HD (I’m at the stage now where I have six 4T Studios). I do that
    at the end of every day before shutting down. I don’t store photos on my HDs (2
    iMacs, 2 MacBook Pros, one Dell XPS); they would fill up too quickly, and
    because of the use they get they’re more likely to become problematic. OK, so
    now I have two copies of my days shooting and any edits on separate externals.
    As a backup to my backup I leave folders on my desktop for at least two full
    days so there’s also a backup on Time Machine (I use 2T WD Studio Portables for
    Time Machine). It may seem like a lot of work and costly, but it works for me.
    As a side note, I’ve been shooting digital – sports and news – since 2005. I
    have at last count 38 hard drives from the old days – internals, desktops,
    portable – everything backed up at least once except for a month and a half in
    2007 when I had a HD failure and learned my lesson. Back up the images you want
    to keep. It’s so fundamental in our business that if you lose images by not
    backing up you have no one to blame but yourself. Never trust any cloud service
    to cover your back.

  • hdc77494

    Amazon Glacier has archive storage available for one cent per gig per month or $10 per terabyte. If you need to recover old files they can drop ship the entire archive to you on a hard drive via FedEx. NEVER maintain a single copy of ANY file you plan to keep. I agree the Dropbox software glitch is terrible, but that risk is only one of the reasons to always maintain duplicate files, preferably in multiple physical locations. Your house can burn down, your PC or laptop stolen, a virus gets downloaded, you get the idea. A PhD in computer science? You’ve GOT to be kidding me.

  • hdc77494

    The only flaw in your system is that you keep both copies in the same building, always a bad idea. Do yourself a favor and get a NAS so you can automate the entire process. Doing it manually will eventually get tedious.

  • hdc77494

    Many cloud providers automatically backup customer files in multiple data centers in different parts of the country. You’d be hard pressed to reach their level of data integrity or security on your own.

  • markz

    Indeed they do, and most of these also have this as part of your service level agreement.
    Where I currently work as my 9-5 (v’s my private freelance) we use one of the top 5 global providers but our user still have to log a request with us, who pass it on to our service representative, who passes it on the their equivalent who passes it on to their service desk who pass it on to one of their many anonymous technicians…. and then ~3’ish days later your files reappear. All these people are responsible, in part, for ensuring that the cloud supplier performs to , or above, the service level agreed to when we signed up
    For the privilege of these thousands of terabytes and management of said we pay _millions_ of dollars per annum

    for your average punter on iClouDropDrive… you get … well, some level of customer service in the ball park of what you’re paying for.

  • Brian Sahagun

    Apart from the digital backups, maintain a printed backup in case there comes a solar flare strong enough to wipe us all out. And our data.

  • docholliday666

    Remember…somebody’s gotta graduate last!

  • kimaldis

    “using Dropbox as a sole backup of your files”

    Are you completely mad?

  • nerdbomber

    I’ve lost some data on DropBox as well. And I only found out about the Packrat feature when it was much too late. If only we could all get the same service as you have.. getting your files recovered.

    In other news, Dropbox if you’re you’re reading this… lower your prices to be more competitive with Google Drive and Onebox already!

  • Peng Tuck Kwok

    YEP. It ain’t too expensive to buy one of the 2 bay ones and you can get decent performing drives to go with it. If you need more invest in a 2U storage solution that lets you stack eight drives or more if more capacity is required.

  • Ryan Villanueva

    Many years ago when I was a naive little boy I put thousands of my cherished digital photos on several CD-Rs as “backup” and deleted the photos on my hard drive to save space. Some of the CDs got scratched and I never recovered a huge chunk of my photos again. Like what happened to the author it is always foolish to keep only one copy of any file. Now I keep thousands of photos on my Flickr, Google Drive, and my local hard disks.

  • Cinekpol

    Rule no.1 of survival in 21 century: Never store any data in cloud without having local copy.

  • Cinekpol

    It’s like saying that a surgeon have a right not to know what a fever is.

  • Zetahills

    Moral of the story: NEVER trust cloud storage.

    Always use your own external hard drives (two of them is best for redundancy) to backup and store files.

  • Tobias W.

    Yeah, this whole rant backfired on him. He looks like an idiot now, letting everybody know that his academic background failed to teach him the basics of his academic field. And why did he even care to include his academic background information in this rant? Did he even think about this? Maybe he woke up and thought: “Hey, today is a great day to destroy my professional credibility by making a fool out of myself on the Internet!”

    Yes, you’re right. It doesn’t get better than this…

  • user1983

    One time I went away for a weekend and came home to find my external hard drive on the floor, it had all of my pictures in it and it was the only drive I had with my pictures from 2006 in it, I was between computers. I lost all of my pictures from 2006. I learned my lesson.

    I have three external hard drives and two clouds. Never trust any one back up.

  • Marc Weisberg

    Bravo!

  • Bryan Merrick

    £30 for a trilibyte External Drive mines was a fiver in a sale from W-H-Smiths what a result,5 years down the line i still have my full untouched C-Drive, on an External Drive especially on Windows or Mac’s as it also takes so much weight off your laptop or device. Cloud is all free even with my VirginMedia or as you did in Dropbox but sorry if it aint broke why start messing-on with deleteing your much needed files Zip + Save + on a External Drive for my back-up’s to screen-savers to my programs large files adware,spyware,Ect……C’mon big chap dont pay some bogus cloud set-up if your money can be used so much better and have a easier life….PEACE

  • Bryan Merrick

    YUP : )))

  • http://visuex.com/ Michael Tunnell

    I had Dropbox delete my files as well, back in 2012…arbitrary deletion without my permission and I was no where near the limit. I of course had backups but it is still something people need to know.

    Dropbox is NOT FOR BACKUP! DO NOT TRUST IT!

  • ajfudge

    Like what he said, it has been 2 months until he noticed that his Dropbox pictures were deleted. Within 2 months, his hard disk pictures most probably have been written over or what can be recovered are just fragments now.

  • Jim McCarthy

    I loaded dropbox yesterday on Win 7 and it tried to install what looked to be like Win 8 shell on my computer! Something is seriously wrong with the software. I played holy heck deleting it! NEVER again will I try this program..it’s effed up big time.