I need help!!!! Please, ive been overwhelmed with trying to find out the best way to organize all of my storage. Between my cell phone (which is obviously used with my drone), to the SD on the Drone itself, to my laptop. Currently all three are almost full. Suggestions on best ways to organize. Thanks in advance!
Understand your problem. The amount of data generated by drones and cameras generally overwhelms people at some point, especially when you ramp up the quantity and quality.
Data forensics is my business and photography is my passion and both, especially the former, require data storage needs that push the limits. Storing data, archivally, is an absolute business requirement for me and my photography benefits from that.
One of the very first and foremost things you must understand is this; every hard drive, SSD, media card of any type will eventually fail. Stop! Reread that and ponder it.
That having been said and understood, any data you care to store or archive or otherwise not destroy, should be stored twice, i.e. even your main archival storage must be replicated. There are many ways to do this, but you must store it twice, once to archive and one to disaster-proof it.
The second thing you must understand is this; the manner in which DJI stores data on the media devices it uses, be it internal memory, on-board SSD, or external MicroSD card, will be either FAT formatted file systems, if less than 32 GB's, or EXFAT formatted file systems, if 32 GB's or greater. They do that because FAT or EXFAT are cross-platform, meaning they can be read and written to by DJI (using Linux), Windows, Mac, or Linux. Unfortunately, neither FAT nor EXFAT are journalling file systems, meaning they are not fault-tolerant. If there is a power blip during a write operation, it can and often does end in a corrupted file system. A journalling file system operates in a manner that safeguards data from power blips by writing data changes to a journal. Before that occurs a dirty bit is set and the data is written first to the journal. Next the actual changes to the data are written from the journal. When done, the dirty bit is cleared. Thus if a power blip occurs during a data write operation, when the power comes back, the file sees the dirty bit, remedies the corruption, clears the bit, and the data is preserved. That's the Reader's Digest version.
So FAT and EXFAT are not safe file system formats on which to store any data your care about for any length of time beyond which is necessary. Further, storing data on a MicroSD that is flying around the air, subject to crash, fly-away, water-immersion, etc is simply not a good practice unless it is at least backed up on the ground.
At the end of any flying session, my data is immediately removed from DJI storage and stored twice on hard drives that are formatted with a journalling file system. On Mac, that's HFS or APFS. On Windoze that's NTFS. No exceptions. The media card is then cleared, formatted.
So we now understand that we must store data twice (one primary storage and one backup) and that the file systems hosting that storage should be journaled file systems.
Once you get those principles into your mindset, you can decide how much storage you need. Another decision is where to have your backup. A fire takes out both if they are located together.
How much storage depends on your needs. Regardless of how much you buy, you will eventually fill it up. So buy twice what you think you'll need and double it again for your backup.
How you discipline yourself to store / organize your data will vary according to your personality and needs.
On any given shoot, you will have keepers and obvious throw-aways. Those are the easy ones. The middle ground is more difficult. On some shoots, expensive safari, you'll save probably everything. Shooting in your backyard, you'll probably just keep the really good ones.
So my 'keeper originals' go to a folder on my archive storage that is backed up. I have a folder for my still cameras, one for each model, and another folder for my DJI birds. That's a temporary holding area. Before they go there, I use Adobe Bridge to tag them in batches with keywords so they can be eventually searched. At some point, I open up that archival machine and run an exiftool command that reads the exif data, namely the creation date/time, creates a folder for that day, and then moves files into folders based on the creation day. That looks like screenshot #1. So you can go the year folder and to a specific day folder to locate files. The tags allow you to search if you don't have a date. Since my 'post-processed' images always retain the original file name, prefixed by something more meaningful, I can always search for that original file name.
My keepers will have a processed version stored elsewhere, also backed up. For any given day's activity, I create a folder beginning with the ISO 8601 date format YYYYMMDD and then append to that a meaningful name. That appears below as screenshot #2. By having the ISO date format as a folder name prefix, a folder name sort puts them in the order they were created. Any file in such a folder is also prefixed by the YYYYMMDD, which is followed by a meaningful name and suffixed by the original file name. Again, easy sorting and searching. See screenshot #3.
That's my system. It works for me. Regardless of how you name your files and organize them, just remember to store them twice and to store them on journaled file systems.
If anyone is interested in using exiftools as a an automated file management tool, let me know and I'll create a mini-tutorial on how to do it.
Best....
