HELP ASAP...WITH STORAGE PLS

[sbunting]

My thanks for making me (more)paranoid

My pleasure!

Seriously, though, it's something we should all consider more carefully. The cloud, in general, came to being in a huge, fast wave. Simply put there were not enough knowledgeable people out there nor a body of knowledge for cloud security. And that's the external threat. And then there are the insiders. Who are these people? Having been in data centers, I've seen it. People come. People go. They are from various vendors. The person who admits them, the gatekeeper, does so if they are on a list, but once they are inside, they can go anywhere and do anything. At one, I had complete and total root access to the entire system, globally. Yet, I needed an escort only to use the restroom. Once inside, I could do anything. No one checked what I was doing or checked anything I carried in or out.

So, again, choose wisely my friends.

 

[MS Coast]

Neither FAT or exFAT are fault tolerant. EXFAT is more extensible (more and larger files), but not more fault tolerant. Best always to have UPS in place to level out the power bumps, which exist on almost every electrical system. That helps, but NTFS is far better, as it has journalling and has robust security features as well. HFS / APFS for Apple are the better choice for Macs with those same features.

Phil Harvey's tool is the one I reference. Aside from the obvious, viewing and searching EXIF data, it has some file management capabilities as well. I use this command to do as I described above:

exiftool '-Directory<CreateDate' -d /Volumes/DATA/ImageArchives/%Y/%Y%m%d%a -r /Volumes/DATA/TempStor/

This is for a Mac system, but a simple swap will change it to run on Windoze, i.e.

exiftool '-Directory<CreateDate' -d D:/DATA/ImageArchives/%Y/%Y%m%d%a -r D:/DATA/TempStor/

Just tweak the part in blue to fit your Windoze directories and drives. The first parameter in blue describes where the finished data will go. The second parameter in blue describes where the data to be processed resides. The -r tells it to function recursively, i.e. all data at and below the described path.

There are a few GUI's out that for both Mac and Windoze that use exiftool under the hood. I use it strictly command line, but Google should land you where you need to be if you go that route. I'm not sure if any GUI would provide for the above command, but perhaps. Harvey's tool is free and the true power and flexibility of it lies in the command line.

Thanks very much. Good stuff.

I'm comfortable and content executing programs with command lines. I learned with CP/M on a North Star Horizon (came in a wooden cabinet, like an old stereo receiver) and a TRS-80, back before Microsoft got involved.

 

[marcosvc]

I use external HDD to store my photos and videos. I have a external HDD for back-up, and a 2nd external HDD for back-backup, and is stored in other location.
The best edited videos, and pictures I also store it in internet/cloud space. Eg. Google Drive and OneDrive.
I already had a HDD crash, but never lost any material.

 

[GFields]

I need help!!!! Please, ive been overwhelmed with trying to find out the best way to organize all of my storage. Between my cell phone (which is obviously used with my drone), to the SD on the Drone itself, to my laptop. Currently all three are almost full. Suggestions on best ways to organize. Thanks in advance!

I use both Google Hard Drive and Microsoft One Drive. Although I have plenty of local space, I use the off-site platform for two reasons.

1. I allow the client to download their images/videos from here.
2. In the event I have a hard drive failure, my work is safe and secure.

 

[WilliamDaugherty]

Here's my workflow:
1 - Copy MicroSD contents to 4TB scratch disk on Workstation #1 (WS1). Files are organized according to year/month/day folder hierarchy.
2 - Copy this same data from WS1 to a 50TB Synology NAS.
3 - Once NAS copy is complete, I format MicroSD card and return to specific aircraft.
4 - WS1 does near-real-time backup to Google Cloud
5 - When it is time to edit, I access the WS1 drive from Workstation #2 (WS2) and copy relevant files to very fast NVMe drive internal to WS2. Once final product is generated I move those files to Microsoft Cloud for distribution to customer. MS Cloud files are in turn downloaded to three workstations (full bidirectional sync).

That's my normal routine every time I absorb new footage. At the end of each year I copy the yearly folders from 3 years ago to external SSD, then delete them from the respective scratch drives. NAS files are never deleted. External SSD drives are replaced every five years, with increasingly larger units. Older drives are stored in a fire safe.

If this sounds like a lot, it is, but it provides a high degree of redundancy and allows me to sleep at night. I've been shooting stills and video decades before I started flying drones, so the habits have been long in the making.

One last note. If I'm flying a particularly important mission I will change cards when I change batteries. I will often fly multiple aircraft also, just in case there's a problem with the bird that isn't apparent while flying.

 

[MS Coast]

Neither FAT or exFAT are fault tolerant. EXFAT is more extensible (more and larger files), but not more fault tolerant. Best always to have UPS in place to level out the power bumps, which exist on almost every electrical system. That helps, but NTFS is far better, as it has journalling and has robust security features as well. HFS / APFS for Apple are the better choice for Macs with those same features.

Phil Harvey's tool is the one I reference. Aside from the obvious, viewing and searching EXIF data, it has some file management capabilities as well. I use this command to do as I described above:

exiftool '-Directory<CreateDate' -d /Volumes/DATA/ImageArchives/%Y/%Y%m%d%a -r /Volumes/DATA/TempStor/

This is for a Mac system, but a simple swap will change it to run on Windoze, i.e.

exiftool '-Directory<CreateDate' -d D:/DATA/ImageArchives/%Y/%Y%m%d%a -r D:/DATA/TempStor/

Just tweak the part in blue to fit your Windoze directories and drives. The first parameter in blue describes where the finished data will go. The second parameter in blue describes where the data to be processed resides. The -r tells it to function recursively, i.e. all data at and below the described path.

There are a few GUI's out that for both Mac and Windoze that use exiftool under the hood. I use it strictly command line, but Google should land you where you need to be if you go that route. I'm not sure if any GUI would provide for the above command, but perhaps. Harvey's tool is free and the true power and flexibility of it lies in the command line.

@sbunting, I took your advice and picked up another SSD so I could shuffle files back and forth and then reformatted all my SSDs from exFat to NTFS. Thanks again for the suggestion and the background information.

 

[sbunting]

@sbunting, I took your advice and picked up another SSD so I could shuffle files back and forth and then reformatted all my SSDs from exFat to NTFS. Thanks again for the suggestion and the background information.

If nothing happens, you'll know it was a good decision. Peace of mind is a good thing when it comes to data. Data recovery is a part of my business, much like a dairy farmer sells veal. So I hear no end to horror stories about data loss. Every case was totally preventable. NONE of them had backups or they would not have been calling ;->

 

[Solsearcher]

My career background is IT infrastructure (Servers, Switches Routers, SANs etc.), for over 25 years for what it's worth. I specialized in Enterprise (Big Business) BDR (Backup and Disaster Recovery) solutions for several years.
I'll put this in the simplest layman's terms as possible.
Putting data on any a single drive (regardless what flavour... solid state/SD Card or whatever) is NOT a viable backup plan. Always have redundancy. NEVER put data on a single drive thinking you have a backup.

Buy a two bay NAS appliance (Synology or QNAP), stick two hard drives in it and have them RAID protected. (Data is copied onto both drives so if one fails you still have it on the other) Never put your data on a single drive thinking that you now have a backup. (Yes, I said it twice!) If the drive fails *poof", no more data. Best solution if the budget can support it is a combination of a NAS locally (in your home) and cloud storage as well.