Why can't ftp be used to download dat files?

[rfc]

Just got a June 2018 built MPP under refresh; Using @BudWalker's excellent instructions to keep the number of dat files to a minimum using ftp to the bird, I noticed there were a few from December 2017. Out of curiosity, I tried to download them using DJI Assistant 2, and (as many have reported), no files were in sight. But I know they're there because I saw them using ftp.

So why can't the files be transferred off the bird using ftp?

 

[BudWalker]

Just got a June 2018 built MPP under refresh; Using @BudWalker's excellent instructions to keep the number of dat files to a minimum using ftp to the bird, I noticed there were a few from December 2017. Out of curiosity, I tried to download them using DJI Assistant 2, and (as many have reported), no files were in sight. But I know they're there because I saw them using ftp.

So why can't the files be transferred off the bird using ftp?

They won't transfer if they are bigger than about 100 MB. But, if they do transfer they will be encrypted.

 

[sar104]

The obvious follow up question - if they are there, why doesn't DJI Assistant 2 see them?

 

[rfc]

They won't transfer if they are bigger than about 100 MB. But, if they do transfer they will be encrypted.

When you say encrypted, do you mean any different than if the files came down through Assistant2? If that's the case, then the answer to SAR's question probably lies with DJI...They make both the hardware and the software "on both ends of the line".
Other than the randomness of this, it can't be by total accident that it's getting harder and harder to consistently download these files. Might just be my conspiratorial thinking.

 

[BudWalker]

When you say encrypted, do you mean any different than if the files came down through Assistant2? If that's the case, then the answer to SAR's question probably lies with DJI...They make both the hardware and the software "on both ends of the line".
Other than the randomness of this, it can't be by total accident that it's getting harder and harder to consistently download these files. Might just be my conspiratorial thinking.

The ftp server on the Mavic first encrypts the .DAT before transferring it. You can actually see it creating the encrypted version by looking in one of the directories (I forget which, at the moment). Apparently, there is a bug in the software that creates that encrypted version as it hangs up at the 100 MB limit I mentioned. If the encrypted version does get created then it will get transferred.

That encrypted version is easily decrypted if you know the key and initial vector. At present that key and initial vector are recorded in two environment variables that can be seen by rooting the Mavic and then running a shell under adb. I did all this but didn't see the point of making it available because of the 100 MB limit. And, I could see that the lost .DAT files couldn't be made available without the Mavic being rooted and then some special purpose software installed. Instead I opted for the providing the FTP cleanup procedure being made available.

As to the question about DJI's intent I have a theory. What DJI has done is like installing the best lock on your front door and then leaving a key under the mat in front of the door. All you need to do is keep that key with you to get the best security. DJI wants to sell to customers that need or want unbreakable security. Maybe they already are. Like, for example, a government that wants 1000 military purpose drones. DJI doesn't really care if we, retail customers, can find the key under the door mat. It'd be easy to provide a configuration where only the customer could choose the key and initial vector and there would be no way for a third party to obtain them. Just my 2 cents.

BTW, I've noticed that the newest FW (01.04.0400) may have fixed the problem where old .DATs aren't deleted when space is required for a new .DAT. Still looking at this.

 

[rfc]

Fascinating, Sir.
To be clear: If one (like me) just wants the security of knowing they can retrieve the files in the case of an event, so that the gurus here (yourself included), can analyze the greatest amount of data available, it's no good to get the files via ftp (even if they're smaller than 100mb) because they're encrypted, correct?

That leaves me with Assistant 2, in which I can't see them at all, unlike my older (Nov 2017) bird. I do see I have the option in Assistant 2 to get from 01.04.0400 back to .0100, (That's what was on the previous bird), so I might try that.

This is just disaster planning, nothing more, lol. Should probably have more confidence in myself (as well as the MPP).

Finally, You're saying that for every Mavic sold, DJI has " the key under the mat" without doing backflips to root, etc., correct?

 

[BudWalker]

Fascinating, Sir.
To be clear: If one (like me) just wants the security of knowing they can retrieve the files in the case of an event, so that the gurus here (yourself included), can analyze the greatest amount of data available, it's no good to get the files via ftp (even if they're smaller than 100mb) because they're encrypted, correct?

That leaves me with Assistant 2, in which I can't see them at all, unlike my older (Nov 2017) bird. I do see I have the option in Assistant 2 to get from 01.04.0400 back to .0100, (That's what was on the previous bird), so I might try that.

This is just disaster planning, nothing more, lol. Should probably have more confidence in myself (as well as the MPP).

Finally, You're saying that for every Mavic sold, DJI has " the key under the mat" without doing backflips to root, etc., correct?

To be clear: If one (like me) just wants the security of knowing they can retrieve the files in the case of an event, so that the gurus here (yourself included), can analyze the greatest amount of data available, it's no good to get the files via ftp (even if they're smaller than 100mb) because they're encrypted, correct?
Correct.

Finally, You're saying that for every Mavic sold, DJI has " the key under the mat" without doing backflips to root, etc., correct?
Sorry, I don't understand the question here. My key under the mat reference was just me trying to say that DJI has created an unbreakable encryption scheme but has then made it relatively easy to bypass (by leaving the key under the door mat). So the drones that you and I buy from DJI aren't really secure. But if customer wants to pay for a secure system it's easy to remove the ability to bypass (don't hide the key anywhere that can be found). E.g., a military wants to develop a recon drone that is worthless to the enemy if captured.