Mavic Flight Log Retrieval and Analysis Guide

[sar104]

For all the good work you’ve done for us here, and helping so many recover their lost drones, someone should’ve got you an MM for Christmas.

I don't know - those things are pretty skittish - at least that's what people are saying....¡

 

[BudWalker]

BRAVO !!!

Thank you so much for such an informative, accurate, and useful post. I fear that we will be seeing an exponential rise in MM "fly-away" and crash posts in the coming weeks and days.

I don't see DJI's logic in making the aircraft DAT log files inaccessible to us with the latest drones. The information in those files helps everyone learn. Since they can assure the on-board FC data cannot be manipulated, there really is no reason to hide it from the owners/pilots. The only reason anyone could come up with is that they don't want a third-party data analysis to conflict with their conclusion when a crash occurs. However even that logic is faulty because we still have the controller logs...up to a point.

Happy Holidays to you and thank you for all the insight you give to this forum!

Another possibility is that the AC .DATs are encrypted because DJI wants to keep it's technology from being understood and used by competitors.

BTW, most of the log files are not encrypted - they are just encoded. It's just the on board .DATs of the Mavic Air, Mavic 2, and, probably the Mavic Mini that are encrypted. The .txt produced by the Go App or the Fly App is weakly encrypted. For more look here

Mavic Mini crash (motor falling)

 

[Skywatcher2001]

Thank you @sar104 . . . many drones made it home because of you, @BudWalker and the others. This work will save countless others.

Carry on Mister . . .

 

[sar104]

Thank you @sar104 . . . many drones made it home because of you, @BudWalker and the others. This work will save countless others.

Carry on Mister . . .

Thanks. Hopefully it will be an ongoing project.

 

[DanMan32]

Another possibility is that the AC .DATs are encrypted because DJI wants to keep it's technology from being understood and used by competitors.

BTW, most of the log files are not encrypted - they are just encoded. It's just the on board .DATs of the Mavic Air, Mavic 2, and, probably the Mavic Mini that are encrypted. The .txt produced by the Go App or the Fly App is weakly encrypted. For more look here

Mavic Mini crash (motor falling)

I originally thought the AC DAT file was encrypted to prevent tampering and falsifying flight data to get an issue warranteed. There would be other ways of ensuring data authenticity and still be readable. But then after reading Sar's explanation that the txt and mobile DAT files don't contain all raw sensor data, then I too considered that DJI is trying to prevent reverse engineering of the FC algorithms.

 

[tednol]

Can I just say how fascinating I’ve found reading threads in this section of the forum. I’m a new MM owner. I’ve had a number of faultless and enjoyable flights, in part no doubt due to some good fortune. I’ve learnt so much from a few hours reading the experiences of people who have been less lucky, and I’m going to be a more thoughtful and considerate pilot going forward. It’s also brought home to me a few home truths about RTH, and what can and cannot be realistically expected of it. I’d probably have tried to rely on it if I ever found myself in trouble prior to reading other people’s experiences. I now understand it is far more an option of last resort. That might save me a painful lesson down the road one day. Thanks again to everyone who has shared stories and all who have worked to make some of the endings happier than they otherwise may have been.

 

[tednol]

One question I meant to ask - can any other MM pilots tell me whether they have .dat files in the location specified in the first post? My folder is empty. Perhaps a reinstall is needed.

 

[sar104]

One question I meant to ask - can any other MM pilots tell me whether they have .dat files in the location specified in the first post? My folder is empty. Perhaps a reinstall is needed.

I don't have a Mini, but I've looked at quite a few DAT files from Minis, so they are definitely being created.

 

[jhe888]

Fascinating.

 

[Dave Maine]

Do a search on “.dat” at the top level of the directory.

 

[new2mavic]

I used to believe that each flight ends up in the creation of a pair of log files ( .TXT and .DAT ) but I just found that this is not so at least for the Mini.

I have just done an experiment in which ten flights were made indoor. The phone was an iphone 11 pro. In each flight I just made the Mini ascend and descend for a number of times without flying around. In the Nth flight, N ascend/descend cycles were made so that I can tell easily from the VSP height chart the start and stop of each flight.

I ended up in getting ten .TXT files, one for each flight but only three .DAT files. The VPS height chart from each .DAT file is given below. It seems that the .TXT file is closed after landing but the .DAT file will remain open and closed only when some other events occur. In this experiment, only two events have triggered the closure of .DAT files :

- drone turned off
- DJI FLY app is killed

 

[slup]

I used to believe that each flight ends up in the creation of a pair of log files ( .TXT and .DAT ) but I just found that this is not so at least for the Mini.

I have just done an experiment in which ten flights were made indoor. The phone was an iphone 11 pro. In each flight I just made the Mini ascend and descend for a number of times without flying around. In the Nth flight, N ascend/descend cycles were made so that I can tell easily from the VSP height chart the start and stop of each flight.

I ended up in getting ten .TXT files, one for each flight but only three .DAT files. The VPS height chart from each .DAT file is given below. It seems that the .TXT file is closed after landing but the .DAT file will remain open and closed only when some other events occur. In this experiment, only two events have triggered the closure of .DAT files :

- drone turned off
- DJI FLY app is killed
View attachment 91832

Can confirm this, my Air do this also ... had a recent flight session consisting of 3 .txt but only 2 .dat. In reality it was 2 flights but in one of them I landed & took off again.

 

[sar104]

I used to believe that each flight ends up in the creation of a pair of log files ( .TXT and .DAT ) but I just found that this is not so at least for the Mini.

I have just done an experiment in which ten flights were made indoor. The phone was an iphone 11 pro. In each flight I just made the Mini ascend and descend for a number of times without flying around. In the Nth flight, N ascend/descend cycles were made so that I can tell easily from the VSP height chart the start and stop of each flight.

I ended up in getting ten .TXT files, one for each flight but only three .DAT files. The VPS height chart from each .DAT file is given below. It seems that the .TXT file is closed after landing but the .DAT file will remain open and closed only when some other events occur. In this experiment, only two events have triggered the closure of .DAT files :

- drone turned off
- DJI FLY app is killed
View attachment 91832

That's correct - as explained in post #1 in the section on log file types.

 

[BudWalker]

I used to believe that each flight ends up in the creation of a pair of log files ( .TXT and .DAT ) but I just found that this is not so at least for the Mini.

I have just done an experiment in which ten flights were made indoor. The phone was an iphone 11 pro. In each flight I just made the Mini ascend and descend for a number of times without flying around. In the Nth flight, N ascend/descend cycles were made so that I can tell easily from the VSP height chart the start and stop of each flight.

I ended up in getting ten .TXT files, one for each flight but only three .DAT files. The VPS height chart from each .DAT file is given below. It seems that the .TXT file is closed after landing but the .DAT file will remain open and closed only when some other events occur. In this experiment, only two events have triggered the closure of .DAT files :

- drone turned off
- DJI FLY app is killed
View attachment 91832

The .DAT on the AC itself starts recording at about 2 secs after the battery is turned on and then terminates when the battery is turned off. But, if the size of the on board .DAT exceeds a size threshold then the current .DAT is terminated and the next .DAT is started. E.G., on the P3 if FLY003.DAT is being recorded and reaches a size of 350 MB then FLY003.DAT is closed and FLY004.DAT is started.

The App version of .DAT will have a label that reflects the one being recorded by the AC. EG an on board FLY050.DAT will create an App .DAT label like 20-01-20-02-10-28_FLY050.DAT. There have been several instances where the the App .DAT will terminate for no apparent reason and a new .DAT starts recording. This new .DAT still has the same FLYxxx.DAT but the prefix will change to reflect the time it starts recording. In this example the new .DAT could have a label like 20-01-20-02-11-42_FLY050.DAT.

In your experiment
1) the App .DAT number should be different for the first two .DATs
2) the App .DAT number should be the same for the 2nd and 3rd .DATs. Bur, they should have different prefixes like described above.

BTW, the .csv obtained from converting the .txt has the column HOME.dataRecorderFileIndex that will have the index of the on board .DAT. This value makes it way to the static value display area in CsvView

 

[new2mavic]

In your experiment
1) the App .DAT number should be different for the first two .DATs
2) the App .DAT number should be the same for the 2nd and 3rd .DATs. Bur, they should have different prefixes like described above.

Yes, that's exatly the case :

2020-01-24_13-15-39_FLY016.DAT
2020-01-24_13-18-52_FLY017.DAT
2020-01-24_13-23-30_FLY017.DAT

 

[Deleted member 94047]

I have had, in at least one occasion, what looked like a DAT file beginning, closing immedietly, and another with the the same FLY# starting; resulting in one txt log and two DATs. But the first proved to be a 1KB file with seemingly no data at all.

 

[BudWalker]

I have had, in at least one occasion, what looked like a DAT file beginning, closing immedietly, and another with the the same FLY# starting; resulting in one txt log and two DATs. But the first proved to be a 1KB file with seemingly no data at all.

I should have mentioned this. That small .DAT is likely the DatHeader which contains, among other data, the data required to determine the platform and, from that, the clock speed. The next .DAT won't have a DATHeader since it was split off. To process this DATHeader_less .DAT DatCon and CsvView need to know that an invalid DATHeader is OK. This is true by default and can be changed


The other option is to create a whole .DAT by concatenating the small DATHeader with the larger DATHeader_less file. Something like
%cat [small_DATHeader] [larger_DATHeader_less] > whole_DAT

 

[RodPad]

@sar104,
I just found this thread.

Rod

 

[new2mavic]

Hi, I have got a .dat file from my friend who just lost his M2Z. However I cannot covert it to csv using CsvVeiw which gave an error message "Unrecognized log file". May I know what's wrong with the file ?

 

[BudWalker]

Hi, I have got a .dat file from my friend who just lost his M2Z. However I cannot covert it to csv using CsvVeiw which gave an error message "Unrecognized log file". May I know what's wrong with the file ?

The problem is that the extension was changed from .DAT to .dat. DatCon explicitly requires the extension be .DAT.

There are 2 or 3 .dat files that can be found in the Go or Fly app directories. None of these work - the only one that works has a .DAT extension. That's capital D-A-T