Welcome Mavic Pilot!
Jump in and join our free DJI Mavic community today!
Sign up

Flight log retrieval and analysis guide

sar104

Dic mihi solum facta, domina.
Premium Pilot
Joined
May 27, 2017
Messages
9,919
Reaction score
11,850
Location
Los Alamos, NM
With the introduction of the Mavic Mini there has been a noticeable uptick in the number of posts requesting help with lost or crashed aircraft. While there is plenty of help and advice available on this forum, and detailed guidance and options on several other websites, I thought that perhaps a single summary of the subject might be useful. If it does prove useful then I'll try to keep it updated, either in this post or followup posts, and perhaps other forum members will contribute additional thoughts on the subject to address errors and omissions.

1. INTRODUCTION

Starting back with the Phantom 4, DJI aircraft and control software record an impressive amount of flight data to various log files both on the aircraft and on the mobile control device (phone, tablet etc.) which is extremely useful in determining the cause and outcome of unexpected events. These log files contain far more data than the simple flight summary files that appear in the DJI GO 4 app, but are not as simple to access. Most of them are encrypted, but many can be read using widely available software or web-based converters.​
Before discussing the log files themselves it's possibly worth a brief description of how these flight control systems work. Skip to section 2 if you are not interested in this level of detail. I've included some links for optional further reading.​
Ignoring fully manual mode, which is only available by changing a firmware parameter, aircraft flight is controlled entirely under the hood by the onboard flight controller (FC). It converts stick input commands to move up/down, forwards/backwards, left/right and CW/CCW into motor adjustments to achieve the requested movement. Sticks centered means hold position, altitude and heading (P-GPS mode) or hold altitude and heading with pitch and roll set to zero (ATTI mode). Stick inputs simply request specific movement relative to those baselines.​
Exactly how these flight control systems work is a rather deep and arcane subject. At the simplest level, the aircraft have either one or two MEMS strapdown IMU packages that comprise 3-axis accelerometers and rate gyros, together with 3-axis magnetometers (compass), a barometric sensor, and a GNSS module that receives GPS and GLONASS signals. Anyone with a cursory familiarity with basic mathematics will note that there is, therefore, redundancy in the flight data. In theory, given an initial position and heading, transformation of the accelerometer and rate gyro data into the earth frame of reference followed by integration with respect to time yields velocity and displacement as a function of time - pure inertial navigation. In practice it doesn't work with these kinds of IMUs, due to bias (zero offset errors) and drift (small changes in sensor gain), which lead to errors in the integrated quantities. The result, which is easy to replicate since the raw sensor data are available, is that after just a few tens of seconds of flight the inertial solution ends up noticeably incorrect.​
However, the accelerometer and rate gyro data are, in fact, the only data measured fast enough to provide responsive aircraft control, and thus they are the primary data source for the control algorithm. Once the aircraft yaw (heading) is initialized by the compass, the aircraft attitude, velocity, position and height are computed primarily inertially. The IMU sensor drift and bias issues are corrected more gradually using the GNSS, compass and barometer data by a sensor fusion algorithm, most likely a Kalman filter or equivalent, keeping the absolute velocity, position, heading and altitude correct within the accuracy of those sensors. The aircraft attitude is actually not tracked as the derived Tait–Bryan angles (pitch, roll and yaw) since that description becomes indeterminate for certain orientations. Instead the FC uses rotation quaternions which, while a more obscure branch of mathematics, has a number of advantages.​
Most flight control problems are caused by:​
  1. Loss of GNSS positioning, which will generally lead to the FC switching to ATTI mode unless it has active VPS positioning available;
  2. Disagreement between the IMU and compass yaw values, often caused by taking off from a locally magnetically distorted site;
  3. Wind speeds in excess of the design limits of the aircraft;
  4. Lost/damaged props or motor problems, leading to loss of propulsion;
  5. Power interruption due to battery disconnects;
  6. FC/IMU processor issues.

2. LOG FILE TYPES

Mobile device DJI TXT logs
These logs are created by the standard DJI SDK and run from motor start to motor stop (or signal loss), recording telemetry transmitted from the aircraft to the RC at 10 Hz in a few hundred data fields. These logs don't include any raw sensor data - only the IMU solutions that are the output from the sensor fusion algorithm. They also include battery data, gimbal and camera data, flight status and error flags, but no motor data.​
They are often all that is needed for analysis of simple events, but the lack of raw sensor data makes it difficult to identify exact causes if there are IMU problems, and the lack of motor data is a problem when there are propulsion issues. Additionally, the relatively slow data rate and a couple of tenths of a second latency means that events rapidly followed by aircraft shutdown (e.g. many crashes) are not recorded.​
Aircraft DAT files
These contain the most comprehensive data, logged at the highest rates. A DAT file is started at aircraft power up and continued until power down, and includes numerous boot sequence, sensor calibration and diagnostic data in the flight event stream. There are also hundreds of data fields, many of which are flags or diagnostic computations of unknown types, probably understood only by DJI. However, among these are the raw and processed sensor data and the IMU solution for aircraft attitude, position, velocity and heading, together with battery and motor data, recorded at rates varying from 5 Hz to 200 Hz. These data are very valuable for diagnosing flight control problems. The files exist independent of the mobile device control app being used.​
Unfortunately, on recent DJI models (Mavic Air, Mavic 2 and Mavic Mini) the decryption keys are hidden, and so those are not readable except by DJI.​
Mobile device DAT files (DJI GO 4 app)
These contain a subset of the aircraft DAT files, including most of the raw sensor data, recorded at a lower data rate of 10 Hz. They are also started at aircraft power up and closed at aircraft shutdown, provided that the app is connected - otherwise it will be the subset of that duration when the app is connected. Not as good as the aircraft DAT files, but nearly as good for most needs, and always available if the control app is DJI GO 4. DAT files are not created when using Litchi or other third party control apps.​
Mobile device third-party control app log files
Third-party control apps, such as Litchi, create their own custom log files in various formats using the DJI SDK, generally in CSV format. None that I have seen are as comprehensive as the DJI TXT log, let alone the DJI DAT log.​
Since Litchi is probably the most common third-party app, it's worth noting that, under iOS and Android, Litchi creates its own CSV log and a standard DJI TXT log in a separate directory. No DAT file though.​
3. LOG FILE RETRIEVAL

Mobile device DJI TXT logs
The log naming convention, based on the date and time of the start of the flight, is: DJIFlightRecord_YYYY_MM_DD_[hr-min-sec].txt.​
Detailed instructions are available on @msinger's excellent PhantomHelp website and also on AirData, but the basic details are as follows:​
With iOS devices running DJI GO 4 you need to access the app files via computer, either using iTunes or with a file system browser such as iExplorer. The TXT logs are in Apps » DJI GO 4 » FlightRecords. The same method can be used to access DJI Fly files in DJI Fly » FlightRecords, but with DJI Fly they are also accessible using the iOS Files app by selecting Browse » On My iPhone » DJI Fly » FlightRecords.​
With Android devices the file system should mount when plugged into a Windows machine, or via various software options on a Mac. With DJI GO 4 the TXT logs are in DJI » dji.go.v4 » FlightRecord. With DJI Fly they are in DJI » dji.go.v5 » FlightRecord.​
If you want help analyzing the data then you can either retrieve the logs and post them here directly, or upload them to PhantomHelp or AirData and post the link back here.​
AirData also has automatic log upload (sync) options directly from most of the common control apps.​
Aircraft DAT files
These are only readable from the Mavic Pro and Mavic Pro Platinum (plus Phantom 3, Inspire 1, Phantom 4, Phantom 4 Pro, Inspire 2, Inspire 2 Pro, Matrice 100, Matrice 200, Matrice 600, and Spark). A guide to retrieving them from the aircraft can be found on @BudWalker's DatCon website. They are much larger files than the mobile device files, and generally too large to post directly to this forum. The best solution is to upload them to Dropbox or similar and then post a link.​
Mobile device DAT files (DJI GO 4 & DJI Fly)
The DAT file naming convention, based on the date and time of the start of the file, is: YY-MM-DD-hr-min-sec_FLYXXX.DAT., where XXX is the flight recorder file index from the HOME_dataRecorderFileIndex field in the txt log.​
These are retrieved by the same method as the TXT logs. Under both iOS and Android they are in a subfolder, MCDatFlightRecords, in the folder that contains the TXT logs. In some cases, for reasons not fully explained but possibly mobile-device hardware related, and most often under Android, DAT files are not created and that folder is empty. Uninstalling and reinstalling the app sometimes fixes that. One user also found that the process required manual deletion of the app folder (apparently not deleted automatically in the uninstall process) before reinstalling the app.​
The DJI Fly app appears to delete the DAT files if and when it syncs flight records with the DJI servers, and so they are often missing from the MCDatFlightRecords folder.​
Litchi log files
On iOS devices, the Litchi CSV logs are in Litchi » Documents » flightlogs, and the DJI TXT logs are in Litchi » Documents » SDK_logs » FlightRecord.​
On Android devices the Litchi CSV logs are in LitchiApp » flightlogs. Litchi also creates a standard DJI TXT log, which for some reason it stores in a subdirectory in the DJI app directory, DJI » com.aryuthere.visionplus » FlightRecord.​
In the case of automated Litchi missions, it's also important to post a link to the flight mission profile.​
4. LOG FILE CONVERSION AND VISUALIZATION

There are various options to visualize and examine the contents of the log files, depending on type. Some of the commonly used ones are described below:​

DJI TXT logs
The PhantomHelp website will display a summary of the flight with some of the telemetry. You can also download the original TXT log and the full file contents in CSV format. It doesn't provide graphical analysis, but you can open the CSV file in Excel or dedicated data analysis programs.​
The AirData website will also read TXT logs, giving a different view of the data. It has a useful condensed view of app notifications, and a fairly advanced analysis of the computed wind field during the flight (subscription only). It doesn't provide a full downloadable version of the log - only a limited subset of the data. It also, by default, allows download of the original TXT log.​
The Windows version of CsvView will convert DJI TXT logs, and provides numerous powerful options to visualize the flight data.​
TXTlogToCSVtool is another offline Windows program that reads and converts TXT logs to CSV format. In fact the PhantomHelp website and CsvView both use this to perform their conversions. There is now a separate version for Mavic Mini logs. There are no visualization options - you simply get the full CSV version of the log that you can open in other data analysis programs.​
Aircraft DAT files
Phantom 4 and Mavic Pro DAT files can be converted to CSV format using CsvView or DatCon, which is a Java application that runs under OS X and Windows. These files are recorded at rates up to 200 Hz for some of the data, and contain hundreds of data fields. They are extremely data rich, but not for the faint of heart when it comes to trying to interpret them. @BudWalker is constantly updating the parsing engine, but a partial description of some of the more basic data fields is on his website.​
Mobile device DAT files (DJI GO 4 app)
These are converted to readable CSV files using CsvView or DatCon, and can be visualized in CsvView under OS X and Windows. They are recorded at just 10 Hz and are much smaller than the aircraft DAT files, but contain many of the same fields. AirData also reads mobile device DAT files and displays a similar set of data as it does with the TXT logs.​
Litchi log files
These logs are recorded as CSV files and so they don't need any conversion. PhantomHelp and AirData will display them, or they can be opened by Excel or data analysis software.​
5. LOG FORENSICS

Any attempt at a description of this subject is going to be very superficial - at this point I could write a short book on telemetry data analysis. Over the past few years I've developed a few streamlined approaches to looking at the data, depending on the type of event (crash, flyaway, loss of control, unexpected behavior etc.), but most of them involve significant data processing in the form of frame of reference transformations, quaternion algebra and some calculus. Those all require a significant data analysis package - I use Wavemetrics Igor Pro for which I've written a number of data processing functions. Igor Pro is particularly suited to this kind of work, and since user functions are compiled, rather than interpreted, it chews through huge datasets very fast. To give a few examples:​
  • In cases of flight control issues, I look for discrepancies between the IMU sensor fusion solutions for position and velocity, and check for appropriate correlations between the stick inputs, motor speeds (DAT files only), aircraft attitude, and horizontal velocities and accelerations. Example.
  • With flyaways (or generally blowaways) in P-GPS the most effective technique is to extrapolate the battery level and solve for the intersection with the autoland, and then extrapolate the aircraft position with drift velocity to determine where it should autoland. Here's a detailed description of that process in practice.
  • For power loss or FC failure leading to motor shutdown I use an aerodynamic aircraft model together with the computed wind field as a function of height to run a drag-dominated finite-difference numerical solution for the descent path of the aircraft, writing to a KML descent track file. Example.
That said, there are various simple checks that can be performed by looking at the telemetry. The data field names below are from the DJI TXT log, but there are equivalent fields in the DAT files. Some are text fields, while others are numerical.​
  1. OSD.flyTime – elapsed flight time;
  2. OSD.flycState – the flight control mode (P-GPS, ATTI, RTH, autoland etc.)
  3. SMART_BATTERY.battery – the battery level in percent;
  4. SMART_BATTERY.goHomeBattery – the computed battery percentage that triggers RTH, based on distance from the home point;
  5. SMART_BATTERY.landBattery – the computed battery percentage that triggers autolanding, based on height above the home point;
  6. OSD.pitch, OSD.roll, OSD.yaw – the aircraft attitude as pitch, roll and yaw (IMU sensor fusion yaw solution - not the compass heading);
  7. OSD.height – height above the home point;
  8. OSD.sWaveHeight – VPS height above the ground;
  9. OSD.gpsNum – number of GNSS satellites locked;
  10. OSD.gpsLevel – navigation health level (1 - 5), which is a measure of confidence in the sensor fusion solution;
  11. OSD.isGPSused – is GNSS used for positioning;
  12. OSD.isVisionUsed – is VPS (rather than GNSS) being used for positioning;
  13. RC.aileron, RC.elevator, RC.rudder, RC.throttle – the recorded stick inputs;
  14. OSD.latitude, OSD.longitude – the sensor fusion solution for position (not the raw GNSS position data, which is only in the DAT file);
  15. OSD.xSpeed, OSD.ySpeed, OSD.zSpeed – velocity north, east and down, respectively;
  16. MC_PARAM.failSafeAction – the failsafe action (RTH, hover or land).
That's just a partial list of some of the most useful data fields when it comes to trying to identify flight problems.​

In all but the most simple cases, I recommend posting to the forum a description of the flight together with both the DJI TXT log and mobile device DAT log. There are several forum members who have experience in interpreting these files by various methods. For those who might be interested, the best tutorials for understanding and replicating the analysis methods are probably the forensic threads on this forum, and any questions for further explanation are always encouraged.

Comments welcomed...

EDITS:

12/26/19: Added the full range of DJI aircraft with readable onboard DAT files.
12/26/19: Added the location of Litchi DJI SDK TXT log files.
12/27/19: Added explicit retrieval directions for DJI Fly.
02/10/20: Added that DJI Fly auto deletes DAT files.
 
Last edited:

BigAl07

Administrator
Staff Member
Premium Pilot
Joined
Jun 1, 2017
Messages
3,218
Reaction score
5,023
Age
49
Location
Western NC, USA
Great work bud. This is now a STICKY here :)
 

BigAl07

Administrator
Staff Member
Premium Pilot
Joined
Jun 1, 2017
Messages
3,218
Reaction score
5,023
Age
49
Location
Western NC, USA
They are threads that are always at the TOP of the forum. We "designate" a thread as a sticky and it stays at the top.
 

RayOZ

Well-Known Member
Joined
Nov 16, 2017
Messages
1,364
Reaction score
1,290
Location
Brisbane, Australia
Thanks for writing that. I guess the details flies over a lot people’s head.
I think the main take away is: if you lost your drone and want to know what happened or how to find it, please provide at least one of the the flight logs described above. A single paragraph of what you saw happened is of little use.
 
Last edited:

PeRo

Member
Joined
Oct 11, 2018
Messages
17
Reaction score
4
Age
57
Thanks to sar104 and all other contributors for for this comprehensive summary. Absolutly worth reading it - as all of their issued FlightLog analysis.

Question reg. the Litchi logs statement above:
"On Android devices the Litchi CSV logs are in LitchiApp » flightlogs. As mentioned previously, Litchi doesn't create a DJI TXT log under Android."

Flying MP1 for 2 years I used the AC dat file for flight path analysis. As this file became inavailable on MP2P due to encryption, I recognized that - besides the mentioned Litchi CSVlogs - a txt.file is created under Android as well. It is stored under "DJI / com.areyouthere.visionplus / FlightRecord / DJIFlightRecord_YYYY-MM-DD".txt. It can be viewed with CsvView and I consider the .txt content more reliable than .csv logs.

Any idea how Litchi does create the FlightRecords in DJI FileSystem? And why?
Litchi and Go4 are considered as independent apps without any connection to each other.
 

Camino Ken

I fugere solum ad utendum
Premium Pilot
Joined
Dec 12, 2019
Messages
246
Reaction score
313
Age
68
Location
Camino, CA
Major kudos to you, extremely well written and thorough.
 
  • Like
Reactions: sar104

sar104

Dic mihi solum facta, domina.
Premium Pilot
Joined
May 27, 2017
Messages
9,919
Reaction score
11,850
Location
Los Alamos, NM
Thanks to sar104 and all other contributors for for this comprehensive summary. Absolutly worth reading it - as all of their issued FlightLog analysis.

Question reg. the Litchi logs statement above:
"On Android devices the Litchi CSV logs are in LitchiApp » flightlogs. As mentioned previously, Litchi doesn't create a DJI TXT log under Android."

Flying MP1 for 2 years I used the AC dat file for flight path analysis. As this file became inavailable on MP2P due to encryption, I recognized that - besides the mentioned Litchi CSVlogs - a txt.file is created under Android as well. It is stored under "DJI / com.areyouthere.visionplus / FlightRecord / DJIFlightRecord_YYYY-MM-DD".txt. It can be viewed with CsvView and I consider the .txt content more reliable than .csv logs.

Any idea how Litchi does create the FlightRecords in DJI FileSystem? And why?
Litchi and Go4 are considered as independent apps without any connection to each other.
Thank you for pointing that out. I had briefly wondered if Litchi might store it somewhere in the DJI directory, but dismissed the idea because that would then require the DJI GO 4 app to be installed, and so I didn't look any further. But you are correct - that rather obscure directory contains the DJI TXT logs created by Litchi. I'll update the post to include that.
 

JAW

Did I Just Back Into That Tree?
Premium Pilot
Joined
Jan 4, 2019
Messages
789
Reaction score
528
Age
72
Location
Coastal NC Up to Long Island
@sar104

Not only do your detailed "post crash" analyses provide the unfortunate pilots with a clear understanding of exactly what happened, they also provide the rest of us with a narrative of lessons learned which hopefully prevent us from going down the same path. Now we have some idea of the numerous sources of information that must be examined then provided in a language the rest of us can understand.

Your efforts are much appreciated.

ADDED:
I would be remiss if I did not acknowledge the others who similarly provide insight into aircraft incidents. My thanks to them as well.
 
Last edited:

FoxhallGH

Well-Known Member
Premium Pilot
Joined
Oct 12, 2018
Messages
2,166
Reaction score
2,154
Location
UK - Suffolk
Thanks for the knowledge share sar104 ... I both appreciate & respect the way that you unselfishly share your knowledge, processing power and time on & with the Forum.
Et dormire faciet vobis hat domine mi ...
 

Antshill

Member
Joined
Dec 25, 2019
Messages
8
Reaction score
4
Location
CA
Fantastic post @sar104! Would everything that you posted regarding DJI GO4 apply to DJI Fly for the mini as well?
 
  • Like
Reactions: sar104

sar104

Dic mihi solum facta, domina.
Premium Pilot
Joined
May 27, 2017
Messages
9,919
Reaction score
11,850
Location
Los Alamos, NM
Fantastic post @sar104! Would everything that you posted regarding DJI GO4 apply to DJI Fly for the mini as well?
I'm not sure about the directory structure under the DJI Fly app since I don't have a Mini, but otherwise yes. But that raises a good point - since the motivation for the post was in large part the large number of posts asking for help with the Mini, I really need to include that explicitly. Perhaps someone with a Mini can confirm those details.
 

PhantomFandom

Well-Known Member
Joined
Jan 1, 2019
Messages
393
Reaction score
646
BRAVO !!!

Thank you so much for such an informative, accurate, and useful post. I fear that we will be seeing an exponential rise in MM "fly-away" and crash posts in the coming weeks and days.

I don't see DJI's logic in making the aircraft DAT log files inaccessible to us with the latest drones. The information in those files helps everyone learn. Since they can assure the on-board FC data cannot be manipulated, there really is no reason to hide it from the owners/pilots. The only reason anyone could come up with is that they don't want a third-party data analysis to conflict with their conclusion when a crash occurs. However even that logic is faulty because we still have the controller logs...up to a point.

Happy Holidays to you and thank you for all the insight you give to this forum!
 

Prismatic

Well-Known Member
Joined
Feb 26, 2018
Messages
403
Reaction score
396
Age
66
Location
Northern Colorado
... Et dormire faciet vobis hat domine mi ...
Latin! A tip o' the hat indeed!

You get some amusing translations of that on the interwebs. 😆

Google Translate is off in the weeds, misunderstanding the context: "And sleep will do you the hat, sir."

e-translate.com is courtly, but confused about who's who here: "The tip your hat, lord."

eprevodilac.com starts strong, then trails off non-committally: "The tip my hat to you, my ..."

Thanks for the entertainment!
 

sar104

Dic mihi solum facta, domina.
Premium Pilot
Joined
May 27, 2017
Messages
9,919
Reaction score
11,850
Location
Los Alamos, NM
BRAVO !!!

Thank you so much for such an informative, accurate, and useful post. I fear that we will be seeing an exponential rise in MM "fly-away" and crash posts in the coming weeks and days.

I don't see DJI's logic in making the aircraft DAT log files inaccessible to us with the latest drones. The information in those files helps everyone learn. Since they can assure the on-board FC data cannot be manipulated, there really is no reason to hide it from the owners/pilots. The only reason anyone could come up with is that they don't want a third-party data analysis to conflict with their conclusion when a crash occurs. However even that logic is faulty because we still have the controller logs...up to a point.

Happy Holidays to you and thank you for all the insight you give to this forum!
I'm not sure on the DAT file question. They have always been encrypted, but on the earlier models the keys were easily accessible. Whether that was an oversight and DJI never meant them to be readable isn't clear, at least to me.
 
  • Like
Reactions: mereflyer and car

RayOZ

Well-Known Member
Joined
Nov 16, 2017
Messages
1,364
Reaction score
1,290
Location
Brisbane, Australia
I'm not sure about the directory structure under the DJI Fly app since I don't have a Mini, but otherwise yes. But that raises a good point - since the motivation for the post was in large part the large number of posts asking for help with the Mini, I really need to include that explicitly. Perhaps someone with a Mini can confirm those details.
For all the good work you’ve done for us here, and helping so many recover their lost drones, someone should’ve got you an MM for Christmas. ;)