DJI Mavic 2 Pro - Positive Feedback Loop Crash

[BeefStew43]

I was flying with a friend testing active track 2.0 on the Mavic 2 Pro. He was driving his car and I was tracking in "trace" and everything was going well. He stopped and turned around on a street and the Mavic 2 tracked as expected. Shortly after the turn around, I started to pan the drone off to the left while still following in active track. At one point the car was slightly hidden by a row of trees and the Mavic 2 lost the car and started to track the set of trees as the car continued down the road. I then took over (still in active track but not currently tracking a subject) the controls and started chasing after the car in P-Mode. The drone started acting up and slightly tilting to the right. The horizon became off centered as the drone continued to tilt to the right. The gimbal then started acting funny and even went around to show the front landing gear on the right. I tried to adjust and go against the faults but nothing helped. The drone ended up crashing to the right and breaking both front arms. Not sure if the gimbal is ok. I put the battery in and it still starts up fine.

Just wondering what could cause this issue and how to avoid this in the future. Thanks for any help or suggestions.

DJI Flight Log Viewer | Phantom Help

Upload and view flight logs from your DJI drones

www.phantomhelp.com

Possible YouTube explanation:

 

[slup]

Straight out from the TXT logs comes no cause to this ... it only confirm your story. If you could recover & post the DAT log ending with FLY006.DAT from your phone... that perhaps can give more in depth info to pinpoint a possible cause. It smells HW failure though ...

But honestly ... don't think you need much more then this to approach DJI.

You were out of trace mode & in GPS mode when this occurred. The winds were manageable & averaging around 5,7m/s with gust up to 8,6m/s. From 438sec you started to have constant notEnoughForce errors that remained until the log ended, from approx. 449sec you start to get out of limit errors from the gimbal ... then located on 49m height. Later on all 3 gimbal axis eventually goes out of limit.

The AC mainly did what's commanded until 414sec when the AC starts to yaw & you trying to counteract, then it flies away rotating mainly uncommanded until it stops around 429sec. The AC then uncommanded flies back sideways while you command full descend & full elevator (forward). Then you mainly tries to get it back to ground.

Stick inputs:



With corresponding AC movements:



Error messages flight time related & plain text messages from the log:

 

[BeefStew43]

Attached is the DAT log file.

It looks like in the YouTube video I included that if the gimbal yaw and the aircraft yaw get out of sync (through a problem in the software or hardware), and a gust of wind comes along, it can enter into a positive feedback loop that results in the same symptoms I experienced before the crash.

The video shows the gimbal and OSD yaw plotted on a graph, and they track closely together until instantly the lines split to opposite sides of the graph; which is when his AC went out of control. It looks like my gimbal yaw and OSD yaw also went out synch at the same point the AC went out control, and appears to produce the same exact graph shown in the video if I were to plot the points on a graph. The yaw went from 179.6 to -179.6 instantly; while the other yaw stayed consistent at 156.5. I was comparing the values in the verbose csv extract from phantom viewer, and it looks like from those 2 yaw values alone it probably entered into the exact same positive feedback loop due to wind as shown in the YT video (skip to 7-10 minutes to see the explanation and data). I was hoping to confirm if that was the case, or there was some other explanation as to what caused the crash.

 

[new2mavic]

There is a lot of uncommanded yaw near the end of the flight. I believe there is some problem with the IMU giving the flight controller false information about the attitude of the craft hence the abnormal behaviour.

I don't quite buy the theory put forward by the guy in the youtube video. The craft does not need to know the direction the nose is pointing in deciding whether it should tilt right or left to hold it's position. Imagine that you are blindfolded and standing in a moving bus so you do not know which direction you are facing but you can still feel the motion of the bus and correctly shift your body around to maintain balance.

 

[Prismatic]

... I don't quite buy the theory put forward by the guy in the youtube video. The craft does not need to know the direction the nose is pointing in deciding whether it should tilt right or left to hold it's position. Imagine that you are blindfolded and standing in a moving bus so you do not know which direction you are facing but you can still feel the motion of the bus and correctly shift your body around to maintain balance.

This has been discussed at length. The aircraft most definitely does need to know which direction it's facing to hold its position. Your bus analogy had several flaws.

Here's a better one: you're on a boat at night in dense fog. You have GPS, so you know where you are, but you have no compass so you have no idea of your heading. There are rocks in every direction but south, with strong currents that vary dramatically with the tide. But you lost your tide tables, so you don't know which direction they're running. The GPS shows you are dangerously close to some rocks and tracking nearer, but without a compass, you don't know if they're ahead, astern, port or starboard.

What's your plan?

 

[slup]

Attached is the DAT log file.

It looks like in the YouTube video I included that if the gimbal yaw and the aircraft yaw get out of sync (through a problem in the software or hardware), and a gust of wind comes along, it can enter into a positive feedback loop that results in the same symptoms I experienced before the crash.

The video shows the gimbal and OSD yaw plotted on a graph, and they track closely together until instantly the lines split to opposite sides of the graph; which is when his AC went out of control. It looks like my gimbal yaw and OSD yaw also went out synch at the same point the AC went out control, and appears to produce the same exact graph shown in the video if I were to plot the points on a graph. The yaw went from 179.6 to -179.6 instantly; while the other yaw stayed consistent at 156.5. I was comparing the values in the verbose csv extract from phantom viewer, and it looks like from those 2 yaw values alone it probably entered into the exact same positive feedback loop due to wind as shown in the YT video (skip to 7-10 minutes to see the explanation and data). I was hoping to confirm if that was the case, or there was some other explanation as to what caused the crash.

Going through the DAT file reviles an under-performing left front prop ...both the RPM's & the motor command becomes very erratic just after the trace mode have been cancelled & you just have past a row of trees with full elevator & throttle. Together with this the yaw, tilt & roll nudge at that point. My bet to this is either a bird strike coming from the trees just past over or some other failure reason of the left front motor. All gimbal errors should then be secondary problems coming from a very unstable flight performance.



Below you see the unstable roll & pitch after that point.



The prop rpm's from here together with command percentage ... note that towards the end of the flight that motor is commanded to 90% but only produce medium rpm's.

 

[new2mavic]

This has been discussed at length. The aircraft most definitely does need to know which direction it's facing to hold its position. Your bus analogy had several flaws.

Here's a better one: you're on a boat at night in dense fog. You have GPS, so you know where you are, but you have no compass so you have no idea of your heading. There are rocks in every direction but south, with strong currents that vary dramatically with the tide. But you lost your tide tables, so you don't know which direction they're running. The GPS shows you are dangerously close to some rocks and tracking nearer, but without a compass, you don't know if they're ahead, astern, port or starboard.

What's your plan?

You are right, I mixed up holding attitude and holding position.

 

[sar104]

Going through the DAT file reviles an under-performing left front prop ...both the RPM's & the motor command becomes very erratic just after the trace mode have been cancelled & you just have past a row of trees with full elevator & throttle. Together with this the yaw, tilt & roll nudge at that point. My bet to this is either a bird strike coming from the trees just past over or some other failure reason of the left front motor. All gimbal errors should then be secondary problems coming from a very unstable flight performance.

View attachment 95039

Below you see the unstable roll & pitch after that point.

View attachment 95041

The prop rpm's from here together with command percentage ... note that towards the end of the flight that motor is commanded to 90% but only produce medium rpm's.

View attachment 95040

I think that you have misdiagnosed the problem - there is nothing wrong with the motors. The motor speed / PWM ratio is good. The problem is with the IMU - the IMU and compass yaw on both IMUs goes bad at 374 seconds. Note that IMU1 was the active IMU.



And note that the integrated z-axis gyro data from both IMUs also disagree with the yaw and magnetic yaws and with each other - very unusual. Looking at the rate gyro raw data makes the problem more obvious:





Since it seems improbable that both IMUs would fail identically, that suggests to me that it was a processor failure, rather than sensor failure.

 

[DanMan32]

This has been discussed at length. The aircraft most definitely does need to know which direction it's facing to hold its position. Your bus analogy had several flaws.

Here's a better one: you're on a boat at night in dense fog. You have GPS, so you know where you are, but you have no compass so you have no idea of your heading. There are rocks in every direction but south, with strong currents that vary dramatically with the tide. But you lost your tide tables, so you don't know which direction they're running. The GPS shows you are dangerously close to some rocks and tracking nearer, but without a compass, you don't know if they're ahead, astern, port or starboard.

What's your plan?

Start paddling in one direction and compare to the results in GPS.

You'd need a circular raft to equate to what you're trying to convey. Otherwise you would know which way is forward.

Most of the time FC would know which is forward unless the gyro/accelerometer is really messed up.
But that's for a different discussion. In this case the gyro may have gotten out of whack.

 

[sar104]

Start paddling in one direction and compare to the results in GPS.

You'd need a circular raft to equate to what you're trying to convey. Otherwise you would know which way is forward.

Most of the time FC would know which is forward unless the gyro/accelerometer is really messed up.
But that's for a different discussion. In this case the gyro may have gotten out of whack.

As I've pointed out to you many times - that's trial and error navigation. You cannot build a flight control system on trial and error data.

 

[Prismatic]

As I've pointed out to you many times - that's trial and error navigation. You cannot build a flight control system on trial and error data.

Exactly. You probably hit the rocks before you get it figured out.

 

[Deleted member 94047]

@sar104, a question if you can entertain one. This is the latitude, longitude plot from the txt log

This is the same plot from IMU(1) in the DAT log

And this is IMU(0) from the same DAT log


The txt log's record corresponds with the one from IMU(0) in the DAT, doesn't that imply IMU(0) was the active IMU?

 

[Prismatic]

Start paddling in one direction and compare to the results in GPS.

You'd need a circular raft to equate to what you're trying to convey. Otherwise you would know which way is forward. ...

The aircraft always knows which way is forward.

 

[slup]

I think that you have misdiagnosed the problem - there is nothing wrong with the motors. The motor speed / PWM ratio is good. The problem is with the IMU - the IMU and compass yaw on both IMUs goes bad at 374 seconds. Note that IMU1 was the active IMU.

View attachment 95043

And note that the integrated z-axis gyro data from both IMUs also disagree with the yaw and magnetic yaws and with each other - very unusual. Looking at the rate gyro raw data makes the problem more obvious:

View attachment 95044

View attachment 95045

Since it seems improbable that both IMUs would fail identically, that suggests to me that it was a processor failure, rather than sensor failure.

Yeah, of course ... I'm still to locked in to that compass yaw error only occur in connection to the start moment & shortly after ... one more brick for me to put on the pile of knowledge

But the left front to me still looks odd ... especially towards the end on 78-80 meters height. Can it be secondary to the IMU failure ... or I'm I just seeing something that doesn't exist & the sudden erratic behavior from engines from the event point is similar & connected to an AC "paddling in all directions" to find out the heading (to use other's analogy to a boat in fog ) ?

 

[sar104]

@sar104, a question if you can entertain one. This is the latitude, longitude plot from the txt log
View attachment 95060
This is the same plot from IMU(1) in the DAT log
View attachment 95061
And this is IMU(0) from the same DAT log
View attachment 95062

The txt log's record corresponds with the one from IMU(0) in the DAT, doesn't that imply IMU(0) was the active IMU?

You are being mislead by the lack of position data early in the DAT file:



The position data are all the same in the DAT file - GNSS position.

Looking at the yaw values tells the real story:



IMU1 was active.

 

[sar104]

Yeah, of course ... I'm still to locked in to that compass yaw error only occur in connection to the start moment & shortly after ... one more brick for me to put on the pile of knowledge

But the left front to me still looks odd ... especially towards the end on 78-80 meters height. Can it be secondary to the IMU failure ... or I'm I just seeing something that doesn't exist & the sudden erratic behavior from engines from the event point is similar & connected to an AC "paddling in all directions" to find out the heading (to use other's analogy to a boat in fog ) ?

The front left struggles a bit at the end, having been running very high for some time, but that's long after the problems start. But that looks like a result of the flight control issues, not the cause of it.

 

[sar104]

Also note, if you are looking at the DAT data, that offsetTime is incorrectly scaled for this aircraft. The flight time was 452 seconds, as indicated by the txt log, not 385 seconds as shown in the DAT.

 

[Deleted member 94047]

Thanks sar, also found these in the event stream log after I posted.

-18.518 : 1052 [L-FDI]ns req:fdi,0to1,reason:normal,result:succeed
-18.518 : 1052 [L-IMU]

state: drift
-17.698 : 1100 [L-FDI]first switch when startup
-17.647 : 1103 [L-IMU]

state: fixed

47.667 : 4930 [L-FDI]NS(0) GPS(0): fault on , conformity

227.720 : 15480 [L-FDI]NS(0) GPS(0): fault on , height_drift
230.280 : 15630 [L-FDI]NS(0) GPS(0): fault off, height_drift

313.907 : 20530 [L-FDI]NS(0) GPS(0): fault on , height_drift

330.120 : 21480 [L-FDI]NS(0) GPS(0): fault off, height_drift

332.305 : 21608 [L-FDI]NS(0) FUSION(1): fault on , magn_heading_err_large

342.016 : 22177 [L-FDI]NS(0) FUSION(1): fault off, magn_heading_err_large
342.135 : 22184 [L-FDI][CTRL]: fault on , height_ctrl_fail
342.152 : 22185 [L-RC]craft ctrl failed!!!
​

The entry at T-18.518 seems to indicate a switch from IMU(0) to (1) and FUSION errors are reported later on IMU(1). But note that there are also GPS errors as early as T+47s. Could the GPS errors be the root cause here?​

 

[DanMan32]

The aircraft always knows which way is forward.

If that were true, Sar wouldn't be arguing against my point.

 

[DanMan32]

As I've pointed out to you many times - that's trial and error navigation. You cannot build a flight control system on trial and error data.

Isn't all FC control, even recalibration mid flight (yaw correction) "trial and error"?
I realize my proposals are oversimplified, but I don't think it is as impossible as you make it out to be either.