Another Flyaway

[Oliver_I]

Wow. At several points it had alerts that there was a GPS connection problem but on the left it never showed GPS was not connected. You also had a warning that an engine was in distress, being obstructed, but only once, and not at the end when you encountered the tree, unless you may have bumped into it more than once. At 72 feet I would not think you could encounter a tree but I don't know, you might have trees that tall. Did you see where you were on the map before you took off? Is this a regular place you fly from or did you travel to get to this launch place?

The App said GPS connected, I'm pretty sure about that. The engine distress message was probably from the first tree hit. It hit several branches of the same tree... didn't sound good. :-/
It's a regular place I fly, but it wasn't the same place as the flight before the accident (but not far away). The place on the map looked correct when I started. At least not totally off.

 

[MavicCF]

Heck, I had a whole 18MB downloaded!

 

[Oliver_I]

Thanks for that, we have unlimite, but it's still a major pain. Let me know if you figure out how to open it.

The file was created with DJI assistant 1.2.3... can I do anything to make it readable for you?

 

[sar104]

The file was created with DJI assistant 1.2.3... can I do anything to make it readable for you?

I'm not sure. We might need @BudWalker to take a look. I can extract FLY060.DAT from the DJI Assistant file, but then it just reports "not a DAT file" when I try to load it. I'll try a couple more things.

 

[sar104]

Was it particularly windy?

 

[Oliver_I]

Was it particularly windy?

No, not at all.

 

[BudWalker]

The file was created with DJI assistant 1.2.3... can I do anything to make it readable for you?

@sar104 , @MavicCF
Several Mavic Air pilots have been unable to retrieve the .DAT from the AC. The .DAT from the tablet may have enough data to analyze this incident. Look here to see how to retrieve the tablet .DAT. It will be in the FlightRecord/MCDatFlightRecords folder.

I've looked some at this .DAT and others from the Mavic Air. They all appear to be encrypted or, possibly, a new record structure. I think they are encrypted. I don't have any immediate plans to look into this so we are stuck with the tablet .DAT file.

 

[Oliver_I]

OK, thanks! Here's the dat file from my iPhone.

 

[MavicCF]

Hey cool! That one took 30 seconds to download!

 

[Oliver_I]

Hey cool! That one took 30 seconds to download!

;-)

 

[Prismatic]

@sar104 , @MavicCF
Several Mavic Air pilots have been unable to retrieve the .DAT from the AC. The .DAT from the tablet may have enough data to analyze this incident. Look here to see how to retrieve the tablet .DAT. It will be in the FlightRecord/MCDatFlightRecords folder.

I've looked some at this .DAT and others from the Mavic Air. They all appear to be encrypted or, possibly, a new record structure. I think they are encrypted. I don't have any immediate plans to look into this so we are stuck with the tablet .DAT file.

This is, of course, very distressing. Encrypted log files suggest that DJI wants to make independent analysis impossible. That would be bad for everyone!
I’d wondered, because I had no luck at all trying to use CsvView with the log files pulled from my Mavic Air.

 

[MavicCF]

Was it particularly windy?

So what exactly is the difference between the magnetic yaw value and the regular yaw? I assume that the magnetic yaw is yaw in relation to true north whereas the regular yaw value is yaw in relation to the take off heading? Is that correct?

 

[BudWalker]

So what exactly is the difference between the magnetic yaw value and the regular yaw? I assume that the magnetic yaw is yaw in relation to true north whereas the regular yaw value is yaw in relation to the take off heading? Is that correct?

You can always look here to see what the various signals are.
V3 .CSV column descriptions

magYaw is computed by DatCon, it doesn't come directly from the AC. magYaw is heading that's derived from the magnetometers and then corrected with roll and pitch. It's mostly dependent on the magnetometers. In contrast, Yaw is the FC value for heading that's derived from the IMU (gyros and accelerometers) with very little input from the magnetometers. magYaw is useful for those incidents where Yaw could be compromised.

totalGyro:Z is also a DatCon computed signal that can be used to check Yaw.

 

[MavicCF]

You can always look here to see what the various signals are.
V3 .CSV column descriptions

magYaw is computed by DatCon, it doesn't come directly from the AC. magYaw is heading that's derived from the magnetometers and then corrected with roll and pitch. It's mostly dependent on the magnetometers. In contrast, Yaw is the FC value for heading that's derived from the IMU (gyros and accelerometers) with very little input from the magnetometers. magYaw is useful for those incidents where Yaw could be compromised.

totalGyro:Z is also a DatCon computed signal that can be used to check Yaw.

Thanks, added that page to my favorites.

 

[bytebiz]

This is, of course, very distressing. Encrypted log files suggest that DJI wants to make independent analysis impossible. That would be bad for everyone!
I’d wondered, because I had no luck at all trying to use CsvView with the log files pulled from my Mavic Air.

That would definitely be bad... Perhaps DJI's concern is file size (which would make sense since the files reside on the drone itself). In that case they may be 'compressing' them to save space rather than 'encrypting' them. In order to test out that theory you can try changing the .DAT extension to .ZIP (or another compression format like .TAR) after you have downloaded the file and then see if you can open it with the de-compression utility associated with that file extension and compression format.

 

[sar104]

This is the second time in a few days that we have seen DATs and TXTs that disagree substantially on basic parameters. In this case, we can get IMU yaw from both the DAT and the TXT for comparison - I would expect them to be identical. We can also compare with the magnetic yaw value from the DAT. In this case the comparisons are not good:



There are a couple of obvious problems here. Firstly there is the DAT yaw, which is off from the TXT yaw by +147°. It turns out that's because the DAT yaw is calculated in DatCon using IMU0. Computing yaw directly from the IMU_ATTI_0 quaternions reveals that the FC is using IMU1, not IMU0. That gives agreement between the DAT and the TXT.



But the other problem is that it appears that the IMU yaw was not initialized to the magnetic yaw data from either compass. The two compasses agreed very closely throughout the flight, but disagree with the IMU yaw by -144°. That said - I'm unable to replicate the DatCon magnetic yaw calculation from the magnetometer data - I calculate an initial heading of 30° which still disagrees with IMU1.

In any case, while the data are resisting vigorously, it is clear that there was a magnetic heading error at around 7 s and also the GPS data were very poor. However, with the data being somewhat contradictory it's hard to pinpoint exactly what happened.

 

[Prismatic]

That would definitely be bad... Perhaps DJI's concern is file size (which would make sense since the files reside on the drone itself). In that case they may be 'compressing' them to save space rather than 'encrypting' them. In order to test out that theory you can try changing the .DAT extension to .ZIP (or another compression format like .TAR) after you have downloaded the file and then see if you can open it with the de-compression utility associated with that file extension and compression format.

Seems like this is a parallel thread; sorry everyone!

While possible, it would take very specialized software to compress the data stream on-the-fly, and the resulting format wouldn't likely be familiar. As I say, possible but difficult and probably not open-source.

In my case, ExtractDJI works fine on the full DAT from the aircraft; it builds a lengthy list of DAT files, one for each flight. But those DAT files are unintelligible by CsvView, which also stalls if I try it on "Flight 1" of the full-size DAT. So the gross structure of the DAT is still visible to ExtractDJI (it can extract individual flight DATs), but the flight data within a DAT is unintelligible to CsvView. That speaks of encryption more than compression to me, though I freely admit that's some hand-waving when I know nothing about the magic of either tool. I do know enough to say encryption on-the-fly would be easier than compression on-the-fly (again, both are possible).

Still, decompression may be worth a shot. But I'm of little use there. I only have WinZIP, which I can try ... for a sample of one.

 

[sar104]

OK - after some consultation, the DatCon magnetic yaw calculation is correct - I had reversed the frames of reference. So, the aircraft was actually pointing NW on takeoff. There was no magnetic interference at the takeoff location, as evidenced by the lack of change in magnetic yaw as it ascended, but there must have been some significant magnetic interference at the location where the aircraft was first powered up, prior to the RC and app connecting to it.

That also makes sense of the velocity and pitch/roll data. Recomputing the forward and lateral velocities with the magnetic yaw data shows that the aircraft moved appropriately in response to pitch and roll, as expected. The reason for the uncontrolled nature of that motion was the discrepancy between the magnetic yaw and the IMU1 yaw that was being used by the FC, so that the FC's attempts at course correction, when it did not move in the expected direction, did not stabilize position but instead led to acceleration in the wrong direction (east).



This is most likely another example of needing to check that the initialized heading is consistent with the actual orientation of the aircraft, but it was also exacerbated by taking off with very poor GPS health which was probably what caused the FC to try to make corrections in the first place.

 

[sar104]

Seems like this is a parallel thread; sorry everyone!

While possible, it would take very specialized software to compress the data stream on-the-fly, and the resulting format wouldn't likely be familiar. As I say, possible but difficult and probably not open-source.

In my case, ExtractDJI works fine on the full DAT from the aircraft; it builds a lengthy list of DAT files, one for each flight. But those DAT files are unintelligible by CsvView, which also stalls if I try it on "Flight 1" of the full-size DAT. So the gross structure of the DAT is still visible to ExtractDJI (it can extract individual flight DATs), but the flight data within a DAT is unintelligible to CsvView. That speaks of encryption more than compression to me, though I freely admit that's some hand-waving when I know nothing about the magic of either tool. I do know enough to say encryption on-the-fly would be easier than compression on-the-fly (again, both are possible).

Still, decompression may be worth a shot. But I'm of little use there. I only have WinZIP, which I can try ... for a sample of one.

That was my conclusion too - if ExtractDJI works on the file then it seems unlikely that it could be compressed. Of course it's possible that the individual DATs are compressed before being combined by DJI Assistant, but that also seems unlikely.

 

[Oliver_I]

OK - after some consultation, the DatCon magnetic yaw calculation is correct - I had reversed the frames of reference. So, the aircraft was actually pointing NW on takeoff. There was no magnetic interference at the takeoff location, as evidenced by the lack of change in magnetic yaw as it ascended, but there must have been some significant magnetic interference at the location where the aircraft was first powered up, prior to the RC and app connecting to it.

That also makes sense of the velocity and pitch/roll data. Recomputing the forward and lateral velocities with the magnetic yaw data shows that the aircraft moved appropriately in response to pitch and roll, as expected. The reason for the uncontrolled nature of that motion was the discrepancy between the magnetic yaw and the IMU1 yaw that was being used by the FC, so that the FC's attempts at course correction, when it did not move in the expected direction, did not stabilize position but instead led to acceleration in the wrong direction (east).

View attachment 34763

This is most likely another example of needing to check that the initialized heading is consistent with the actual orientation of the aircraft, but it was also exacerbated by taking off with very poor GPS health which was probably what caused the FC to try to make corrections in the first place.

View attachment 34766

Thank you very much for your detailed analysis! Fantastic!

The yaw part makes sense to me. I usually point the mavic away from me during start up, so that would have been NW. Sometimes also towards me (SE), so when I saw SE on the map I thought that's correct. Might have been wrong though.
What I don't understand is: If the direction is not correct, why doesn't the mavic just turn around instead of speeding in eastern direction?

Also I turned it on right on the spot where I started, I don't see any source for magnetic interference there. That's a bit strange.
And the GPS thing... the "health" is not shown in the App, or is it? I just saw P-GPS mode and 9 or 10 sats locked and thought that's fine.

Thank you again!
Oliver