CsvView support for Fly App .txt log files

[BudWalker]

CsvView has been unable to accept .txt log files created by FlyApp version 1.2.2 and later. CsvView 4.2.2 can accept the .csv file created by submitting a .txt to either the PhantomHelp Flight Reader (DJI drone flight log viewer | Flight Reader) or the PhantomHelp free online viewer( DJI Flight Log Viewer | Phantom Help). This includes the encrypted .txt log files created by the FlyApp version 1.2.2 and higher.

CsvView 4.2.2 can be obtained here
CsvView Downloads

With the Flight Reader first load the .txt and then select the Aircraft CSV option.

This will open the .csv spread sheet. Then save the .csv to a location that can be submitted to CsvView.

With the PhantomHelp free online viewer first upload the .txt log file then select Download CSV option.

Then submit the downloaded .csv to CsvView.
@jclarkw , @slup

 

[Macinfo]

I feel like I am missing something, a step? You note that since Fly 1.2.2 an unknown version of CSV View did not work, but CSV View 4.2.2 will take files from newer versions of Fly and allow them to be submitted for more analysis? Or did I miss the point completely?

 

[BudWalker]

I feel like I am missing something, a step? You note that since Fly 1.2.2 an unknown version of CSV View did not work, but CSV View 4.2.2 will take files from newer versions of Fly and allow them to be submitted for more analysis? Or did I miss the point completely?

CsvView versions 4.2.1 and before are unable to process the .txt created by the Fly App versions 1.2.2 and later. The problem is that FlyApp 1.2.2 started encrypting the .txt and CsvView can't decrypt it.

CsvView 4.2.2 still can't decrypt those encrypted .txt log files. Instead, it relies on the PhantomHelp Flight Reader (DJI drone flight log viewer | Flight Reader) or the PhantomHelp free online viewer( DJI Flight Log Viewer | Phantom Help) to do the decryption.

 

[slup]

Installed CsvView+DatCon 4.2.2, downloaded a log from a MA2 from the free PhantomHelp (.csv)... here --> My new Mavic is at the bottom of a lake!, unpacked the rar archive & tried to put it into CsvView ... but only get a "internal error".

@BudWalker ... any hints, should the .csv be placed somewhere to get it to work?

 

[BudWalker]

Installed CsvView+DatCon 4.2.2, downloaded a log from a MA2 from the free PhantomHelp (.csv)... here --> My new Mavic is at the bottom of a lake!, unpacked the rar archive & tried to put it into CsvView ... but only get a "internal error".

@BudWalker ... any hints, should the .csv be placed somewhere to get it to work?

Works for me. Can you provide the .csv that you used? And, provide the CsvView log file, i.e. [HomeDir]/.CsvViewLog

EDIT:
Just to be clear the .txt has to be processed through PhantomHelp to obtain the .csv that it creates. That .csv is then submitted to CsvView

 

[BudWalker]

The bug in version 4.2.2 that @slup reported is being worked on. Until it's fixed I've removed CsvView 4.2.2 from the download site.

If you're interested in testing a fixed version please PM me.

 

[BudWalker]

CsvView Version 4.2.3 is available at
CsvView Downloads

It fixes the bug in version 4.2.2 that @slup reported. Specifically, CsvView 4.2.2 was OK with a .csv created by the PhantomHelp offline Flight Reader but would choke on a .csv created by the PhantomHelp online viewer. Version 4.2.3 works with both the online viewer and the offline Flight Reader.

 

[msinger]

@BudWalker does the order of the columns in the CSV generated by my log viewers matter? I'm curious since I will likely modify the output of my online log viewer at some point and the columns are customizable in Flight Reader (they can be added/removed/reordered).

 

[BudWalker]

@BudWalker does the order of the columns in the CSV generated by my log viewers matter? I'm curious since I will likely modify the output of my online log viewer at some point and the columns are customizable in Flight Reader (they can be added/removed/reordered).

No, order doesn't matter.

 

[Andris]

I created a csv file with the djiparsetxt program (Documenting the Format of DJI Log Files)

It is readable in any text editor program but csvView cannot parse it for some reason.

I checked the file and the first line is missing the sep=, line. However after I add the line it still cannot parse it.

I attach a problematic file and a good file.

Thank you for your help!

 

[BudWalker]

I created a csv file with the djiparsetxt program (Documenting the Format of DJI Log Files)

It is readable in any text editor program but csvView cannot parse it for some reason.

I checked the file and the first line is missing the sep=, line. However after I add the line it still cannot parse it.

I attach a problematic file and a good file.

Thank you for your help!

The name CsvView is a bit misleading. CsvView is not a general .csv viewer. It only knows about .csvs created by DatCon or the PhantomHelp reader.

 

[Andris]

So it looks for specific header columns? (As far as I can see in the provided files the header column names are almost matching.)

 

[BudWalker]

So it looks for specific header columns? (As far as I can see in the provided files the header column names are almost matching.)

If CsvView sees the header

OSD.flyTime [s]

it knows to load the column definitions for the Phantom Help reader. You could modify your .csv and rename OSD.flyTime to

OSD.flyTime [s]

The column definitions are found in

[installDirectory]/defs/phantomHelpSignals.xml

 

[Andris]

I figured out that if I export a csv from datcon with the option "Show units in column headings" unticked, I can see lots of extra fields because the definitions are without units. Just a tip if someone sees this topic.

 

[BudWalker]

I figured out that if I export a csv from datcon with the option "Show units in column headings" unticked, I can see lots of extra fields because the definitions are without units. Just a tip if someone sees this topic.

Hmmm. You're right. I believe you are the first to notice this since it was implemented 2 or 3 years ago.

Is there a reason you don't just use CsvView which runs DatCon internally? There shouldn't be any missing fields. Be sure and select the Signal Groups you want though.

 

[Andris]

Thanks for your attention.

Acually I tried to use csvview to process the dat file but it says "Unrecognized log file /Users/Andris/dji-tools/fly.dat"

So first I run datcon to get a csv and event log separately and then I feed the csv to csvview. This was I cannot open the eventlogplayer unfortunately. I have the latest version of datcon and csvview. If you are interested I can send you the dat file to investigate, if you think it is a bug.

I get the DAT file from ExtractDJI-ing a fam_DJI_ASSISTANT_EXPORT_FILE_2022-06-01_13-22-09.DAT. Then I get a folder flyctrl where there is a FLY015.DAT. I decrypt it and now it can be opened with datcon but not csvview.

 

[BudWalker]

Thanks for your attention.

Acually I tried to use csvview to process the dat file but it says "Unrecognized log file /Users/Andris/dji-tools/fly.dat"

So first I run datcon to get a csv and event log separately and then I feed the csv to csvview. This was I cannot open the eventlogplayer unfortunately. I have the latest version of datcon and csvview. If you are interested I can send you the dat file to investigate, if you think it is a bug.

I get the DAT file from ExtractDJI-ing a fam_DJI_ASSISTANT_EXPORT_FILE_2022-06-01_13-22-09.DAT. Then I get a folder flyctrl where there is a FLY015.DAT. I decrypt it and now it can be opened with datcon but not csvview.

It's not a bug - it's a feature. CsvView doesn't like the .dat extension. Change it to .DAT. That's capital D-A-T.

In addition to the .DAT that CsvView can process there are several .dat files which CsvView can't process. It seems that no matter how many times users are told that these .dats can't be processed they will retrieve them then complain that CsvView can't process them. Having CsvView reject these .dats seemed to help.

Ultimately, it's DatCon that can't process these .dats and so why you may ask does DatCon accept a .dat. DatCon users are generally more tech savvy and didn't seem fall into this problem.

 

[Andris]

@BudWalker Thank you for clarification! Now it works as intended!

Do you know any info about opening the Mavic Mini 2 DAT files?

 

[BudWalker]

@BudWalker Thank you for clarification! Now it works as intended!

Do you know any info about opening the Mavic Mini 2 DAT files?

The MM2 .DATs found on the mobile app are encrypted and can not be processed by DatCon.

 

[Andris]

The header seems to be the same but maybe inside it is scrambled then.