Mavic 2 Enterprise Advanced Thermal Map in Pix4D

[jaja6009]

I finally had time to go out and obtain the images to make a thermal map with the Enterprise Advanced.

Pix4D wanted 90 front and 90 side overlap per the directions and I found this to make the aircraft fly at only 2-3 mph. I do think that DJI Pilot does not have the correct information about the camera settings for mapping as it could of flew much lower and faster as there was such a long period between images.

When processing in Pix4D the software recommended using the settings for a FLIR Tau 2 sensor with 640 resolution.
When I tried using the Mavic Enterprise Advanced 620x512 setting the images would not stitch. I am going to try to make my own camera setting using the settings from the FLIR, but tweaking them a bit.

Overall I am very happy with the output though as when I compare it to regular images, it stitched this low sloped roof extremely well.
I chose this building as it is known to have leaks all over and I know where they are. It is basically a perfect place to train on qualitative thermography dealing with low slope roof moisture in the insulation.

Roof Specs:
Built-up Roof Covering (Not Modified Bitumen as far as I can tell due to the spacing between layers)
Unknown insulation under the covering
Corrugated metal decking for sheathing

Today's environmental factors made it an okay day for this.
Temperature got over 45* F
The sun was out and thermally loaded the roof
The wind was 10mph, but had gusting over 20mph at times.



Orthomap of thermal images.
Take note that until DJI releases and SDK that Pix4D can use, you can only make a thermal map without the radiometric data embedded in it. The above map is more like an RGB ortho.

Compare the above with this image of the building and you can see that the stitching was very good.


For those who want to try on solar panels, keep in mind that due to the angle of the panels you must get the drone to line up the same way or you will run into problems. Raptor Maps explains this well in their article on how to use this new drone for solar mapping.

I did not have this problem as my target has only the slightest of an angle.

 

[Faquaral]

You got some very nice results there!
We have tried to process in Pix4D too but with no success, it just won't stitch
Would you be ok to share your tweaked camera profile settings?

Thanks!

 

[jaja6009]

First make sure you have very high overlap. I used 90 and 90, but will experiment with lower in the future.
Next do NOT choose the Mavic 640x512 camera option. Use the Flir Tau 2 option.
Do not choose to make a thermal map, but choose to make an RGB map.
It should work from here. When done processing, go back to the Images Properties Editor under Process and then go to Selected Camera model. Choose Edit, then Load Optimized Parameters. You can save for the future or not save. You can also save as your default, but I did not. Now go back and Reoptimize (Quicker) or run Step 1 all over again.
Your Camera Optimization on your Quality Report should now be under 5%.

Keep in mind, this is NOT a radiometric thermal map, it is an RGB processed map with no temperature values in it.
Also keep in mind, due to this thermal camera not allowing you to lock in a range, the same temperature assigned to a color will vary. Since the temperature range is not locked for the flight and as the scenery and temperatures in regards to highest and lowest in FOV change, 85 degrees in one image may be bright white and in another it may be several shades darker.

Let me know if you need help, I can PM you.

 

[Faquaral]

Many thanks for your reply!
I'm getting somewhere with the stitching now in rbg map, though with the help of a few manual tie points. It's still not perfect probably not quite enough overlap but still I'm glad it works

Regarding the map not being actual thermal but rgb, plus with no lockable range the result is not super useful for any kind of analysis if I'm understanding correctly...
So what if we were able to convert the jpeg to flir rjpeg and process that, would it become usable for a true radiometric thermal map?
There is a guy who has a converted currently in beta, they talk about it on reddit

To view this content we will need your consent to set third party cookies.
For more detailed information, see our cookies page.

Accept third party cookies

I have tested the tool and it seems to work to convert to rjepg and/or tiff, however no success yet on stitch the thing yet with Pix4D even with the custom camera parameters

 

[jaja6009]

I think your stitching issues are from a lack or sufficient overlap as you said.

The lack of locking range is a problem but in my example, it did not affect the final outcome. I compared my images to the orthomap, and it is spot on.

I know that with the Matrice 300 RTK's H20T (Which shares the same exact thermal sensor), there have been several people that have been able to convert their thermal outputs into an RTiff and then were able to use these in Pix4D to make a true radiometric orthomap. So I think in the future someone will figure this all out and we will be able to get a stitched radiometric thermal map.

Thank you for the above link, I will definitely look into this.

 

[jaja6009]

I was able to use Drone Deploy for the same dataset and the results were interesting. From the link given on the Reddit post listed above someone stated that Drone Deploy could process a true radiometric orthomap with temperature data.

And yes it did work.

Screen Shot



It did not stitch as nicely as Pix4D, but it is not bad.

Temperature comparison. Comparing the Drone Deploy radiometric orthomap temperatures to temperatures in the RJPEG images using the DJI Thermal Analysis Tool.

Pix4D - No temperatures given as it is an RGB stitch
Drone Deploy
Fire Pit (Bright Spot in the front left corner of building.

Drone Deploy has the highest temperature as 82.5* F


DJI Thermal Analysis Tool shows a Max of 350*F


Drone Deploy has some work to do on this, but I am happy that they have made progress.
I hope the rumored massive update for the Enterprise Advanced is true and soon. Maybe we get some of the features that we have been asking for.

 

[Faquaral]

Thanks a lot for your input and sharing your research, definitely interesting to know that it works out of the box with DroneDeploy, though not very accurately yet.

How high above ground were you flying for this particular dataset? DroneDeploy makes a mess of our images, so there really is a problem with our flight pattern and/or height

 

[jaja6009]

Thanks a lot for your input and sharing your research, definitely interesting to know that it works out of the box with DroneDeploy, though not very accurately yet.

How high above ground were you flying for this particular dataset? DroneDeploy makes a mess of our images, so there really is a problem with our flight pattern and/or height

I was at 138 feet AGL.
90 front and 90 side overlap
Nadir camera angle
White Hot palette

I chose the Thermal Camera Option and not RGB in DJI Pilot.

When you take a good look at the Drone Deploy output, it had some stitching issues. The Camera Optimization is very poor. I think it will get better as they tweak the settings.

 

[Faquaral]

Thanks I will report back when we have done the flight and tested on Pix4d
Weather is currently stormy around here so we have to wait…

 

[Faquaral]

So we finally managed to get a flight done and thanks to your input were able to get a good dataset that did stitch in Pix4D rgb mapping


Then after converting the files to rjpeg using the tool mentioned previously and the camera calibration data from the rgb processing, we successfully stitched a thermal map with Pix4D, outputting a nice thermal reflectance map

All temperatures are in degrees centigrade

Reflectance map in Pix4D.


Zoom on an area of the Pix4D map with a different scale to focus on the hot spots


Same area on DJI Thermal Analysis Tool (just not the same orientation)

It seems the values are not too far off from the ones we get with DJI Thermal Analysis Tool, though as you may notice there is a very bizarre darker band in the Pix4D map, which roughly corresponds to the first part of the mission. That part of the building is not heated, so it's normal that it appears colder but the tarmac on the ground is the same all around the building.
Checking the temperatures in Thermal Analysis, it does fit with the Pix4D map, so this difference either exists or there is another source of error.
The first flight was performed before dawn and the second finished just after but it was an heavily overcast cold day (around 3°C) so not really that much radiation from the sun. Is it possible that there is a drift caused by the thermal sensor heating slightly during the flight as it's an uncooled sensor?
To give you an idea, the first flight was started on north of the building, second flight ended on the eastern side. For some unknown reason the drone flew slower or took more picts on the second flight even though the parameters were the same in dji pilot.


We also tried to stitch the data with DroneDeploy but it made a mess of it all, so yeah their camera optimization is not on point yet.

 

[jaja6009]

So we finally managed to get a flight done and thanks to your input were able to get a good dataset that did stitch in Pix4D rgb mapping
View attachment 139096

Then after converting the files to rjpeg using the tool mentioned previously and the camera calibration data from the rgb processing, we successfully stitched a thermal map with Pix4D, outputting a nice thermal reflectance map

All temperatures are in degrees centigrade
View attachment 139098
Reflectance map in Pix4D.

View attachment 139099
Zoom on an area of the Pix4D map with a different scale to focus on the hot spots

View attachment 139100
Same area on DJI Thermal Analysis Tool (just not the same orientation)

It seems the values are not too far off from the ones we get with DJI Thermal Analysis Tool, though as you may notice there is a very bizarre darker band in the Pix4D map, which roughly corresponds to the first part of the mission. That part of the building is not heated, so it's normal that it appears colder but the tarmac on the ground is the same all around the building.
Checking the temperatures in Thermal Analysis, it does fit with the Pix4D map, so this difference either exists or there is another source of error.
The first flight was performed before dawn and the second finished just after but it was an heavily overcast cold day (around 3°C) so not really that much radiation from the sun. Is it possible that there is a drift caused by the thermal sensor heating slightly during the flight as it's an uncooled sensor?
To give you an idea, the first flight was started on north of the building, second flight ended on the eastern side. For some unknown reason the drone flew slower or took more picts on the second flight even though the parameters were the same in dji pilot.
View attachment 139102

We also tried to stitch the data with DroneDeploy but it made a mess of it all, so yeah their camera optimization is not on point yet.
View attachment 139101

Wow, great job on this!

The Pix4D RGB map came out excellent. What settings did you use?
Overlap Side and front?
Altitude?
Nadir?

I will have to try the RJPEG to RTIFF Tool to process my data to see how my Pix4D Reflectance map turns out.

I want to try and figure out is I am overdoing it with my 90/90 overlap.

I still think that if we could lock the temperature range/span, the outputs would come out better.

Drone Deploy needs to tweak their camera settings as well as the temperature values, but seeing how they have always been very good with adding new support I think it will come.

When I get some time I will try a Reflectance map in Pix4D with my original data.

I will also be adding a solar panel field. Raptor Maps has a good video showing how to use DJI Pilot to obtain the correct orientation for solar fields.

 

[Faquaral]

Wow, great job on this!

The Pix4D RGB map came out excellent. What settings did you use?
Overlap Side and front?
Altitude?
Nadir?

For the flight we used settings as you suggested:
40m AGL (138ft)
90% overlap front and sides
100% Nadir images
DJI Pilot camera type: M2EA thermal
Photo mode timed interval shots
Don't remember the flight speed but very slow, total flight duration was around 40min
472 thermal images in total

For the RGB processing, I just used the standard pix4d mapping template

Camera calibration parameters obtained for the RGB map

Then for the Thermal map, I used the Thermal Camera preset:



And with the following custom camera calibration parameters:

I will have to try the RJPEG to RTIFF Tool to process my data to see how my Pix4D Reflectance map turns out.

I want to try and figure out is I am overdoing it with my 90/90 overlap.

I still think that if we could lock the temperature range/span, the outputs would come out better.

It would be interesting to know if stitching can be done with less than 90% overlap as it considerably increases flight time, during which external factors such as sun radiation will vary considerably I guess.

Let us know how it goes, especially regarding solar fields as we don't really have such things around here


I have been thinking about our reflectance map and this strange temperature variation around some part of the building. Notice that it's only around the heated part of the building, so I'm starting to think that we are seeing the radiated heat of the building being reflected on the tarmac.
I just watched this interesting webinar about emissivity and they talk about how reflectivity from other sources can be a very important factor

To view this content we will need your consent to set third party cookies.
For more detailed information, see our cookies page.

Accept third party cookies

My current theory so far, but I might be wrong:

 

[jaja6009]

Update on processing with photogrammetry software.

I just tried a data set of 215 thermal images on Capturing Reality - Reality Capture and the results were interesting.

It took around 2 minutes to process. I do know that your workstation will influence this.

The results are a non-radiometric orthomap that is better quality than Pix4D or Drone Deploy.
I forget what my overlap was, but I know it was most likely less than 90/90 for the thermal sensor. It may have been 90/90 for the RGB, but I am not sure what that relates to as far as the overlap for the thermal sensor.






While again it is not radiometric, it is a very high quality ortho considering the pixel count of the thermal sensor.

The resulting 3D model was not too good. I will run the image set I have with 90/90 on the thermal sensor and post my results.

This solar panel array lacked the proper overlap and Pix4D could not process it at all.

 

[jaja6009]

Here is the original data I gave from a moisture survey done in Reality Capture. It is 90/90 overlap on the thermal sensor.



It is similar to Pix4D's output.

So from my limited testing:

Pix4D fails to process lower overlap datasets, while Reality Capture can handle them.

In high overlap datasets, they both seem to make a similar quality output.

Once again, these are both non-radiometric orthos.

 

[Faquaral]

Very nice looking ortho, very nice
I wasn't aware reality capture was able to do ortho, let alone such nice ones. Good job and thanks for sharing

It's frustrating that it isn't radiometric ortho though!
Since we can't lock the scale on the M2EA, I guess these kinds of ortho may be affected by color scale changes, making it look like some areas are hotter or colder when they are not right?
The area south of the building (is it vegetation or dirt?) shows a curious pattern of warmer and colder patches, same for the area separating from the neighbor building, curious pattern of darker squar-ish patches. I wonder if it's the color changes causing this or something else?

 

[jaja6009]

Very nice looking ortho, very nice
I wasn't aware reality capture was able to do ortho, let alone such nice ones. Good job and thanks for sharing

It's frustrating that it isn't radiometric ortho though!
Since we can't lock the scale on the M2EA, I guess these kinds of ortho may be affected by color scale changes, making it look like some areas are hotter or colder when they are not right?
The area south of the building (is it vegetation or dirt?) shows a curious pattern of warmer and colder patches, same for the area separating from the neighbor building, curious pattern of darker squar-ish patches. I wonder if it's the color changes causing this or something else?

Since we can't lock the scale on the M2EA, I guess these kinds of ortho may be affected by color scale changes, making it look like some areas are hotter or colder when they are not right?
That is what I am thinking. In my opinion since this is RGB processing the fact that different images have different color shading from the range shifting due to the MAX and MIN temperature in the field of view changing, the same point in the map was made of images that had a different color of that point.

The area south of the building (is it vegetation or dirt?) shows a curious pattern of warmer and colder patches, same for the area separating from the neighbor building, curious pattern of darker squar-ish patches. I wonder if it's the color changes causing this or something else?
This sounds like a very plausible explanation.

Here is a good example of the shifting color palette due to no locking range.
Notice that when the very cold thin metal caps come into view the color shifts on the trucks.






Is the radiometric conversion software still available that you listed earlier in the thread? I am interested in this very much. And what is the pricing scheme? Pay to own or subscription?

DJI really should make their own RJPEG to TIFF application and that would pretty much solve our issues. They have Autel offering a very viable competitor in the Evo 2 Dual, DualR, and 640T. The Autel aircraft also are having problems right now with radiometric data in orthomaps. Believe it or not, several people think the Autel Evo 640T's images are the most superior in terms of it's sharpness through it image enhancement algorithms. I would like to see a comparison of the radiometric accuracy too though.

And FLIR's Boson core will mature during its life cycle through firmware updates, so I think you will see its outputs get better. I read this in an electronics forum when people there were comparing the Boson core to the old but tried and true Tau2.

 

[Faquaral]

It's curious I don't really see much of a difference between the two images you attached, actually they do look exactly the same but yeah I agree with you and it's definitely an issue to be aware of when processing RGB.

Yes the conversion software is still available, still in beta though but works fine.
you can write to [email protected] to ask for a license. Beta pricing is 150£ for annual and 400£ for perpetual, prices will go up once out of beta (soon). Below is an email all beta tester received.

I do agree it's ridiculous DJI didn't provide an option to at least export rjpeg from their thermal analysis tool, though it might be because of sanctions they are not allowed to use flir proprietary format perhaps.
What's perplexing is that they say the thermal pic format is r-jpeg in the specs sheet: Mavic 2 Enterprise Advanced - Specs - DJI

Yeah might be very interesting to have a comparison Autel vs DJI, though I have heard lots of stories about not so great customer service and products from Autel but if their camera is better, then it could be a nice option as I had the M2EA on loan and I'm thinking about getting a unit.
Just noticed the Evo2Thermal has a refresh rate of 30Hz for US and 9Hz for all other countries, which is pretty bad... Not specified on the 640T spec page however, so maybe this one is not limited for exports

And FLIR's Boson core will mature during its life cycle through firmware updates, so I think you will see its outputs get better. I read this in an electronics forum when people there were comparing the Boson core to the old but tried and true Tau2.

That sounds very promising but I'm not versed into the sensor details, so any drone using this boson core yet?
Both Autel and DJI seem to be using the same Uncooled VOx Microbolometer sensor, is that a Tau2?

 

[jaja6009]

Sorry, I posted two of the same image.

Here's an example.




Another example of the ground becoming brighter when the metal caps are in the picture.




Once the cold metal cap is out of the image the trucks are now darker.


The pricing for the conversion software is a bit out of my budget right now (Since it was not budgeted for).
I am hoping that DJI offers a simple conversion program along the lines of it Thermal Analysis Tool.

You can try your original images that did not process in Pix4D in Reality Capture. It is free to process, you only have to pay when you are happy with the results and want to export the data out. Since my original images of the solar panel array did not process in Pix4D but did in Reality Capture, you may find the same results. And as a bonus Reality Capture has a pay as you go payment system and the pricing for the 640 resolution thermal images is a fraction of RGB processing. My map cost 5 cents of credits that I had previously purchased.

 

[Faquaral]

Ah yes it's quite obvious now sadly. If only we could lock the scale, that would allow for at least visually consistent mapping, as sometimes radiometric processing isn't necessary and RGB would be enough just to spot various things/problems etc.
I will try my hand at reality capture yeah, seems pretty nice for other 3D reconstructions as well

Thanks for sharing and the suggestions!

 

[georgey]

First make sure you have very high overlap. I used 90 and 90, but will experiment with lower in the future.
Next do NOT choose the Mavic 640x512 camera option. Use the Flir Tau 2 option.
Do not choose to make a thermal map, but choose to make an RGB map.
It should work from here. When done processing, go back to the Images Properties Editor under Process and then go to Selected Camera model. Choose Edit, then Load Optimized Parameters. You can save for the future or not save. You can also save as your default, but I did not. Now go back and Reoptimize (Quicker) or run Step 1 all over again.
Your Camera Optimization on your Quality Report should now be under 5%.

Keep in mind, this is NOT a radiometric thermal map, it is an RGB processed map with no temperature values in it.
Also keep in mind, due to this thermal camera not allowing you to lock in a range, the same temperature assigned to a color will vary. Since the temperature range is not locked for the flight and as the scenery and temperatures in regards to highest and lowest in FOV change, 85 degrees in one image may be bright white and in another it may be several shades darker.

Let me know if you need help, I can PM you.

Sure Glad I ran into you guys, I aim in the same spot as you guys, after reading and trying for a few weeks, its been a challenge but I learned a lot. I hope they get this M2EA working proper with the Apps, The RTK to me seems so nice, as in steady, especially in the wind, you should get a More stable Drone and Pics. Thanks again for the info.