Thermal Ortho at Night-Problems Stitching

[Errantlunchbox]

I'm having an issue with stitching thermal images I capture at night. Trying to make a thermal ortho of commercial roofs for roofing inspections. I can run the same mission in the day and i have no problems stitching the thermal images from that mission then at night most images are not able to calibrate in PIX4D. My main theory is that since there is not a way to disable the visual camera when flying a thermal mission, the visual camera only has auto exposure settings and is having a longer exposure for the night environment which leads to less overlap than what is needed to stitch thermal images together.

Wondering if anyone else has run into this issue or if anyone has any ideas to work around this problem.

thanks!

 

[DanJ]

Are you using the DJI app to do the stitching?
You might want to try another app like PTGUI to do it and see if it works.
My experience is that one program doesn't do everything perfectly. I use a few different apps to stitch.

 

[jaja6009]

He's not stitching a panorama. He's using photogrammetry/structure from motion.
But you do have a point in trying a different piece of software to see if it helps.

OP, are you sure you have the correct camera settings for focal length, sensor size etc.?

When I tried my M2EA thermal images in Pix4D, the resulting non-radiometric ortho was not that good. Agisoft was a bit better and Drone Deploy not only made a good looking ortho, but ir also used the proprietary DJI thermal parameters to make a radiometric-ortho.

 

[Errantlunchbox]

Are you using the DJI app to do the stitching?
You might want to try another app like PTGUI to do it and see if it works.
My experience is that one program doesn't do everything perfectly. I use a few different apps to stitch.

PIX4D

 

[Errantlunchbox]

He's not stitching a panorama. He's using photogrammetry/structure from motion.
But you do have a point in trying a different piece of software to see if it helps.

OP, are you sure you have the correct camera settings for focal length, sensor size etc.?

When I tried my M2EA thermal images in Pix4D, the resulting non-radiometric ortho was not that good. Agisoft was a bit better and Drone Deploy not only made a good looking ortho, but ir also used the proprietary DJI thermal parameters to make a radiometric-ortho.

I'm also using the M2EA and Pix4D. The daytime mission stitches together perfectly, RGB and thermal. It is the night flights that the thermal ortho is having issues and I think it's related to the RGB camera trying to get a balanced exposure which is taking too long and messing with my overlap.

I came across another thread on here where it went into great detail about how to get the thermal images to stitch well with the M2EA and Pix4D. They said to use the Tau2 sensor and that made a massive improvement with the stitching. I might try and add a custom camera with all the correct parameters for the M2EA and see if it stitches as well on a known map I already made to see how it does.

I plan on running a test tonight where I lower the EV as low as I can get it to see if it will speed up the RGB camera and keep my overlap where it needs to be.

 

[Errantlunchbox]

UPDATE-
There is definitely a connection of the visible camera trying to take a long exposure at night that cannot be adjusted. All camera setting are on auto. I went through all the settings for the camera and there is no way to get manual camera settings. Guess DJI would have to some out with a fix to allow selection of which sensor the pilot would want to use for each case.
Below is the same flight with all the same settings and AGL. Only difference was the time of day. When I looked at the RGB photos, most were around a one second exposure time.

 

[jaja6009]

UPDATE-
There is definitely a connection of the visible camera trying to take a long exposure at night that cannot be adjusted. All camera setting are on auto. I went through all the settings for the camera and there is no way to get manual camera settings. Guess DJI would have to some out with a fix to allow selection of which sensor the pilot would want to use for each case.
Below is the same flight with all the same settings and AGL. Only difference was the time of day. When I looked at the RGB photos, most were around a one second exposure time.
View attachment 171152

Can you choose the setting to not save RGB and current view images?

 

[Errantlunchbox]

Can you choose the setting to not save RGB and current view images?

Not that I'm aware of.... both cameras are active at all time and no way to only record images from a single source.

 

[jaja6009]

There is a setting in in FPV view in Pilot. Press Menu, Camera Icon, Save Photo. You can set what images are saved: Current View, Visible Photo and Infrared.
I assume these settings will hold even in a Mapping Mission.

Also when setting up your mapping mission, are you using Mavic 2 Enterprise Advanced Thermal for the Camera Type?

You have made a really good find showing that at night with the Auto settings the Visible Camera's long exposure will interfere with the overlap of images.

Some points.
Are you trying to disbale saving the Current View and Visible using the settings above?
Are you Using Mavic 2 Advanced Thermal for Camera Type?
What are your overlap settings. The Front settings should be 90 for a thermal ortho.

The last thermal ortho I made had my altitude at 148 ft AGL with 90 overlap front and 90 overlap side. My speed was 2.3 mph. The ortho stitched fine in Agisoft Metashape and Drone Deploy.

 

[jaja6009]

I just did a test.

Test
I used the Save Photo to only save the Thermal images.
I used the Save Photo to save both Thermal and Visible images.
90/90 overlap
Flight Speed at 2.3 mph
Altitude 150' AGL
I used Mavic 2 Enterprise Advanced Thermal for Camera type
Mission flown in the dark.

I saw no difference in overlap between when both images were saved and only the thermal was saved. (I only did one flight line with both images being saved).

 

[Errantlunchbox]

I just did a test.

Test
I used the Save Photo to only save the Thermal images.
I used the Save Photo to save both Thermal and Visible images.
90/90 overlap
Flight Speed at 2.3 mph
Altitude 150' AGL
I used Mavic 2 Enterprise Advanced Thermal for Camera type
Mission flown in the dark.

I saw no difference in overlap between when both images were saved and only the thermal was saved. (I only did one flight line with both images being saved).

Thank you for that info!
I will try another test but I was not able to find the save photo setting that you are referring to. I'll check around again to make sure I didn't miss it. To be clear I am also using the Mavic 2 Enterprise Advanced

 

[jaja6009]

I don't have the drone right now as it is at DJI for Basic Maintenance.

The settings are in the camera settings somewhere.
You can choose whether to save RGB, Thermal and Current View.

You may not have the correct firmware for choosing what file types to save.
You may have to update your firmware.

From DJI M2EA Downloads:

Date: 2022.3.30Aircraft Firmware: v01.00.0428Smart Controller Firmware: v01.01.0064DJI Pilot App: v2.5.1.3DJI Assistant 2: v2.1.3What’s New Optimized Mission Flight：1. Fixed the issue where flight route cannot be uploaded if the gimbal tilt is set to over 0 degrees.2. Added support to automatically add waypoints and record camera zoom when taking photos in LiveMission Recording.3. Removed the minimum distance limitation of 0.5 m between two consecutive waypoints in LiveMission Recording. Allowed manual selection of photo types for saving.

If you cannot find it, I will let you know as soon as my aircraft returns from DJI.

My test was a bit different than your.
I will try your test also, with a daylight and then light mapping grid mission and look at my results.