Automated Super-Resolution 360 Panorama Capture for Litchi

DJ Wes · Mar 6, 2024

As many here already know, I have a utility that allows one to incorporate one or more 360-degree panorama captures into a waypoint mission. This utility is working well and I can see that many are using it.

While that utility is working well, it has some limitations. For example, it was written to mimic the photo capture process when using a typical wide-angle lens found on most/all drones. I have written a new 360-degree panorama capture mission creator for Litchi that computes the photo orientations (headings and gimbal pitches) based on your choice of lens and desired overlap. This way, one can select a telephoto lens and my application will compute the orientation for all of the photos that will make up a high (or super) resolution 360-degree panorama.

Although, this utility creates a Litchi CSV file, that CSV file can then be converted to a DJI Fly waypoint mission and executed with a Mavic 3, Air 3, or Mini 4 Pro drone. DJI Fly waypoint missions are stored using the WPML language. WPML does support multiple camera actions at a single waypoint. However, DJI Fly’s implementation of WPML does not. For that reason, when using “DJI Fly Compatibility Mode”, each “Take Photo” action is attached to its own waypoint with dummy waypoints are placed between each “Take Photo” waypoint. This results in many waypoints, but it is the only method available to me for capturing these photos.

I am looking for testers. I have tested this using a Mavic 2 Zoom with the camera set to 2x with excellent results. I am interested to hear from people who own one of the drones with telephoto lenses such as a Mavic 3 (Classic or Pro) or an Air 3. For those drones, it will require a two-step process:

Create a 360-degree panorama mission while specifying the telephoto lens.
Convert the resulting CSV file to a DJI Fly Waypoint mission using one of my other utilities.

I am also curious if anyone has attempted to create a DJI Fly mission with consecutive waypoints sharing the exact same latitude and longitude coordinates. If that were possible, it would eliminate the need to place “dummy” waypoints between them.

Here is a direct link to this new utility:

360° Panorama Creator

Wes Barris

www.litchiutilities.com

Chrislaf · Mar 6, 2024

Looking forward to trying this out on my Mavic 3 Pro particularly with the 166mm lens. I usually fly manually to the area I want to do a 360 pano and use the automated function especially because we are still getting some cold weather days. Being able to use the 166mm camera and do a 360 pano is an exciting prospect for some of the things I photograph. Going to create some missions to test out and try once the weather improves.

Thanks,
Chris

superskareb · Mar 14, 2024

Thanks for developing this tool for customizable 360 panoramas.

Would you be intending to expand the range of lens FOV in the near future? i noticed you have options for 20 FOV all the way till 100 FOV in increments of 10, it would be nice if you could include an option for us to enter a custom FOV value.

I want to try to capture a 360 panorama with the Panasonic 25mm on my inspire 2, the FOV in FF equivalent is 46.8, so i am left to decide on whether i should use 40 or 50 FOV value, otherwise i'll have to manually trial and error to see so many extra rows & columns i would have to add after the flight.

Appreciate all your good work on this!!

DJ Wes · Mar 14, 2024

superskareb said:
I want to try to capture a 360 panorama with the Panasonic 25mm on my inspire 2, the FOV in FF equivalent is 46.8, so i am left to decide on whether i should use 40 or 50 FOV value, otherwise i'll have to manually trial and error to see so many extra rows & columns i would have to add after the flight.

If you can provide the sensor aspect ratio or preferably point me to the specifications for that camera, I can add it to my application.

MS Coast · Mar 19, 2024

DJ Wes said:
I am also curious if anyone has attempted to create a DJI Fly mission with consecutive waypoints sharing the exact same latitude and longitude coordinates. If that were possible, it would eliminate the need to place “dummy” waypoints between them.

I've tried a few waypoint flights with the Mavic 4 Pro using multiple waypoints at the same location and performing various pauses or camera actions at each waypoint. I also created waypoints while holding lateral position constant and changing altitude. All seems to proceed well, with one disquieting thing.

While flying along waypoints all having the same lateral position but different altitudes, the drone flies in a smoothed path in the Z dimension, as it does in the XY plane. Rather than climbing or descending, it moves to the side along the way before returning to the same lateral position at the next waypoint.

DJ Wes · Mar 19, 2024

MS Coast said:
I've tried a few waypoint flights with the Mavic 4 Pro using multiple waypoints at the same location and performing various pauses or camera actions at each waypoint. I also created waypoints while holding lateral position constant and changing altitude. All seems to proceed well, with one disquieting thing.

While flying along waypoints all having the same lateral position but different altitudes, the drone flies in a smoothed path in the Z dimension, as it does in the XY plane. Rather than climbing or descending, it moves to the side along the way before returning to the same lateral position at the next waypoint.

That's interesting. It sounds like I can remove those intermediate waypoints when one of these missions is created for DJI Fly.

markroberts · Apr 12, 2024

DJ Wes said:
That's interesting. It sounds like I can remove those intermediate waypoints when one of these missions is created for DJI Fly.

I've tried the 360 Pano mission and it seems to ignore / mess with the heading offsets. All of the files (litchi_mission.csv, mission.csv, mission.kml) have what look like correct offsets but when I import it into the Mission Hub, the arrows all point north, north west or north east, never in the southern half of the circle.

I've used your Litchi Mission to DJI Fly or Pilot 2 Waypoints converter to import the mission.csv and convert it to .KMZ and swapped it into a dummy waypoint mission on my Mini 4 Pro and it flies the mission taking shots of the northern side only. What am I doing wrong?

DJ Wes · Apr 12, 2024

markroberts said:
I've tried the 360 Pano mission and it seems to ignore / mess with the heading offsets. All of the files (litchi_mission.csv, mission.csv, mission.kml) have what look like correct offsets but when I import it into the Mission Hub, the arrows all point north, north west or north east, never in the southern half of the circle.

In order for the heading values in the CSV file to be used, Litchi must have its heading mode set to "Custom".

markroberts · Apr 12, 2024

DJ Wes said:
In order for the heading values in the CSV file to be used, Litchi must have its heading mode set to "Custom".
View attachment 174241

The Heading Mode is already set to "Custom". When I toggle through all of the waypoints created by the utility, the
"Rotate Aircraft" field shows correct headings with the full 360 degrees covered, but the arrows displayed on the Mission Hub all point North or North West, not matching the settings for each waypoint as in my screen shot sent by email yesterday.
I've now found that the Panorama function is native to the Mini 4 Pro, so I won't be using the Litchi version. It would be good to resolve this for other users (unless there is some reason why it is unique to me!). Your other utilities that I've tried so far seem to work really well!

DJ Wes · Apr 13, 2024

markroberts said:
The Heading Mode is already set to "Custom". When I toggle through all of the waypoints created by the utility, the
"Rotate Aircraft" field shows correct headings with the full 360 degrees covered, but the arrows displayed on the Mission Hub all point North or North West, not matching the settings for each waypoint as in my screen shot sent by email yesterday.
I've now found that the Panorama function is native to the Mini 4 Pro, so I won't be using the Litchi version. It would be good to resolve this for other users (unless there is some reason why it is unique to me!). Your other utilities that I've tried so far seem to work really well!
View attachment 174288 View attachment 174289

Ahh. Now I understand what you are saying. In the Mission Hub, the headings used for the waypoint icons are the headings configured at each waypoint, which are zero. However, the headings used for taking the 360 deg photo components are part of the camera/waypoint actions. Those camera actions are not used to display the waypoint icons. I hope that makes sense. What you are seeing in the Mission Hub is correct and expected.

markroberts · Apr 13, 2024

DJ Wes said:
Ahh. Now I understand what you are saying. In the Mission Hub, the headings used for the waypoint icons are the headings configured at each waypoint, which are zero. However, the headings used for taking the 360 deg photo components are part of the camera/waypoint actions. Those camera actions are not used to display the waypoint icons. I hope that makes sense. What you are seeing in the Mission Hub is correct and expected.

Are you saying that the arrow directions in the Mission Hub at each waypoint are not the actual headings of the drone at those waypoints? If so, that's not correct. The shots are in the directions of each arrow at each waypoint as in my screenshot. I can show you my RC2 screen recording if you like.
The native Mini 4 Pro Panorama works perfectly, so I won't be using this utility anyway.

DJ Wes · Apr 13, 2024

markroberts said:
Are you saying that the arrow directions in the Mission Hub at each waypoint are not the actual headings of the drone at those waypoints? If so, that's not correct. The shots are in the directions of each arrow at each waypoint as in my screenshot. I can show you my RC2 screen recording if you like.
The native Mini 4 Pro Panorama works perfectly, so I won't be using this utility anyway.

I cannot help you if you keep jumping from one piece of software to the next. You have mentioned Litchi’s Mission Hub, Litchi, and now, the RC2 (which is running DJI Fly). The responses I give will depend on what software you are using.

Let’s start with Litchi’s Mission Hub. The waypoint icons in the Mission Hub point in the direction of the heading defined at each waypoint. The heading defined at each waypoint from my panorama software is zero (unless the waypoint is either the last or first waypoint of a multi-panorama mission). This is by design and expected. Each photo action is preceded by a “rotate aircraft” action. There are multiple “rotate aircraft” and “take photo” actions configured at each waypoint. You can see these actions by scrolling through the actions in the Mission Hub. These actions will not change the waypoint icons in any way.

This configuration has been tested with multiple drones using Litchi. Before you mention any other software, I need you to understand how the mission is configured and how it will appear in the Mission Hub. Once that is established, we can move on to flying one of these mission with the Litchi app.

markroberts · Apr 13, 2024

DJ Wes said:
I cannot help you if you keep jumping from one piece of software to the next. You have mentioned Litchi’s Mission Hub, Litchi, and now, the RC2 (which is running DJI Fly). The responses I give will depend on what software you are using.

Let’s start with Litchi’s Mission Hub. The waypoint icons in the Mission Hub point in the direction of the heading defined at each waypoint. The heading defined at each waypoint from my panorama software is zero (unless the waypoint is either the last or first waypoint of a multi-panorama mission). This is by design and expected. Each photo action is preceded by a “rotate aircraft” action. There are multiple “rotate aircraft” and “take photo” actions configured at each waypoint. You can see these actions by scrolling through the actions in the Mission Hub. These actions will not change the waypoint icons in any way.

This configuration has been tested with multiple drones using Litchi. Before you mention any other software, I need you to understand how the mission is configured and how it will appear in the Mission Hub. Once that is established, we can move on to flying one of these mission with the Litchi app.

I can’t fly it with the Litchi Hub. I use the RC2 controller. Earlier in this thread you said you are looking for testers. That’s why I’m trying to make it work. Earlier in this thread you said the litchi mission can be converted to a DJI mission using one of your other utilities. I assume this is “Litchi Waypoint Mission to DJI Fly Waypoint Mission”. Is that correct? If so, it doesn’t work. The mission flies by taking pictures in the direction of the arrow heads in the screen shots I attached.
I’m trying to provide you with testing results, not to actually use the software. I haven’t been jumping from one piece of software to the next, rather I’ve been using the applications as I have previously to convert to a DJI mission.
The DJI Fly native Pano function does the job.
I’ll end my testing now. Thanks for reading my comments