VPS infrared height sensors. What do they ACTUALLY accomplish??

[Zbip57]

Not the front/side/top/bottom obstacle avoidance sensors. And not the optical sensor used for horizontal position hold. I'm talking about the infrared sensors of the Downward Vision Positioning System (VPS).

The system is called different names in different models, and on older models used ultrasonic sensors rather than infrared. They function similarly in any case, sending a pulsed signal which bounces off the ground, with the reflected signal then received by the drone (like a radar altimeter). The time it takes (Time of Flight) between the signal being sent and returned is used to very accurately calculate the height of the drone above whatever lies below. That part I understand.

I also understand it has limited range. The user manual for my Mini says its operating range is 0.5-30m, but "works best" between 0.5-10m, and obviously the strength of the reflection is affected by different surfaces. When within range of a good reflection, the height is recorded in the flight log as VPS-height. The app for my old Phantom 3 Pro actively displayed the VPS height on screen, which was sometimes very useful, whereas the DJI Fly app for my Mini only ever shows the barometric altimeter referenced to zero from the takeoff location.

I previously posted a playlist of five different videos demonstrating how my Mini never reacts in any noticeable way to changes in the measured VPS height during flight or hover, UNLESS during autolanding or whenever a VPS height of less than 0.5m (~2ft) is detected. Other than in those exceptions, it is always the IMU and barometric altimeter which determine how the drone maintains constant height.

So what does the drone ACTUALLY DO with that VPS height information up to ~30m height? Yes, it is accurately measuring height. But for what purpose?

First, go read this very interesting thread, and give it some thought:
mavicpilots.com/threads/low-gps-signal-bad-vision-system-data-leading-to-fly-away-and-crash.148451/
Also see more details in the corresponding thread over on the DJI forum:
forum.dji.com/forum.php?mod=viewthread&tid=318816&pbc=mF6h4ZTt

That case involved a DJI Mini 3 Pro flown low over a fast moving river in the bottom of a steep canyon, resulting in loss of the drone for what seemed like obvious reasons. But the conclusions in that thread have caused me to reconsider pretty much everything I've ever believed and previously wrote about the functioning of the VPS system.

I've had some new thoughts about that now, which I will explain in my following five posts here. It involves "speed limits" and how those are implemented.

Hold your fire, keep your powder dry, read those other two message threads, and then read my following five posts...

 

[Zbip57]

The conclusion in that Chinese river canyon thread was that, when the drone was left to hover, its VPS optical sensor locked onto the moving river current causing the drone to drift away downstream. That made sense. Deep canyon. Poor GPS reception. VPS takes over and fools the drone into thinking it is "stationary" while it was actually tracking the foamy surface of the water flowing downstream. That can happen.

The more disturbing question was, why didn't the drone come back when the pilot gave it full forward stick input? And that's where we get into the question of "speed limits".

I used to fly model airplanes, and later a Cessna 172 and Piper Cherokee. It always made sense to me that, when flying at some fixed airspeed, your actual groundspeed will vary depending on whether you're struggling against a headwind or benefiting from a tailwind. Same thing happens with boats and canoes, making progress more slowly or faster depending on whether you're travelling upriver or downriver.

I always assumed it works the same way with our DJI drones. The user manual for my Mini lists different maximum speeds depending on which flight mode is selected.
13m/s S-Mode
8m/s P-Mode
4m/s C-mode
I assumed that maximum "airspeed" was regulated by limiting the pitch-angle of the drone. At 4m/s in Cinematic-Mode, with a 4m/s headwind it would effectively be stopped and not making any forward progress. And with a 4m/s tailwind it would be ripping along downwind at 8m/s groundspeed.

But that's not how it works.

The drone is "physically capable" of going at least as fast as 13m/s in Sport-Mode. So it's certainly strong enough to overcome a 4m/s headwind. And that's what happens. Even in C-mode it will still travel forward against a 4m/s headwind, while limited to a 4m/s "groundspeed". And travelling downwind in C-mode, it'll never go any faster than 4m/s groundspeed (unless pushed by a hurricane...).

The reason for that is, it calculates groundspeed, not airspeed, by using GPS info. [Actually it calculates all that stuff near instantaneously by integration from its IMU gyro and accelerometer data, but it supplements and verifies its positional calculations with frequent polling of the GPS info.]

That all makes sense, because Cinematic mode is meant to be slow and smooth with speed limited to 4m/s in every direction, regardless of windspeed (up to a physical limit of course).

So what happens in the absence of sufficient GPS reception? Well, then the VPS system takes over (provided sufficient lighting and altitude limits, etc).

But that's where things get interesting, and complicated...

 

[Zbip57]

The Vision Positioning System (VPS) is actually two separate [but combined] systems. The optical sensor detects patterns on the ground and is used for horizontal positioning whenever the drone is in hover. The infrared sensors (or older ultrasonic sensors) are used to accurately measure height.

But then what?

In the video playlist I posted earlier, the height sensors never appear to do anything useful unless detecting a height less than 0.5m (~2ft). So, what's the point of being able to measure heights up to ~30m (~100ft)???

I previously believed the VPS optical sensor only ever took over control of holding horizontal position in the absence of sufficient GPS reception, like when flying indoors. I strenuously argued that against other people who insisted I was wrong. I made that video dragging the rug under my Mini convinced it would prove my case.

As evident on the video, I was stunned to discover that I was wrong. Even with a solid GPS lock, the drone instead chose to follow the movement of the rug using its optical sensor. Apparently, the drone can select whichever source it trusts more, one or the other, GPS or VPS, while ignoring conflicting data received from the other system.

In either case, I was still convinced this problem (drifting away in hover) could ONLY ever occur while the drone is left to sit in hands-free hover. The autopilot then takes over control to maintain horizontal positioning however it sees fit. The pilot can always resume control at any time by moving the control sticks.

Well, it turns out I was wrong about that too! ...

 

[Zbip57]

Let's start with only the optical sensor. I'm going to simplify this as much as I can. If my understanding of any of this stuff is (yet again) out to lunch, PLEASE, anyone, feel free to step in and correct me to keep me from making too big a fool of myself...

The optical sensor is essentially a mini camera. I have no idea what resolution the sensor is actually capable of. Let's say just for a hypothetical it's 100x100 pixels.

Whenever the flight controller chooses to use the VPS, rather than the GPS, for holding hover position [all the way up to ~30m?], the optical sensor records and compares consecutive snapshots, checking to see whether the drone has moved. If the image shifts more than a certain number of pixels, the flight controller reacts to move the drone back to where it's supposed to be.

The drone knows only that the image has shifted by some number of pixels. But how does it know how strongly to react to that movement? Feedback loops of this type require a damping factor. If too much power is applied, the autopilot will overshoot the target and it'll end up oscillating back and forth chasing the target. But, if not enough power is applied, it'll take forever to reach the target.

If the drone is hovering only 0.5m (~2ft) above the ground, the entire 100x100 pixel image might be covering a footprint of only less than a square-foot. In that case, an image shift of a few pixels either way might not be worth any reaction at all.

But if the drone is hovering at a height of 30m (~100ft), that same 100x100 pixel image might be covering a huge footprint, where even a single pixel shift in any direction might equate to a drift of significant distance!

How does the optical sensor know whether each pixel is seeing only a square-inch, a square-foot, or an entire acre whenever the image shifts by one pixel?

That's why the VPS system requires an accurate measure of the height between it and the image snapped by the optical sensor!

The barometric altimeter can only display the height difference referenced to the takeoff location. It has no way of knowing how much the varying terrain below has risen or descended. The VPS infrared sensor (when within range) provides that function.

The infrared height sensor info, [except when auto-landing or within 0.5m of obstacles] never takes an active role in adjusting the drone's height [as demonstrated in my video playlist], but the infrared sensors play a vital role in determining the size represented by each pixel of the optical sensor.

 

[Zbip57]

The user manual for my Mini shows this diagram illustrating the viewing angle of the VPS sensor. I'm assuming that's for the optical sensor. I've added some details.

Given the viewing angles of the optical sensor (56° & 71°), and the height (h) measured accurately by the infrared sensors, the total length and width of the ground footprint seen by the optical sensor can be calculated using geometry. From this, the area recorded by each individual pixel can be determined.



I don't know how much of that is actually true. The geometry calculations are more complicated if the drone isn't sitting level, and/or the ground is sloped, etc. But that's my theory.

Does this make any sense?

 

[Zbip57]

This all came from thinking more about that curious incident with the drone crashing in the Chinese river valley.

The flight log showed it actually had GPS reception throughout the entire flight, accurately tracing the path taken by the drone. The drone did stop and reliably hold position above the river, but only until the pilot turned it around (rotated in yaw) to have it face back upstream toward himself.

While he was distracted swapping between photo and video modes, the drone suddenly lost confidence in its GPS solution and switched instead to relying on its VPS system. The optical sensor then tracked the fast flowing river current causing the drone to drift away downstream.

The PhantomHelp analysis shows the drone holding near-zero speed while it was actually moving rapidly downstream. That's because its positional information was now coming from the VPS rather than GPS.

Even though the GPS clearly indicates the drone moving, the VPS insisted the drone was standing still holding position to track the flowing river current. So far, it's reacting pretty much exactly as I demonstrated in my video where I'm dragging the rug away.

But here's where it gets confusing. From the pilot's perspective, he finally noticed the drone rapidly drifting away from him. So he gave it full forward stick input to bring it back toward him. But the drone merely came to a halt and wouldn't come back! What the ....?

I've always been convinced that this drifting scenario caused by the VPS sensing moving patterns could only occur while the control sticks are hands-free centred when the autopilot takes over to hold the drone in hover. Why wouldn't the drone come back now with full forward stick input?

It's because of the drone's programmed speed limit!

If it had been using GPS data to figure out its position, the Mini 3 Pro should have come back at it's Normal-Mode speed limit of 10m/s. But, it was relying on its VPS instead. It went from what it thought was a standstill (while actually travelling backwards at 10m/s following the current) to its full forward speed limit of 10m/s relative to the water's surface (now actually not making any progress at all toward the pilot)!

That's what had me puzzling over how the VPS even measures speed. It's only capable of seeing a photo image shifting by a certain number of pixels per second. It needs to know what actual distance is represented when the image shifts by each one of those pixels.

My theory is it calculates that pixel size through geometry by using the viewing angle of the optical sensor's lens and the height measured by the VPS infrared sensors.

Let me know. Does that make sense to anybody?