VPS infrared height sensors. What do they ACTUALLY accomplish??

[Zbip57]

Not the front/side/top/bottom obstacle avoidance sensors. And not the optical sensor used for horizontal position hold. I'm talking about the infrared sensors of the Downward Vision Positioning System (VPS).

The system is called different names in different models, and on older models used ultrasonic sensors rather than infrared. They function similarly in any case, sending a pulsed signal which bounces off the ground, with the reflected signal then received by the drone (like a radar altimeter). The time it takes (Time of Flight) between the signal being sent and returned is used to very accurately calculate the height of the drone above whatever lies below. That part I understand.

I also understand it has limited range. The user manual for my Mini says its operating range is 0.5-30m, but "works best" between 0.5-10m, and obviously the strength of the reflection is affected by different surfaces. When within range of a good reflection, the height is recorded in the flight log as VPS-height. The app for my old Phantom 3 Pro actively displayed the VPS height on screen, which was sometimes very useful, whereas the DJI Fly app for my Mini only ever shows the barometric altimeter referenced to zero from the takeoff location.

I previously posted a playlist of five different videos demonstrating how my Mini never reacts in any noticeable way to changes in the measured VPS height during flight or hover, UNLESS during autolanding or whenever a VPS height of less than 0.5m (~2ft) is detected. Other than in those exceptions, it is always the IMU and barometric altimeter which determine how the drone maintains constant height.

So what does the drone ACTUALLY DO with that VPS height information up to ~30m height? Yes, it is accurately measuring height. But for what purpose?

First, go read this very interesting thread, and give it some thought:
mavicpilots.com/threads/low-gps-signal-bad-vision-system-data-leading-to-fly-away-and-crash.148451/
Also see more details in the corresponding thread over on the DJI forum:
forum.dji.com/forum.php?mod=viewthread&tid=318816&pbc=mF6h4ZTt

That case involved a DJI Mini 3 Pro flown low over a fast moving river in the bottom of a steep canyon, resulting in loss of the drone for what seemed like obvious reasons. But the conclusions in that thread have caused me to reconsider pretty much everything I've ever believed and previously wrote about the functioning of the VPS system.

I've had some new thoughts about that now, which I will explain in my following five posts here. It involves "speed limits" and how those are implemented.

Hold your fire, keep your powder dry, read those other two message threads, and then read my following five posts...

 

[Zbip57]

The conclusion in that Chinese river canyon thread was that, when the drone was left to hover, its VPS optical sensor locked onto the moving river current causing the drone to drift away downstream. That made sense. Deep canyon. Poor GPS reception. VPS takes over and fools the drone into thinking it is "stationary" while it was actually tracking the foamy surface of the water flowing downstream. That can happen.

The more disturbing question was, why didn't the drone come back when the pilot gave it full forward stick input? And that's where we get into the question of "speed limits".

I used to fly model airplanes, and later a Cessna 172 and Piper Cherokee. It always made sense to me that, when flying at some fixed airspeed, your actual groundspeed will vary depending on whether you're struggling against a headwind or benefiting from a tailwind. Same thing happens with boats and canoes, making progress more slowly or faster depending on whether you're travelling upriver or downriver.

I always assumed it works the same way with our DJI drones. The user manual for my Mini lists different maximum speeds depending on which flight mode is selected.
13m/s S-Mode
8m/s P-Mode
4m/s C-mode
I assumed that maximum "airspeed" was regulated by limiting the pitch-angle of the drone. At 4m/s in Cinematic-Mode, with a 4m/s headwind it would effectively be stopped and not making any forward progress. And with a 4m/s tailwind it would be ripping along downwind at 8m/s groundspeed.

But that's not how it works.

The drone is "physically capable" of going at least as fast as 13m/s in Sport-Mode. So it's certainly strong enough to overcome a 4m/s headwind. And that's what happens. Even in C-mode it will still travel forward against a 4m/s headwind, while limited to a 4m/s "groundspeed". And travelling downwind in C-mode, it'll never go any faster than 4m/s groundspeed (unless pushed by a hurricane...).

The reason for that is, it calculates groundspeed, not airspeed, by using GPS info. [Actually it calculates all that stuff near instantaneously by integration from its IMU gyro and accelerometer data, but it supplements and verifies its positional calculations with frequent polling of the GPS info.]

That all makes sense, because Cinematic mode is meant to be slow and smooth with speed limited to 4m/s in every direction, regardless of windspeed (up to a physical limit of course).

So what happens in the absence of sufficient GPS reception? Well, then the VPS system takes over (provided sufficient lighting and altitude limits, etc).

But that's where things get interesting, and complicated...

 

[Zbip57]

The Vision Positioning System (VPS) is actually two separate [but combined] systems. The optical sensor detects patterns on the ground and is used for horizontal positioning whenever the drone is in hover. The infrared sensors (or older ultrasonic sensors) are used to accurately measure height.

But then what?

In the video playlist I posted earlier, the height sensors never appear to do anything useful unless detecting a height less than 0.5m (~2ft). So, what's the point of being able to measure heights up to ~30m (~100ft)???

I previously believed the VPS optical sensor only ever took over control of holding horizontal position in the absence of sufficient GPS reception, like when flying indoors. I strenuously argued that against other people who insisted I was wrong. I made that video dragging the rug under my Mini convinced it would prove my case.

As evident on the video, I was stunned to discover that I was wrong. Even with a solid GPS lock, the drone instead chose to follow the movement of the rug using its optical sensor. Apparently, the drone can select whichever source it trusts more, one or the other, GPS or VPS, while ignoring conflicting data received from the other system.

In either case, I was still convinced this problem (drifting away in hover) could ONLY ever occur while the drone is left to sit in hands-free hover. The autopilot then takes over control to maintain horizontal positioning however it sees fit. The pilot can always resume control at any time by moving the control sticks.

Well, it turns out I was wrong about that too! ...

 

[Zbip57]

Let's start with only the optical sensor. I'm going to simplify this as much as I can. If my understanding of any of this stuff is (yet again) out to lunch, PLEASE, anyone, feel free to step in and correct me to keep me from making too big a fool of myself...

The optical sensor is essentially a mini camera. I have no idea what resolution the sensor is actually capable of. Let's say just for a hypothetical it's 100x100 pixels.

Whenever the flight controller chooses to use the VPS, rather than the GPS, for holding hover position [all the way up to ~30m?], the optical sensor records and compares consecutive snapshots, checking to see whether the drone has moved. If the image shifts more than a certain number of pixels, the flight controller reacts to move the drone back to where it's supposed to be.

The drone knows only that the image has shifted by some number of pixels. But how does it know how strongly to react to that movement? Feedback loops of this type require a damping factor. If too much power is applied, the autopilot will overshoot the target and it'll end up oscillating back and forth chasing the target. But, if not enough power is applied, it'll take forever to reach the target.

If the drone is hovering only 0.5m (~2ft) above the ground, the entire 100x100 pixel image might be covering a footprint of only less than a square-foot. In that case, an image shift of a few pixels either way might not be worth any reaction at all.

But if the drone is hovering at a height of 30m (~100ft), that same 100x100 pixel image might be covering a huge footprint, where even a single pixel shift in any direction might equate to a drift of significant distance!

How does the optical sensor know whether each pixel is seeing only a square-inch, a square-foot, or an entire acre whenever the image shifts by one pixel?

That's why the VPS system requires an accurate measure of the height between it and the image snapped by the optical sensor!

The barometric altimeter can only display the height difference referenced to the takeoff location. It has no way of knowing how much the varying terrain below has risen or descended. The VPS infrared sensor (when within range) provides that function.

The infrared height sensor info, [except when auto-landing or within 0.5m of obstacles] never takes an active role in adjusting the drone's height [as demonstrated in my video playlist], but the infrared sensors play a vital role in determining the size represented by each pixel of the optical sensor.

 

[Zbip57]

The user manual for my Mini shows this diagram illustrating the viewing angle of the VPS sensor. I'm assuming that's for the optical sensor. I've added some details.

Given the viewing angles of the optical sensor (56° & 71°), and the height (h) measured accurately by the infrared sensors, the total length and width of the ground footprint seen by the optical sensor can be calculated using geometry. From this, the area recorded by each individual pixel can be determined.



I don't know how much of that is actually true. The geometry calculations are more complicated if the drone isn't sitting level, and/or the ground is sloped, etc. But that's my theory.

Does this make any sense?

 

[Zbip57]

This all came from thinking more about that curious incident with the drone crashing in the Chinese river valley.

The flight log showed it actually had GPS reception throughout the entire flight, accurately tracing the path taken by the drone. The drone did stop and reliably hold position above the river, but only until the pilot turned it around (rotated in yaw) to have it face back upstream toward himself.

While he was distracted swapping between photo and video modes, the drone suddenly lost confidence in its GPS solution and switched instead to relying on its VPS system. The optical sensor then tracked the fast flowing river current causing the drone to drift away downstream.

The PhantomHelp analysis shows the drone holding near-zero speed while it was actually moving rapidly downstream. That's because its positional information was now coming from the VPS rather than GPS.

Even though the GPS clearly indicates the drone moving, the VPS insisted the drone was standing still holding position to track the flowing river current. So far, it's reacting pretty much exactly as I demonstrated in my video where I'm dragging the rug away.

But here's where it gets confusing. From the pilot's perspective, he finally noticed the drone rapidly drifting away from him. So he gave it full forward stick input to bring it back toward him. But the drone merely came to a halt and wouldn't come back! What the ....?

I've always been convinced that this drifting scenario caused by the VPS sensing moving patterns could only occur while the control sticks are hands-free centred when the autopilot takes over to hold the drone in hover. Why wouldn't the drone come back now with full forward stick input?

It's because of the drone's programmed speed limit!

If it had been using GPS data to figure out its position, the Mini 3 Pro should have come back at it's Normal-Mode speed limit of 10m/s. But, it was relying on its VPS instead. It went from what it thought was a standstill (while actually travelling backwards at 10m/s following the current) to its full forward speed limit of 10m/s relative to the water's surface (now actually not making any progress at all toward the pilot)!

That's what had me puzzling over how the VPS even measures speed. It's only capable of seeing a photo image shifting by a certain number of pixels per second. It needs to know what actual distance is represented when the image shifts by each one of those pixels.

My theory is it calculates that pixel size through geometry by using the viewing angle of the optical sensor's lens and the height measured by the VPS infrared sensors.

Let me know. Does that make sense to anybody?

 

[Yorkshire_Pud]

I am of the opinion that the IR sensor will, normally, reduce the speed of a fast descent once the ground is detected. I am sure I have seen this in my drones, most likely the mavic 2s and mini 1&2. ..... and that's with the throttle fully closed all the way down to landing.

The P3 doesn't seem to do this, as, after getting used to mavic-style behaviour, I found out when next flying my P3.
A burst of full-throttle saved it.

There might be two or three stages of speed reduction.
In what thread did you post you recent 'I was staggered that the VPS system can cause tracking, video'?
I think a few people are citing it but in the wrong context. I looked for it but I can't find the thread.

 

[Meta4]

I assumed that maximum "airspeed" was regulated by limiting the pitch-angle of the drone.

Perhaps it was that way with earlier models.

The drone is "physically capable" of going at least as fast as 13m/s in Sport-Mode. So it's certainly strong enough to overcome a 4m/s headwind. And that's what happens. Even in C-mode it will still travel forward against a 4m/s headwind, while limited to a 4m/s "groundspeed".

The reason for that is, it calculates groundspeed, not airspeed, by using GPS info.

Yes, the more modern models will often achieve the max speed for a certain flight mode with less than the max tilt angle.
But they are holding back a reserve and if battling a strong headwind they will tilt further than what used to be the max tilt angle for that flight mode to achieve greater speed.
I think this started back when a firmware update opened up greater speed potential for the original Mini when so many were unable to make headway against only moderate winds.

 

[hank970]

Just a wild guess, but I suspect that inputs from the IR detector regulate the rate of descent when landing in response to an RTH command -- those last few meters above the ground when descent slows considerably to ensure a soft touch-down.

 

[Zbip57]

I assumed that maximum "airspeed" was regulated by limiting the pitch-angle of the drone.

Perhaps it was that way with earlier models.

I still have my original first generation DJI Phantom. The controller allowed switching directly between GPS-mode or ATTI-mode. It flew faster in ATTI-mode because of a steeper allowable pitch angle.

One annoyance was that the original controller had no centring spring on the throttle stick. It would stay at whichever position it was set, idle at all the way down and full throttle all the way up. It was up to you to find the sweet spot at half-throttle where it would hold altitude.

It did have a barometric altimeter though, because it did use that to automatically climb and hold altitude in RTH.

The three-way switch on the controller by default enabled switching between GPS, ATTI, or ATTI (again). In either mode, when the control sticks were centred the Phantom would automatically level itself. But in ATTI mode it wouldn't brake to a halt and hold position. That third position on the switch could be reconfigured to a choice between ATTI, RTH, or Manual-mode!

In Manual-mode there were no limits to pitch or roll. From centred to full stick input, rather than commanding the Phantom to pitch or roll proportionally up to a firmware limited angle, in Manual-mode the greater the stick input the faster it pitched or rolled, and continued pitching or rolling!

Bringing the sticks back to centre would make the Phantom stop rolling or pitching and hold whatever attitude it was currently at, even if fully inverted. Yikes. I was too chicken to ever try Manual-mode.

[...] I think this started back when a firmware update opened up greater speed potential for the [...]

DJI have made many hardware and firmware improvements since then. It's incredible how fast and how far the technology has progressed.

The original Phantom had a fatal flaw which has been corrected in later models by imposing a "speed limit" on descent rates.

The original Phantom was prone to Vortex Ring State where, while descending too fast straight down, the rotor blades effectively suffered aerodynamic stall as the drone dropped through its own propwash. It could be prevented by always maintaining a little forward, backward, sideways, or even spiral motion so the drone is always moving through "clean" air. But DJI eliminated the issuse altogether by imposing a firmware limit on the maximum descent rate.

Here's a classic example (not mine!) of a Phantom's vortex ring state crash. I've posted this video before . It still cracks me up. Not because of the crash, but because of the solution they came up with to dry the drone after it was recovered from the lake.

To view this content we will need your consent to set third party cookies.
For more detailed information, see our cookies page.

Accept third party cookies

 

[Yorkshire_Pud]

One annoyance was that the original controller had no centring spring on the throttle stick.

If you happen to have a spare P3 controller knocking about could you, open both controllers and compare the layouts and design of the joystick gimbals ?
From memory the of the P3 joystick gimbals rotate the wiper on linear, centre-tapped rheostats, that looked to be of a standardise design.
If the P1 uses a similar arrangmnet might might it be possible to put P3 joystick gimbals in the P1 controller.
If the rheostats differ coild you put P1 rheostats on the P3 gimbals.

Warning, the shell of the P3 contoller seems to be very weak or brittle especially where the bosses containing screw holes are concerned. I wouldn't be surprised to find that the shell of the P1 controller was slimilar. In quite a few of the P3 controllers those bosses where cracked or fragmented. I would therefore suggest that when refitting a screw you place it in contact with the screw hole, press it in slightly and then UNDO the screw. If you pay attention to waht you feel you will feel the screw's thread drop into the female thread. When that happens try screwing the screw in. If it feels unhealthily tight back the screw out and see if it drops into another thread about 180deg different then try screwing it in. In such circumstances I often found that the screw was a 'better' fit in the 'second' thread.
Don't tight the screw using a 10ft lever lol.

 

[Zbip57]

In what thread did you post your recent 'I was staggered that the VPS system can cause tracking, video'?
I think a few people are citing it but in the wrong context. I looked for it but I can't find the thread.

My video, where I'm dragging the patterned rug under my Mini was first posted here:
mavicpilots.com/threads/yes-landing-protection-can-be-disabled-but.134162/
That video is in post#49 on page3 of that thread.

There's a lot of good stuff in that 4-pages of discussion. It's still worth reading again. But I'm now sorry to admit that much of what I wrote back then isn't correct, and I hope what I've posted so far in this thread makes up for that.

[...] it calculates groundspeed, not airspeed, by using GPS info. [Actually it calculates all that stuff near instantaneously by integration from its IMU gyro and accelerometer data, but it supplements and verifies its positional calculations with frequent polling of the GPS info.]

In that same thread from last year, I explained how the flight controller calculates position and velocity by integration of the accelerations recorded by the IMU accelerometers and gyros.

Remember, an accelerometer can ONLY measure acceleration, i.e. the rate at which velocity is increasing/decreasing. Using calculus, the flight controller can calculate backwards to determine what the velocity now is, and integrating yet another step backwards it can figure out how far its position has moved because of those changes in acceleration.

I explained in greater detail how that works starting from post #38 to #42 in that same thread from last year. Read that, it's pretty good.

But for that type of dead-reckoning calculation to maintain any accuracy, it needs to know what the initial velocity and initial position was before that acceleration was applied. Any small roundoff error rapidly reduces the accuracy of the calculations. So an additional sensor is required and polled at intervals to confirm and recalibrate the flight controller with the currently known position and speed. That comes from either the GPS sensor, or the VPS optical sensor.

 

[Droning on and on...]

The Flight Controller controls the speed of the drone based on ground speed, derived from GPS. This is the speed displayed in the lower left of Fly. There is no sensor for air speed.

The ground speed upwind and downwind will be the same with the same proportional stick deflection. Upwind it will fly with a greater forward pitch than downwind. I have log data from a test flight for just this purpose if I need to "prove" this.

However I strongly encourage everyone to prove it to themselves, and not take my word for it.

 

[Yorkshire_Pud]

My video, where I'm dragging the patterned rug under my Mini was first posted here

great thanks.

 

[Droning on and on...]

VPS ranging sensors, in my experimentation taping them over, seem to be used for autolanding, and Landing Protection. While the measured height is displayed when in range, it doesn't seem to affect regular flight in any way, unless within a few feet of the ground.

 

[Zbip57]

I am of the opinion that the IR sensor will, normally, reduce the speed of a fast descent once the ground is detected.

Just a wild guess, but I suspect that inputs from the IR detector regulate the rate of descent when landing [...]

Yes, definitely in both cases, the descent rate is slowed during landing. But at what point during descent from greater heights?

It's agreed that above a certain height, once the VPS system is out of range or has lost effectiveness, height is measured solely by the barometric altimeter and horizontal position is determined by GPS. [Some people have suggested that height is also measured by the GPS sensor, but that's nonsensical.]

It probably varies depending on all the different DJI models, but the manual for my Mini says the VPS operating range is 0.5-30m, but "works best" between 0.5-10m. We can assume (given sufficient lighting etc), that means between 0.5-10m the VPS is always given priority over GPS, even if they're showing conflicting data, [as is demonstrated in my video pulling the patterned rug. I'll attach it again below, in case anyone hasn't seen it yet.]

And somewhere within the height range between 10m-30m (depending of surface quality and lighting) confidence in the VPS degrades to the point where the flight controller switches instead to relying on GPS instead.

The infrared sensors, as per the theory in my first six posts in this thread, measure height from the surface in order for the flight controller to be able to calculate the area size of each pixel in the image captured by the optical sensor.

Beyond that, the drone's hover altitude, or transit altitude, varies in response to changes in terrain height detected by the infrared sensors only when within 0.5m (~2ft).

Does the descent rate from higher altitudes vary depending on when the VPS infrared sensors first detect ground (<30m)? I think not. I still believe the barometric sensor is used to measure and control changes in height. But that should be easy enough to test. Unlike the DJI Fly app, the Litchi app permits enabling/disabling the VPS on my Mini [as shown in the video below]. I might even go out and try that today...

Certainly the infrared height sensors play a role when auto-landing below 0.5m height. At 5:50 in this video, with the VPS disabled, I as usual landed the Mini on my hand. But when I pulled the throttle down I shouldn't have been so surprised that it dropped like a stone instead of doing its usual soft landing.

To view this content we will need your consent to set third party cookies.
For more detailed information, see our cookies page.

Accept third party cookies

 

[Yorkshire_Pud]

. But at what point during descent from greater heights?

I've never taken notice of numerical value but a qualitative recollection is "quite high", in fact I seem to vaguely remember being surprised how high it looked ......,....,but it was a fair while ago.

 

[Zbip57]

The Flight Controller controls the speed of the drone based on ground speed, derived from GPS. This is the speed displayed in the lower left of Fly. There is no sensor for air speed.

The flight controller will use either GPS or VPS for horizontal position hold when in hands-free hover and for calculating ground speed when the drone is responding to active stick inputs.

The problem arises when that speed is being referenced to different things when measured by GPS [actual groundspeed] versus when measured by VPS.

It should be the same thing [groundspeed] in both cases as long as the optical sensor is measuring off fixed terrain features. But the calculated can be entirely different if the VPS optical sensor is tracking and following a feature that is actually itself moving.

The Mini 3 Pro in the Chinese river gorge incident thought it was standing still [VPS zero groundspeed] when in effect it was flying backwards at 10m/s top speed [GPS groundspeed] as the VPS optical sensor tracked the fast flowing river current. Then, given full forward stick input, the drone moved at full 10m/s speed lmit [VPS groundspeed relative to the moving current] while in effect it was stationary [zero GPS groundspeed relative to the rocks]. Bizarre, huh?

It would be nice if DJI again provided a switch on the controller so we could toggle directly to ATTI-mode. Being able to instantly switch off both GPS and VPS would cure most cases of "flyaway", which are likely almost always caused by a misunderstanding of how the flight controller is programmed to react.

I strongly encourage everyone to prove it to themselves, and not take my word for it.

Absolutely, yes! If anyone is unsure of how their drone actually works, set up an experiment in a safe environment to test out your ideas. Share the results so that others can repeat the experiment for themselves.

 

[Zbip57]

I've never taken notice of numerical value but a qualitative recollection is "quite high", in fact I seem to vaguely remember being surprised how high it looked ......,....,but it was a fair while ago.

I need to charge batteries and wait for it stop raining, but I'm itching to test this.

For now, as far as I understand it, the primary use for the VPS height sensor measurement (except within 0.5m height, or in auto-landing) is for calculating the size of the surface area represented by each pixel of the VPS optical image sensor.

If anyone sees any obvious holes in that theory, fire away! And certainly, if anyone can devise a foolproof experiment to prove or disprove, please share.

 

[Droning on and on...]

The flight controller will use either GPS or VPS for horizontal position hold when in hands-free hover and for calculating ground speed when the drone is responding to active stick inputs.

No argument re: holding position. I chimed in because of the speed vs. forward pitch discussion.

What is your evidence, or reference that indicates VPS optical flow is being used to calculate ground speed, and that it used in priority over GPS when available?