This is a thread for those inquisitive minds who like to speculate on "how things work". For more practical people this will probably be an extremely boring reading.
Until recently I didn't pay much attention to collision sensors. I took them for granted, simply registering in my mind whether they were present or absent on certain sides on a certain drone model and thus "protecting" the drone from these same sides. For a long time I didn't take this "protection" seriously because their action was more annoying than helpful in the majority of real-life situations. For instance, when I was flying the Inspire 2 they simply didn't allow the drone to maneuver in tight spaces and often prevented landings for no obvious reasons. I made a habit of switching them off for most of the time.
When Skydio 2 was released I was impressed at how well the collision avoidance system worked on that drone. But then I realized Skydio 2 had an insane amount of computing power at its disposal, using dedicated Nvidia chip to analyze streams from six hi-res cameras in real time just for the purpose of collision prevention. It is when MA2 was released I was simply stunned at how well it used its meager sensor set for the same purpose. Of course it is not on the same level as Skydio 2, but it is much closer than I would ever expect (especially in forward flight), achieving similar result with presumably much inferior hardware. I don't know the exact specs of MA2 sensors, but I don't think they are, hardware-wise, much better than the ones used on older DJI drones. I believe it is the improved analysis of their data that makes all the difference.
Now that collision avoidance technology is maturing to the point of being taken seriously going forward, I feel the urge to understand how it actually works. The materials I found on the subject online are scarce and contradicting. So I propose that we look at the sensors of MA2 (or any other Mavic except Mini) and try to deduce their principles of operation. Here is my attempt.
At the front (as well as on the back and bottom), there are two cameras, separated by approximately same distance from each other. Each pair is arranged at the same plane. These two clues lead me to think that the parallax effect might be employed for detection. When the images from these two cameras are superimposed, the difference between them can be used to determine distance to objects. The more difference, the closer the object. This is how all animals with binocular vision (owls, humans) determine distance -- without even thinking about it. Of course the main problem and probably the most computationally intensive task here is shape recognition, as objects can appear very different when viewed from different angles. For instance, when getting dangerously close to a tree branch there may not be enough absolute difference between the images as distant background fills most of the frame; however if the shape of the branch is recognized and determined to be at sufficiently different position in two images, the collision can be prevented.
The second method for detection I can think of doesn't require two cameras looking at the same object, but it does require motion of the drone to be present, and that motion must be precisely known. When a drone is moving towards an obstacle, the shape/outline of that obstacle is starting to grow. Knowing the speed and direction of the drone and the rate at which the shape of the obstacle grows, we can determine the distance to that obstacle and engage avoidance action when it is too close. In my opinion, the list of important advantages of this method include:
1. No requirement for two cameras arranged in a specific way.
2. Opportunity to utilize the existing MPEG encoding hardware. Coincidentally, when working on a video stream, the encoder does (among other things) precisely what is needed for this type of collision detection: determines shapes of objects and their scaling rates. If this data is fused with speed/orientation data from GNSS/VPS and IMU we can get obstacle detection with relatively little processing overhead. I believe MA2 uses its main video processor (not a dedicated hardware) to achieve this: the main clue is in the fact that collision detection does not work in 4K60P/H.265 mode, when the video processing pipeline is probably taxed to its limits.
The main disadvantages of this method, in my opinion, are:
1. Less accuracy than the parallax-based method, mostly because the speed of the drone cannot be measured very precisely.
2. Should only work reliably with stationary objects. The fact that such detection method only works when the drone itself is moving is not a big disadvantage (when else do we need it?). It's when an object attempts to collide with the drone we're in trouble. I imagine the system may simply go haywire.
Let's take a closer look at the sensors placements. I think the most "protected" side of a drone is it's front. We have two cameras which can use either of the two hypothetical methods described above, plus the main camera -- which can also be used as sensor if the drone has processing power to spare. On the rear, we also have two sensors, which are arranged vertically. That arrangement should not make much difference as far as collision detection is concerned. The reason why they are arranged vertically, I think, has to do with the fact that rear arms protrude much further towards the back of the drone than front arms towards the front. Obviously we don't want props in plain view of the sensors, as their movements may cause confusion. On the bottom, there are also two cameras. Now things get a little more complicated, as we also have an IR-pair there. Why would we need IR when we have two cameras? My hypothesis is we need them because the bottom cameras double as VPS sensors, and when working as VPS sensors, cannot be used for collision detection (that's when IR takes over). Or they work only as VPS sensors despite the appearance (which is doubtful because only one camera is needed for VPS). I am very curious what, apart from altitude, triggers them to switch modes of operation, because such info may help to prevent certain types of crashes.
There are no more sensors on MA2, but there are some extra on its bigger brother, M2. Unfortunately, I don't own M2 myself so I can't test my "theories". I suspect that upward collision avoidance is basically non-existent on M2, because there's only IR pair and no cameras. IR can detect "something" over the drone at certain distance but cannot provide any guidance on how to avoid it. I suspect in such cases M2 would just stop, hover and wait for user input. I also suspect IR would be very unreliable in detecting wires and thin branches over the drone. Please correct me if I am wrong. Side sensors are more interesting. They are cameras, but they are not paired cameras, and therefore cannot use parallax-based detection. They are positioned somewhat awkwardly in full view of rear props. I would expect them to be much inferior in detecting obstacles than either front, rear, or bottom sensors, almost on par with IR pair. Again, correct me if I am wrong. Of course, something is better than nothing, but it would be so great to have a Mavic that had equally robust collision protection from all sides.
What are your thoughts on modern advancements in this field? Do you know/suspect exactly how the sensors work? If you have any insights, please share.
Until recently I didn't pay much attention to collision sensors. I took them for granted, simply registering in my mind whether they were present or absent on certain sides on a certain drone model and thus "protecting" the drone from these same sides. For a long time I didn't take this "protection" seriously because their action was more annoying than helpful in the majority of real-life situations. For instance, when I was flying the Inspire 2 they simply didn't allow the drone to maneuver in tight spaces and often prevented landings for no obvious reasons. I made a habit of switching them off for most of the time.
When Skydio 2 was released I was impressed at how well the collision avoidance system worked on that drone. But then I realized Skydio 2 had an insane amount of computing power at its disposal, using dedicated Nvidia chip to analyze streams from six hi-res cameras in real time just for the purpose of collision prevention. It is when MA2 was released I was simply stunned at how well it used its meager sensor set for the same purpose. Of course it is not on the same level as Skydio 2, but it is much closer than I would ever expect (especially in forward flight), achieving similar result with presumably much inferior hardware. I don't know the exact specs of MA2 sensors, but I don't think they are, hardware-wise, much better than the ones used on older DJI drones. I believe it is the improved analysis of their data that makes all the difference.
Now that collision avoidance technology is maturing to the point of being taken seriously going forward, I feel the urge to understand how it actually works. The materials I found on the subject online are scarce and contradicting. So I propose that we look at the sensors of MA2 (or any other Mavic except Mini) and try to deduce their principles of operation. Here is my attempt.
At the front (as well as on the back and bottom), there are two cameras, separated by approximately same distance from each other. Each pair is arranged at the same plane. These two clues lead me to think that the parallax effect might be employed for detection. When the images from these two cameras are superimposed, the difference between them can be used to determine distance to objects. The more difference, the closer the object. This is how all animals with binocular vision (owls, humans) determine distance -- without even thinking about it. Of course the main problem and probably the most computationally intensive task here is shape recognition, as objects can appear very different when viewed from different angles. For instance, when getting dangerously close to a tree branch there may not be enough absolute difference between the images as distant background fills most of the frame; however if the shape of the branch is recognized and determined to be at sufficiently different position in two images, the collision can be prevented.
The second method for detection I can think of doesn't require two cameras looking at the same object, but it does require motion of the drone to be present, and that motion must be precisely known. When a drone is moving towards an obstacle, the shape/outline of that obstacle is starting to grow. Knowing the speed and direction of the drone and the rate at which the shape of the obstacle grows, we can determine the distance to that obstacle and engage avoidance action when it is too close. In my opinion, the list of important advantages of this method include:
1. No requirement for two cameras arranged in a specific way.
2. Opportunity to utilize the existing MPEG encoding hardware. Coincidentally, when working on a video stream, the encoder does (among other things) precisely what is needed for this type of collision detection: determines shapes of objects and their scaling rates. If this data is fused with speed/orientation data from GNSS/VPS and IMU we can get obstacle detection with relatively little processing overhead. I believe MA2 uses its main video processor (not a dedicated hardware) to achieve this: the main clue is in the fact that collision detection does not work in 4K60P/H.265 mode, when the video processing pipeline is probably taxed to its limits.
The main disadvantages of this method, in my opinion, are:
1. Less accuracy than the parallax-based method, mostly because the speed of the drone cannot be measured very precisely.
2. Should only work reliably with stationary objects. The fact that such detection method only works when the drone itself is moving is not a big disadvantage (when else do we need it?). It's when an object attempts to collide with the drone we're in trouble. I imagine the system may simply go haywire.
Let's take a closer look at the sensors placements. I think the most "protected" side of a drone is it's front. We have two cameras which can use either of the two hypothetical methods described above, plus the main camera -- which can also be used as sensor if the drone has processing power to spare. On the rear, we also have two sensors, which are arranged vertically. That arrangement should not make much difference as far as collision detection is concerned. The reason why they are arranged vertically, I think, has to do with the fact that rear arms protrude much further towards the back of the drone than front arms towards the front. Obviously we don't want props in plain view of the sensors, as their movements may cause confusion. On the bottom, there are also two cameras. Now things get a little more complicated, as we also have an IR-pair there. Why would we need IR when we have two cameras? My hypothesis is we need them because the bottom cameras double as VPS sensors, and when working as VPS sensors, cannot be used for collision detection (that's when IR takes over). Or they work only as VPS sensors despite the appearance (which is doubtful because only one camera is needed for VPS). I am very curious what, apart from altitude, triggers them to switch modes of operation, because such info may help to prevent certain types of crashes.
There are no more sensors on MA2, but there are some extra on its bigger brother, M2. Unfortunately, I don't own M2 myself so I can't test my "theories". I suspect that upward collision avoidance is basically non-existent on M2, because there's only IR pair and no cameras. IR can detect "something" over the drone at certain distance but cannot provide any guidance on how to avoid it. I suspect in such cases M2 would just stop, hover and wait for user input. I also suspect IR would be very unreliable in detecting wires and thin branches over the drone. Please correct me if I am wrong. Side sensors are more interesting. They are cameras, but they are not paired cameras, and therefore cannot use parallax-based detection. They are positioned somewhat awkwardly in full view of rear props. I would expect them to be much inferior in detecting obstacles than either front, rear, or bottom sensors, almost on par with IR pair. Again, correct me if I am wrong. Of course, something is better than nothing, but it would be so great to have a Mavic that had equally robust collision protection from all sides.
What are your thoughts on modern advancements in this field? Do you know/suspect exactly how the sensors work? If you have any insights, please share.
