Image Quality Deep Dive - Mavic Pro vs. Air

Cliff_622 · Feb 9, 2018

So I'm putting together and comprehensive side by side test video of the Mavic Pro and Mavic Air's video cameras.

This is going to be a pretty heavy duty analysis of the two in different situations and there will be 5 tests between the two with ALLOT of zooms and crops and waveform scopes with commentary. I am already half way through the production of this and have found some pretty surprising findings....

One major issue is the Ambarella A9-A1 image processor and it's noise reduction algorithm. I have found that it doesn't just apply noise reduction "evenly" across the tonal spectrum. It literally dumps heavier and heavier noise reduction as the tones move down middle grey. This means the darker the regions of an image get, the heavier the noise reduction is applied. Looking at scopes, I have seen that it applies an almost "infinite" cancellation of noise down to a solid flat black line that goes under 0 IRE! The further something goes deeper into shadows, the worse the algorithm destroys it. I have never seen this before on ANY camera.

Anyway, this is an EXCLUSIVE "sneak peak" of one of my tests that are in the full video to be released over the weekend. This ONE test was pretty eye opening for me and the others that came after were also equally surprising.

WARNING!...this entire video is incredibly NERDY and might be too much for some flyers to watch. However, I do think the serious "video geeks" will like it allot.

Here is Part 1 of the full test video

Wayno1 · Feb 9, 2018

interesting tests. Would a software update for the camera on Mavic Pro be able to fix this?

Cliff_622 · Feb 9, 2018

I dont think so. This Ambarella A9-A1 is an old chip now. It first appeared on the GoPro Hero 3. I don't "think" many of it's setting are actually changeable or adjustable. If so, I'm sure DJI would have done this a long time ago.

Floater · Feb 9, 2018

Would the difference in the focal lengths account for the brightness difference?
Would setting the image sharpness to a higher setting help pull the noise reduction back?
Interesting observations. Thanks for putting this together .

Cliff_622 · Feb 9, 2018

Floater said:
Would the difference in the focal lengths account for the brightness difference?
Would setting the image sharpness to a higher setting help pull the noise reduction back?
Interesting observations. Thanks for putting this together .

I will have the answer to those questions! I made multiple setting test for these tests. I ran them all up though the shutter speed ladder and did tests at 0 sharpening and +1 sharpening.

(Not to be "MEAN" or "CRUEL" to the Mavics....but I had the Phantom 4 Pro on each of the scenes too...My God, what a DIFFERENCE that BEASTLY 1inch-type sensor is!...I mean WOAH!)

What I have found so far is that the Mavic Pro is "OK" with noise reduction on most "bright" parts of your image. As those "bright" areas get lower and lower to the shadows, the noise reduction gets stronger and stronger. So, a tree in the sun is OK, but the grass or bushes in that tree's "shadow" get slammed HARD with noise reduction. It's weighted sooo HEAVILY on anything below middle gray....and it just gets stronger and stronger the darker any image tone is. I dont know why anybody at Ambarella decided that this was a good idea all those years ago.

The Mavic Air has a more "traditional" noise reduction. It seems to apply an EVEN and BALANCED amount of noise cancellation across the entire highlight to shadow range.

All of this will be shown with more samples of outdoor scenes and scopes to look at. The way I did the indoor tests was that I framed the Air first and ran through it's paces. Then, I took the Mavic Pro and positioned it differently so that it's frame was centered and aligned with the Air's frame width. (I used red markers on the boxes to indicate the approximate field of view)

The Mavic Pro is "supposed" to have a faster and brighter lens. As far as sharpness goes, both are sharp enough to resolve what needs to be resolved in highlights. This problem is a "highlight/shadow TONAL" issue, not a lens thing.

As far as the brightness difference? Everything was manually set and I double checked and triple checked multiple times. It really must boil down to the image processor again. Both processors must take in the same raw sensor readings and the both interpret them differently. This even happens in the pro camcorder world. 100 ISO will not be exactly the same on Canon, Sony or Panasonic when you do a side by side. The Mavic Air's processor is adding more brightness to that raw sensor data that it's getting. I didnt have a light meter with me but I'm guessing that the Mavic Air's ISO settings display will be the closer match to a real light meter.

I can CERTAINLY say that both of these image processors are VERY different in what they do.

For a long time now, we have all blamed the Mavic's very high 60mbp/s h.264 compression. Yes, this is a problem but the "true" problem was the A9-A1 noise reduction algorithm and image processor all along. The h.264 encoder is already "getting" the image that way before it even starts its job.

These lab tests became more fascinating to me as I got deeper into them!

CT

Manna · Feb 10, 2018

@Cliff,

awesome analysis you have done!.

What is not obvious to me however is if you used stills for the comparison or frames from video footage.

The reason for asking is I have come across a comparison from a german test, where they concluded JPG stills are far better from the air, while videos from the pro seems to hold more crisp details, please have a look at post #37 in this thread:

~Poll~ Mavic Air or Mavic Pro - Which one will YOU choose?

Really would appreciate your findings.

Cheers

Cliff_622 · Feb 10, 2018

Manna said:
@Cliff,

awesome analysis you have done!.

What is not obvious to me however is if you used stills for the comparison or frames from video footage.

The reason for asking is I have come across a comparison from a german test, where they concluded JPG stills are far better from the air, while videos from the pro seems to hold more crisp details, please have a look at post #37 in this thread:

~Poll~ Mavic Air or Mavic Pro - Which one will YOU choose?

Really would appreciate your findings.

Cheers

Sorry,...I only did several video tests. (Outdoor and indoor) none of them involved still photos. The Ambarella processor in the Mavic Pro "could" handle photography very differently....I just dont know.

I think there is a possibility that the Mavic Pro "could" handle infinity focus detail better than the Air does. Im still hashing this out on the full test video release.

Cliff_622 · Feb 15, 2018

Here is Part 1 of the full test video.

Kilrah · Feb 15, 2018

Cliff_622 said:
but the "true" problem was the A9-A1 noise reduction algorithm and image processor all along

Been saying that since day one but everybody always preferred blaming the bitrate even when you can see plenty of cameras with the same bitrate and none of these problems. Oh well.

Cliff_622 · Feb 15, 2018

Yeah, the bitrate compounds the problem. I would argue that the image processor is literally presenting this image to the h.264 CODEC just like this. It's already 95% this way before he h.264 encoder even starts it's job.

Manna · Feb 16, 2018

Cliff_622 said:
Here is Part 1 of the full test video.

To view this content we will need your consent to set third party cookies.
For more detailed information, see our cookies page.

Again, amazing work!
I really admire people doing so much research.

To me it again appears, even if the processing used within the air is milder, if some sort of sharpness/crispiness/punchiness is missing from the footage, it simply appears "soft", might this also be a lens / soft focus issue (crane / water surface comparison)?
The pro seems to have punchier footage, however looking closer it might only be due to a lot of oversharpening (even at "0") as tons of artifacts seem to be present, but also the video appears undeniably "more" in focus.

"Part 1" in the title implies there will also be a part 2?

Will you also test the infamous sharpening +1 setting?

Best regards

Cliff_622 · Feb 16, 2018

Yes,...there is more. I did tests at +1 sharpness too. I also did a complete dark (pitch black) room test just to see what the noise floor looked like on both. This difference was stunning. The Mavic Pro literally cancels out tones that get near black completely. When I say completely,...I mean it blanks out everything and spits out a solid black line on the waveform scopes. This seems to happen from a dark grey tone and lower. I have never in my life seen any camera actually do this. Ambarella was so intent on scrubbing noise that they increase the noise reduction HEAVILLY and progressively as tones get further and further towards black. Very odd.

I also added the P4P to the test too, which is so far superior that I felt guilty doing this.

J3ffrey · Feb 18, 2018

So how do we fix this.. or if we are filming in bright sunny conditions, do we really need to fix it?

Cliff_622 · Feb 18, 2018

There is no real "fix" really. You can add +1 sharpness and help "some" of the noise reduction clamp down but then you will face overly sharpened image that you might want to soften in post anyway.

I mean,...it is what it is. The Mavic Pro is a great drone that can give you nice results. A lot of people have shot some beautiful stuff with it. Image quality is not the "only" thing that's important when judging a drone's value too.

I think all Mavic Pro owners should be happy with what they have. There will always be "the next best thing" with ALL drones that we own. It's a never ending battle of "keeping up with the Jones's"

Image quality is a very subjective thing. Not everybody has an interest in it. Or at least, everybody sees it differently. For example, I have met MANY people that are perfectly happy with the Spark's 1080 and have zero interest in any 4k at all.

As far as what you can do? Just little things like keep your exposure as bright as you can without over-exposing. If it really bothers someone, then I'd wait until next month when we might get the Mavic Pro-II announcement and go from there.

CT