Auto exposure gives better highlights and shadow details in high ISO than manual at lower ISO

[fnf]

In recent history, DJI's drones have always been favouring high ISO and high shutter speed rather than sticking to as low ISO as possible and lowering the shutter speed to compensate. I'd always found that puzzling.

The recommendation that most people give is to disregard the drone's poor auto-exposure algorithm and go with manual exposure instead, such as in this thread Why is it bumping base ISO100 even at shutter speed of 1/3000? .

As I found out, there is a very good reason as to why the drone does that: highlight recovery. In essence, DJI's exposure strategy usually tries to preserve at least 2-3 stops of highlights then bumps up the ISO to render a more balanced exposure. The result is a slightly noisier image but with less highlights being blown out. As a result, videos taken at higher ISO can be of better quality than lower ISO.

Here are screen grabs from a video taken with Auto ISO/SS at ISO 1600 and 1/500s (Pro mode):


and here's the exact same scene with manual ISO 240 and 1/60s shutter speed:


Notice how both highlight and shadow details are gone from the manual exposure one.

The loss of highlight details were due to the reason above, but the loss of shadow details were likely caused by the noise reduction algorithm (I use Sharpness -2, Noise Reduction -1) and the extra noise helped avoiding it.

I do not want to use Noise Reduction -2 since it causes unnecessary loss of details in general due to bitrate budget. NR -1 gives the best compromise for me in terms of detail retention.

So, as surprising as it is, ISO 1600's picture quality was a lot better than ISO 240. Many recent DJI sensors also have dual native ISO, so high ISO could still look very clean and is an extra reason why we should use Auto exposure with EV to compensate.

I have only tested the Air 3S, Mini 4 Pro, Avata 2 and Neo thus far, but all of them provide better picture quality with Auto ISO/Shutter speed enabled. The best exposure strategy would be ETTR with Auto ISO/SS, rather than manual exposure like with other cameras.

 

[brett8883]

I think you have to figure out if this is an ISO thing or an auto exposure thing before we jump to conclusions on how to solve it. The test for that would be to take two examples, one set with auto exposure and then the second set to the same settings but manually and see if you get the same differences.

-If same difference, then auto exposure is applying some other processing we don't know about.

-If not, then it has something to do with the ISO value possibly the a dual gain or even more possibly it has something to do with the quad bayer sensor which is always the wild card on these drones.

 

[fnf]

I think you have to figure out if this is an ISO thing or an auto exposure thing before we jump to conclusions on how to solve it. The test for that would be to take two examples, one set with auto exposure and then the second set to the same settings but manually and see if you get the same differences.

-If same difference, then auto exposure is applying some other processing we don't know about.

-If not, then it has something to do with the ISO value possibly the a dual gain or even more possibly it has something to do with the quad bayer sensor which is always the wild card on these drones.

Yes, Auto ISO mode applies some HDR-style post-processing to the video feed which isn't done if the same ISO/shutter speed are set manually. It's easy to see for yourself by toggling it. You should observe an immediate difference in the highlights rendered. This is a known fact about the Neo (Avata 2/Mini 4 Pro do behave slightly differently but Auto exposure mode still give better results in different ways).

This post-processing should produce higher-quality video since the drone would be working with raw data. Doing so in post won't be as good due to compression artifacts. Noise will be present regardless if shadows are lifted in post, but if we shoot in low ISO to begin with to preserve highlights, there might be no shadow details to recover due to compression.

My post was focused more on seeing whether it's worth accepting the higher noise in the shadows in exchange for better highlight details. Interestingly enough, the shadows are better in this case too so it's a win all around.

 

[Bsnyder]

To view this content we will need your consent to set third party cookies.
For more detailed information, see our cookies page.

Accept third party cookies

 

[fnf]

To view this content we will need your consent to set third party cookies.
For more detailed information, see our cookies page.

Accept third party cookies

Any particular point in the video you'd want to refer to?.

Edit: found it (~7:20), this aligns with my observation, thanks.

Many modern cameras (particularly phones) employ the same trick. They seem to be based on/inspired by the Dual Conversion Gain technology from OmniVision.

 

[MavicFlyer2]

Yes, Auto ISO mode applies some HDR-style post-processing to the video feed which isn't done if the same ISO/shutter speed are set manually.

So with the Mavic 2 Pro you'd recommend using Auto ISO while using manual settings ?

 

[fnf]

So with the Mavic 2 Pro you'd recommend using Auto ISO while using manual settings ?

I don't have the Mavic 2 Pro. The exposure behaviours we see only seem to start happening since around the Mini 3 Pro.

To confirm, you only need to watch the highlights when you toggle Auto ISO. On recent drones the recovered highlights are usually obvious.

On the Air 3S, there is also a dynamic range advantage when enabling Auto shutter speed as well, that's why I enable them both.

 

[TalesFromOurPocket]

I don't want my exposure changing in auto exposure mode mid shot. Thus I always record video in manual exposure. The rest is academic to me.

 

[fnf]

I don't want my exposure changing in auto exposure mode mid shot. Thus I always record video in manual exposure. The rest is academic to me.

The point is to use Auto exposure but lock it so that exposure won't change mid-shot.

 

[TalesFromOurPocket]

The point is to use Auto exposure but lock it so that exposure won't change mid-shot.

I didn't know that was a thing (mini 2). I'll look into it. Thank you.

 

[Split-Infinitive]

The point is to use Auto exposure but lock it so that exposure won't change mid-shot.

So the DJI logic handling, even in auto exposure and you lock it, it doesnt "default" to manual for that time?

Because as above, having exposure changing mid shot is a nightmare to fix quite often hence using manual. But if an auto with EV an locked isnt internally doing manual then it'll work nicely.

 

[fnf]

So the DJI logic handling, even in auto exposure and you lock it, it doesnt "default" to manual for that time?

Because as above, having exposure changing mid shot is a nightmare to fix quite often hence using manual. But if an auto with EV an locked isnt internally doing manual then it'll work nicely.

You can think of this as the drone doing exposure balancing right in camera, whilst preserving the overall exposure value. Locked auto-exposure still allows that balancing to happen.
There is an immediate change in the scene's DR when this is enabled, but other than that the exposure is static.

 

[Split-Infinitive]

Fair enough. I guess the only tradeoff is does the increased noise mean noise reduction in post is now needed on all clips or not.
Slight increase is tolerable but if i need to shove every single clip through Neat Video then maybe now.

 

[fnf]

Fair enough. I guess the only tradeoff is does the increased noise mean noise reduction in post is now needed on all clips or not.
Slight increase is tolerable but if i need to shove every single clip through Neat Video then maybe now.

There're no tradeoffs if a balanced exposure is what you're looking for. The camera will always do a better job recovering details before compression kicks in, compared to doing it in post. If you're recording raw videos then this doesn't apply.

 

[Split-Infinitive]

There're no tradeoffs if a balanced exposure is what you're looking for. The camera will always do a better job recovering details before compression kicks in, compared to doing it in post. If you're recording raw videos then this doesn't apply.

From my understanding is the tradeoff is it's shooting at a higher iso which is increased noise.
I haven't got a 3s yet (tomorrow) but all my previous DJIs needed severe NR on any iso over 400.
If that is correct then the tradeoff is highlight recovery vs noise.

 

[fnf]

From my understanding is the tradeoff is it's shooting at a higher iso which is increased noise.
I haven't got a 3s yet (tomorrow) but all my previous DJIs needed severe NR on any iso over 400.
If that is correct then the tradeoff is highlight recovery vs noise.

No, there's no noise issue if a balanced exposure is what you're aiming for, as I mentioned previously. You'll need to do so in post, starting from a lower-quality footage and raise the shadows which will bring noise, or drop the highlights that might or might not have blown out already.

On the other hand, if you do not want a balanced exposure i.e., you want crushed shadows or blown highlights for artistic reasons, then it's always possible to drop shadows or lift highlights further in post with no noise penalty.

A higher-DR footage is always a good thing, it's the premise for log profiles (which will also make noise in the shadows more visible) and auto-exposure mode improves the DR further.

 

[Split-Infinitive]

No, there's no noise issue if a balanced exposure is what you're aiming for, as I mentioned previously. You'll need to do so in post, starting from a lower-quality footage and raise the shadows which will bring noise, or drop the highlights that might or might not have blown out already.

On the other hand, if you do not want a balanced exposure i.e., you want crushed shadows or blown highlights for artistic reasons, then it's always possible to drop shadows or lift highlights further in post with no noise penalty.

A higher-DR footage is always a good thing, it's the premise for log profiles (which will also make noise in the shadows more visible) and auto-exposure mode improves the DR further.

That isn't what I meant as such.

If it's increasing the iso to the extent it's inducing a lot more noise which would require more post processing to remove then there's a tradeoff between DR and noise.
I don't go above 200 on my mini 3 for that reason.

 

[fnf]

That isn't what I meant as such.

If it's increasing the iso to the extent it's inducing a lot more noise which would require more post processing to remove then there's a tradeoff between DR and noise.
I don't go above 200 on my mini 3 for that reason.

If you need to lift the shadows to balance the exposure, then more noise will become visible. We can't improve the physical DR with software, it's a hardware limitation.
What could be improved, however, is the way the footage is recorded to allow for better latitude in post-processing. Footage from the same camera taken in raw format will have a vastly better DR compared to a basic 8-bit H.264 format. D Log-M/HLG plus auto-exposures shifts the footage's quality a few steps closer to that of raw video's.

A fair comparison to this is RAW vs. JPEG: lifting shadows by e.g., 2 stops from a RAW photo gives a much different result compared to doing the same for the JPEG version.

 

[Split-Infinitive]

Fully aware how all that works (been my job for 20 years, day job involves CLOG/Log and so on.)

Not every scene requires the absolute maximum dynamic range. Many scenes do nor and in that case its a decision on acceptable noise levels.

For scenes that DO require it, its still a tradeoff between losing highlight/shadow detail or image detail via noise.
Theres a point where the noise level damages the image more than compressed range.

On the Air3S it might be fine on the lower ISOs - I've literally just bought it and can't fly until im in a country where its legal in a week or 2. However on my previous drones (Mavic 1, 2 pro, mini 3 pro) the ISO noise was so bad they were unusable above 400 so that was your limiting factor.

The Air3s may well be better and run lower noise in those ranges which means running a higher ISO to get the better range is worth it.


Currently im in Japan where i cant fly without a train ride then a week in Malaysia who effectively banned all flights from March so i cant actually play yet.
The findings here are useful and give me something to play with for sure - ill create a proper DNG profile for use in LR/PS from the Colourchecker to get accurate colours (guess it still has DJI nuclear greens maybe).

 

[fnf]

Not every scene requires the absolute maximum dynamic range. Many scenes do nor and in that case its a decision on acceptable noise levels.

For scenes that DO require it, its still a tradeoff between losing highlight/shadow detail or image detail via noise.
Theres a point where the noise level damages the image more than compressed range.

Exactly, my topic explores the highlight and shadow details retention between the 2 exposure strategies, that defies the common best practices. It does not discuss noise levels as it's a given that higher ISO will make noise more visible.

We can always drop shadows to reduce noise if those shadow details don't need to be seen, but it's not always possible to lift those shadows backup, if they were already destroyed due to compression.

On the Air3S it might be fine on the lower ISOs - I've literally just bought it and can't fly until im in a country where its legal in a week or 2. However on my previous drones (Mavic 1, 2 pro, mini 3 pro) the ISO noise was so bad they were unusable above 400 so that was your limiting factor.

The Air3s may well be better and run lower noise in those ranges which means running a higher ISO to get the better range is worth it.

Actually, all recent DJI drones will benefit from this even with the puny Neo. It's true that the Air 3S gives better quality at the same ISO than the M4P/Neo, but these 3 models all give improved quality in auto exposure.

Auto exposure does not prevent us from over-exposing to force capturing those shadow details better, if noise is a bigger concern than lost highlight details.

Currently im in Japan where i cant fly without a train ride then a week in Malaysia who effectively banned all flights from March so i cant actually play yet.
The findings here are useful and give me something to play with for sure - ill create a proper DNG profile for use in LR/PS from the Colourchecker to get accurate colours (guess it still has DJI nuclear greens maybe).

I was in Japan last autumn. Flying drones there was.. difficult and was basically frowned upon unless you're in the middle of nowhere, even if the law allows for it there were plenty of bylaws regarding the use of drone.