Exposure Discrepancy - DNG Still Images

[Citizen Flier]

I read the OP again and I certainly misunderstood. That’s my bad.

On the issue of luminance, the camera records brightness on a linear scale but we perceive brightness on a logarithmic scale. Yes the camera records differences in brightness obviously but not the same way our eyes perceive it. It’s up to the raw file reader to convert the data from the sensor to brightness levels as we perceive it or the artist wants the photo to be perceived.

If we just took the the middle value from the camera sensor and used that as the gamma it would be all messed up. For example if the camera has 5 brightness levels it can record 1 2 3 4 5 and we made 3 the gamma value the photo would be all messed up because 2 would be appear twice as bright as 1 but 3 would only be 50% brighter than 2 and 5 would only appear 25% brighter than 4. The raw file converter must apply a tone curve to make it so that the luminosity is evenly distributed throughout the photo. This tone curve or luminosity curve is not contained within the raw data and will vary between raw photo processors and even with different profiles within the raw file reader.

Sure DNGs can come with embedded profiles which contain a LUT to try and produce the same tone curve used by the camera but this is a side car file separate from the raw data from the camera sensor which is the “RAW” file.

"this is a side car file separate from the raw data from the camera sensor which is the “RAW” file."

I suggested in an earlier response that the Sidecar file is perhaps absent or incorrect. I could be wrong of course, but I believe it is the sidecar file that can preserve adjustments in the raw reader. If my client adjusts settings on a raw file, and includes the sidecar, when I open the file in raw reader the client settings are present. Similarly, when I snap a frame on my Canon 7D, the raw file it typically fairly close to what I viewed on screen at capture.

 

[Deleted member 114292]

"this is a side car file separate from the raw data from the camera sensor which is the “RAW” file."

I suggested in an earlier response that the Sidecar file is perhaps absent or incorrect. I could be wrong of course, but I believe it is the sidecar file that can preserve adjustments in the raw reader. If my client adjusts settings on a raw file, and includes the sidecar, when I open the file in raw reader the client settings are present. Similarly, when I snap a frame on my Canon 7D, the raw file it typically fairly close to what I viewed on screen at capture.

Yes, the sidecar file is an XML file that holds the adjustments you have made as the RAW file it self is never changed.
It's because CRAW "knows" about your camera and puts the default setting in to the sidecar file.
The screen on your SC shows a jpeg generated from the RAW using setting embedded in the firmware.
If they look different then it's because the CRAW defaults are either missing or different.

 

[Dave Maine]

If you look inside, you will see that it is a TIFF, not any kind of bayer raw.

 

[Deleted member 114292]

If you look inside, you will see that it is a TIFF, not any kind of bayer raw.

A DNG is a tiff. A TIFF file is just a container. I maintain the the data in the TIFF is the raw data.
You could take a glance at the PDF

https://www.adobe.com/content/dam/acom/en/products/photoshop/pdfs/dng_spec_1.4.0.0.pdf

 

[brett8883]

Yes, the sidecar file is an XML file that holds the adjustments you have made as the RAW file it self is never changed.
It's because CRAW "knows" about your camera and puts the default setting in to the sidecar file.
The screen on your SC shows a jpeg generated from the RAW using setting embedded in the firmware.
If they look different then it's because the CRAW defaults are either missing or different.

So I think I have this figured out and I’ve had the same thing happen to me.

As I mentioned earlier the live view on the SC is just a simulation of the exposure that would result from the shutter length that is set for the photo. The live view isn’t constantly exposing the sensor for the same duration as the photo would be taken. To achieve this the live view would have to be boosted and I’m not sure how this is done but it’s not by slowing down the shutter.

I have had the same thing happen at sunsets or dusk particularly when my scene is backlit by the sun and it’s because it was a longer exposure and the dark parts of the frame are being under exposed. It looks like from the live view that I should get the proper exposure but I don’t.

DJI Go 4 has a fix for this so I can at least see what the photo looks like right after I’ve taken it to tell if exposure is at least in the zone. It’s in the camera settings and it’s called “long exposure preview.” It will show me the photo I’ve just taken for a few seconds after it’s been taken.

 

[brett8883]

The reason it wouldn’t happen in videos is because the longest shutter you can have with video is equal to 1/frame rate, ie 1/25. With photos we could have up to an 8 second exposure.

With a longer exposure a low iso just might not be sensitive enough to pick up the shadow detail while the super bright sun is exposing other areas of the sensor.

What im still confused confused on though is how you were able to recover enough detail to be acceptable. Could you maybe post the photo so we can see what you are talking about? The DNG would be great but the edited jpeg is good enough

 

[Dave Maine]

It is not. It is an RGB translation of the raw Breyer sensor data, and incorporates a lot of assumptions. If you compare a real raw DNG file in Photoshop or Lightroom with it’s TIFF equivalent, there is no comparison.

Lightroom’s AI functionality for image adjustment does not attempt to work on TIFF images, but works great on raw/dng.

 

[WithTheBirds]

It is not. It is an RGB translation of the raw Breyer sensor data, and incorporates a lot of assumptions. If you compare a real raw DNG file in Photoshop or Lightroom with it’s TIFF equivalent, there is no comparison.

Lightroom’s AI functionality for image adjustment does not attempt to work on TIFF images, but works great on raw/dng.

DNG files all comply with the TIFF/EP6 standard so it is fair to say all DNG are TIFF but not all TIFF are DNG. It seems the most important distinction is that noted by you- TIFF files don’t contain the RAW data pre Bayer conversion. I can’t see a huge issue with the tiff containing post RGB conversion other than the loss of opportunity to perform demosaicing at a later time, when you find an improved raw converter for example.

 

[Deleted member 114292]

DNG files all comply with the TIFF/EP6 standard so it is fair to say all DNG are TIFF but not all TIFF are DNG. It seems the most important distinction is that noted by you- TIFF files don’t contain the RAW data pre Bayer conversion. I can’t see a huge issue with the tiff containing post RGB conversion other than the loss of opportunity to perform demosaicing at a later time, when you find an improved raw converter for example.

I think you are thinking the jpg preview is the raw data. There is still raw data, but there is also a jpg so the os and preview programs can display it.

 

[Dave Maine]

I think that that is backwards. A dng file may contain a TIFF image, or a semi-processed bayer image which allows for the greater flexibility.

 

[Deleted member 114292]

I think that that is backwards. A dng file may contain a TIFF image, or a semi-processed bayer image which allows for the greater flexibility.

a dng does contain a jpg preview. Tiff is a file format specification that can contain many objects and meta data. It just happens that dng used this file specification.
For example if you save a tiff from photoshop all
the layers can be saved in it so a tiff can have an unlimited number of images in it.
And ps saves them in its internal format
with a jpeg Preview
Read this read what sar has to say.

DNG images still garbage

After complaining about the quality of my raw files, DJI sent a new MP2 as it was still under warrenty, but the new one exhibits the same problems with regard to the quality of the DNG images. As example, attached left is DNG and right JPG. Any ideas why the raw file looks so pixilated?

mavicpilots.com

 

[WithTheBirds]

I think you are thinking the jpg preview is the raw data. There is still raw data, but there is also a jpg so the os and preview programs can display it.

Your thinking wrong. My statement was a TIFF file does not contain the raw pre de-mosaicing RAW pixel level data that a camera RAW DNG does. I also said that DNG complies with the TIFF/EP6 standard.

 

[Deleted member 114292]

Your thinking wrong. My statement was a TIFF file does not contain the raw pre de-mosaicing RAW pixel level data that a camera RAW DNG does. I also said that DNG complies with the TIFF/EP6 standard.

Well I don’t have a direct way to prove that one way or the other. It would be pretty hard for DJI to claim it’s raw if it’s not and you would think by now somebody would’ve figured that out.
I think it’s the raw image in there you don’t it’s OK we disagree.

 

[Thomas B]

So I think I have this figured out and I’ve had the same thing happen to me.

As I mentioned earlier the live view on the SC is just a simulation of the exposure that would result from the shutter length that is set for the photo. The live view isn’t constantly exposing the sensor for the same duration as the photo would be taken. To achieve this the live view would have to be boosted and I’m not sure how this is done but it’s not by slowing down the shutter.

I have had the same thing happen at sunsets or dusk particularly when my scene is backlit by the sun and it’s because it was a longer exposure and the dark parts of the frame are being under exposed. It looks like from the live view that I should get the proper exposure but I don’t.

DJI Go 4 has a fix for this so I can at least see what the photo looks like right after I’ve taken it to tell if exposure is at least in the zone. It’s in the camera settings and it’s called “long exposure preview.” It will show me the photo I’ve just taken for a few seconds after it’s been taken.View attachment 94695

Hyperlapse will allow shutter settings up to several seconds. If you don’t mind motion blur you can extract one of the frames used.

 

[WithTheBirds]

Well I don’t have a direct way to prove that one way or the other. It would be pretty hard for DJI to claim it’s raw if it’s not and you would think by now somebody would’ve figured that out.
I think it’s the right image in there you don’t it’s OK we disagree.

I have no idea where you are coming from. I never said the DJI DNG does not contain raw data. I also never said anything about JPG previews. It might be you are misquoting and meant to respond to someone else.

 

[Deleted member 114292]

I have no idea where you are coming from. I never said the DJI DNG does not contain raw data. I also never said anything about JPG previews. It might be you are misquoting and meant to respond to someone else.

Sorry you’re right I completely misunderstood you the bottom line is we do agree that’s all.
It just seems to be so much misunderstanding about this I was all too ready to assume that was you.
I guess I was being stoooopid. Maybe I need new glasses because what you said was perfectly clear.
Ps Actually you probably could put a raw image into a tiff file. The spec is pretty flexible as far as adding objects and I have seen some places where it said.TIF was an acceptable extension for a dng RAW file

 

[WithTheBirds]

Sorry you’re right I completely misunderstood you the bottom line is we do agree that’s all.
It just seems to be so much misunderstanding about this I was all too ready to assume that was you.
I guess I was being stoooopid. Maybe I need new glasses because what you said was perfectly clear.
Ps Actually you probably could put a raw image into a tiff file. The spec is pretty flexible as far as adding objects and I have seen some places where it said.TIF was an acceptable extension for a dng RAW file

Can be, need not be, and often isn’t- yes.... That was my point.

btw- in most instances where DJI and other imaging system raw files are claimed to be bad the person complaining is looking at the embedded JPG preview. I fell for it with my first digital camera.

 

[brett8883]

One thing I’ve always been confused about and maybe you guys will know the answer to this...

When you import a non-DNG raw into Lightroom and choose to convert it to DNG you are given the option to embed the original RAW file into the DNG. I assume this makes it one of the objects you refer to that a DNG can contain. How would one later extract this file and is it possible this pre-demosaiced data is stored in a DJI DNG and if so would it be possible to extract so that the demosaicing process can be done by CRAW?

 

[brett8883]

Hyperlapse will allow shutter settings up to several seconds. If you don’t mind motion blur you can extract one of the frames used.

Right because the shutter speed is = or > to the frame rate which in the case of a hyperlapse can be less than a frame every few seconds.

If you had a frame rate higher than the shutter speed than none of the frames would be fully exposed since a new frame would have to begin exposure before the previous frame could be fully exposed.

Also, getting into semantics here a bit, a hyper-lapse is taken as photos and then arranged later into a video. It’s not really recorded as a “video.”

 

[Thomas B]

Right because the shutter speed is = or > to the frame rate which in the case of a hyperlapse can be less than a frame very few seconds.

If you had a frame rate higher than the shutter speed than none of the frames would be fully exposed since a new frame would have to begin exposure before the previous frame could be fully exposed.

Also, getting into semantics here a bit, a hyper-lapse is taken as photos and then arranged later into a video. It’s not really recorded as a “video.”

Perfect explanation of how it works!