“Not real 48MP”?

[brett8883]

Definitely some odd looking artifacts there. I have been playing with some demosaicing algorithms for Bayer sensors. I will have to update my program to explore quad bayer.

Is your project public? Would love to take a look

 

[LivinLarge]

Is your project public? Would love to take a look

Not at this time. I may share it later when it is more complete. I can share some results as it is now. Here is an original image, All channels as seen by the sensor, and a demosaiced image. View them at 100% or larger to see the details.

 

[sar104]

Well that is somewhat what’s happening but there is additional data captured to make better interpolation guesses. You can actually see the polynomial artifacts in the 48 MP if you zoom in far enough. Probably Cubic or Bicubic interpolation.

This is only appropriate to see the processing that camera uses and isn’t an indication of quality or lack there of. I by no means condone pixel peeping to determine the quality of an image.
View attachment 102373

But the real question should be - why are those ugly polynomial artifacts all red?

 

[sar104]

I am sure you are right on the technicalities. But to my eyes, the OPs samples just look like enlarging with noise reduction - like how Topaz GigaPixel does it.

Topaz Labs use some kind of AI on upsampling, which can have other unexpected effects.

 

[brett8883]

Not at this time. I may share it later when it is more complete. I can share some results as it is now. Here is an original image, All channels as seen by the sensor, and a demosaiced image. View them at 100% or larger to see the details.

That’s really cool thanks

 

[LivinLarge]

That’s really cool thanks

This algorithm uses the following method:
At each green pixel location, the red and blue values are calculated by averaging the adjacent red and blue sites on each side.
At each blue pixel location, the red value is is calculated by averaging the 4 red sites at the corners and the green value is calculated by averaging the 4 adjacent green sites.
Each red pixel site is calculated in a similar fashion.

 

[CountryGuy]

Lol
So you think pixel keeping on a thumbnail mean something?
Very amusing.

Well uh... but since they put it there it must mean something!

Ok, ok... kidding of course. Obviously not a way to make critical judgements however at the same time it can be quite useful. For example, I've had times when some setting or the other was off. Forgetting to reset exposure compensation for example when moving to different areas. Quick glance at the thumbnail help make the correction before too much damage is done.

 

[brett8883]

This algorithm uses the following method:
At each green pixel location, the red and blue values are calculated by averaging the adjacent red and blue sites on each side.
At each blue pixel location, the red value is is calculated by averaging the 4 red sites at the corners and the green value is calculated by averaging the 4 adjacent green sites.
Each red pixel site is calculated in a similar fashion.

The Sony IMX586 quad bayer sensor on the Mavic Air 2 looks like this.

 

[LivinLarge]

The Sony IMX586 quad bayer sensor on the Mavic Air 2 looks like this.
View attachment 102438

Yes, I know. I plan to update my program to simulate demosaicing a quad bayer sensor so I can compare the final images. Here is how I expect to do it:
One interesting thing to note is that the red and the diagonally opposite blue pixel would end up with the same color values.

 

[brett8883]

Yes, I know. I plan to update my program to simulate demosaicing a quad bayer sensor so I can compare the final images. Here is how I expect to do it:
One interesting thing to note is that the red and the diagonally opposite blue pixel would end up with the same color values.

Sorry my bad I read your last post out of context. My apologies

 

[brett8883]

One interesting thing to note is that the red and the diagonally opposite blue pixel would end up with the same color values.

And that is the crux of the issue... You can’t use that pattern and treat each individual pixel like you would on an traditional bayer. At some point you are gonna have to have repeating color values or you are gonna have to take values from pixels that are much further away. None of which are good options but what is the lesser of all the evils?

My proposal would be to treat each pixel partially as independent and partially as the larger cluster. For example take the value of the closest red pixel of the 2x2 red cluster and add that to the average of the remaining 3 red in the red cluster and divide by two. Do the same for green and blue. That way every single pixel has a unique RG RB and GB value but favoring the closest pixel of a different color.

It would have more detail than just treating a cluster as one pixel since you’d get increased detail by giving the closest pixel additional weight but it would still minimize the loss of one of the pixels in the cluster records noise.

The way you have it not only would red pixels be the exact same color value as the diagonal blue pixel but every green pixel would share the same RB value as a diagonal green pixel. Wouldn’t you need more than two green pixels to interpolate the value of green on a red or blue pixel and more than 1 red pixel and 1 blue pixel to interpolate red and blue for green pixels? I know you are trying to use only neighboring pixels for interpolation but is that really enough data to make accurate assumptions? In a classic bayer you are using at least 2 pixels per channel to make an assumption and half the time using 4 pixels per channel

Maybe I’ve already given it away but I’m a hack at this so if it sounds like I know what I’m talking about don’t be deceived.

 

[CJG]

Thank you all for an interesting discussion: the 48 from 12 sensor was confusing me (especially as the 48 image in the MPZ is something different again, as I understand it)

 

[pelermon]

How large is your typical 48MP jpeg and raw file?

 

[Ice_2k]

How large is your typical 48MP jpeg and raw file?

Raw files are a little over 90MB. Not sure about jpg right now.

 

[LivinLarge]

And that is the crux of the issue... You can’t use that pattern and treat each individual pixel like you would on an traditional bayer. At some point you are gonna have to have repeating color values or you are gonna have to take values from pixels that are much further away. None of which are good options but what is the lesser of all the evils?

My proposal would be to treat each pixel partially as independent and partially as the larger cluster. For example take the value of the closest red pixel of the 2x2 red cluster and add that to the average of the remaining 3 red in the red cluster and divide by two. Do the same for green and blue. That way every single pixel has a unique RG RB and GB value but favoring the closest pixel of a different color.

It would have more detail than just treating a cluster as one pixel since you’d get increased detail by giving the closest pixel additional weight but it would still minimize the loss of one of the pixels in the cluster records noise.

The way you have it not only would red pixels be the exact same color value as the diagonal blue pixel but every green pixel would share the same RB value as a diagonal green pixel. Wouldn’t you need more than two green pixels to interpolate the value of green on a red or blue pixel and more than 1 red pixel and 1 blue pixel to interpolate red and blue for green pixels? I know you are trying to use only neighboring pixels for interpolation but is that really enough data to make accurate assumptions? In a classic bayer you are using at least 2 pixels per channel to make an assumption and half the time using 4 pixels per channel

Maybe I’ve already given it away but I’m a hack at this so if it sounds like I know what I’m talking about don’t be deceived.

For my first try, I will use the pattern I have described and see what it looks like. I have studied several schemes for demosaicing bayer sensors and some do add weighted average of neighboring pixels beyond the closest ones. The result may be more pleasing, but is it more accurate? For those farther pixels to be relevant, the image has to be blurred before reaching the sensor. Most digital cameras use an anti-aliasing filter over the sensor to do this. It eliminates strange colored artifacts at sharp edges and in repeating patterns, but leaves a softer image.
You can see these artifacts in the image on the left.

 

[LivinLarge]

Raw files are a little over 90MB. Not sure about jpg right now.

JPG files I have found are 15 to 21 MB depending on how much detail is in the image.

 

[Conocth]

this excellent video explained it to me

 

[LivinLarge]

To understand the benefits and limitations of the Quad Bayer sensor, it helps to look at what is actually seen by the sensor and the difficulty of creating an accurate image from the missing data. (Warning: extreme pixel peeping ahead.)
The Bayer vs Quad Bayer image shows what a 48 Mp Bayer sensor would see compared to the 48 Mp Quad Bayer.
The Quad close up image shows that each of the colored squares in the Quad Bayer array is made up of 4 smaller squares with different possible shades.
3/4 of the red and blue data is blocked by the filter and 1/2 of the green data. The missing data is filled in by a SWAG (Scientific Wild *** Guess.)
The Bayer vs Orig vs Quad image shows a demosaiced Bayer image compared to the original compared to a demosaiced Quad Bayer image.

 

[Brojon]

In good lighting, it is resolving more detail that the same sensor in 12 Mp mode. Compared to a true 48 Mp sensor with a full Bayer filter, it would look pretty soft. It is a 48 Mp sensor with a 12 Mp filter array.

I think you have that backwards. It's a 12 mp sensor with a 4x Bayer. They do a sort of pixel "shifting" using the Bayer to do something akin to Super-Resolution algorithms.

Super-resolution imaging - Wikipedia

en.wikipedia.org

 

[sar104]

I think you have that backwards. It's a 12 mp sensor with a 4x Bayer. They do a sort of pixel "shifting" using the Bayer to do something akin to Super-Resolution algorithms.

Super-resolution imaging - Wikipedia

en.wikipedia.org

It's not super-resolution imaging - it does have that 48 million photosites. They are just in a quad-Bayer arrangement.