OLPF design criteria for Bayer vs. Foveon sensors

[Kevin Halverson]

I have been giving some thought to the requirements for the properties of an Optical Low Pass Filter (OLPF) and how its characteristics for a Bayer patterned sensor system are impacted by the sensors non uniform chromatic resolution. Consider that even with a theoretically perfect (text book brick wall) stop band performance for the chromatic filters of the Bayer layer, the requirements for the low pass filter differ between the green receptor sites versus the red/blue ones. Simple Nyquist thorium would dictate that in order to avoid aliases (out of band images) the character of the filter would need to be a minimum of 2x the inter receptor spacing. Red or blue inter receptor spacing is defined as 2x the receptor pitch, while Green is the square root of 2x the receptor pitch (29.3% less). This difference would seem to present a dilemma in that you can’t optimize for the green pitch without aliasing the red/blue and, conversely, if you optimize for red/blue, you end up burdening the green pitch with an overly aggressive filter. Add to this that the low pass filter’s skirts are far from infinitely steep so good practice would dictate that its characteristic would need to be somewhat larger than the 2x minimum suggested by Nyquist.

I would be very interested to learn how a purely monochromatic point source would behave when it deviated from a mean wavelength of 510 nm. A test that I have considered would be a pair of slits driven by a monochromatic source, the tendency to alias could be defined at various portions of the spectrum. By simply changing either the magnification factor or the distance between the source and the sensor while searching for alias behavior.

I do understand the desire to optimize the S/N ratio performance of the sensor based upon the energy distribution differences of the three color approach, but I wonder what other costs (in terms of ultimate performance) a Bayer approach has? In the area of the OLPF design it would seem that a Foveon approach would have an advantage by having uniform chromatic resolution and with this could have a less aggressive OLPF while still assuring an alias free result.

[Graeme Nattress]

Foveon wastes that uniformity advantage by the necessity for brutal chroma noise reduction.

Graeme

[T. Singh]

Simple Nyquist thorium would dictate that in order to avoid aliases (out of band images) the character of the filter would need to be a minimum of 2x the inter receptor spacing. Red or blue inter receptor spacing is defined as 2x the receptor pitch, while Green is the square root of 2x the receptor pitch (29.3% less). This difference would seem to present a dilemma in that you can’t optimize for the green pitch without aliasing the red/blue and, conversely, if you optimize for red/blue, you end up burdening the green pitch with an overly aggressive filter.

It isn't as bad as that, in practice, given the strong correlation between the channels. Information from one channel can be used to cancel aliasing in the others.

-T

[Kevin Halverson]

Curious.

To what do you attribute the "...necessity for brutal chroma noise reduction"? Are there transmissive losses to the deeper layers in the Foveon approach that impact the S/N performance?

I have designed a number of products that utilized large format sensors over the years (machine vision applications), but lack any first hand experience with the Foveon approach. I do find some aspects of it intriguing none the less.

[Kevin Halverson]

It isn't as bad as that, in practice, given the strong correlation between the channels. Information from one channel can be used to cancel aliasing in the others.

-T

This would seem to be true in a broad spectrum signal, but in my example of a monochromatic one, I doubt that enough information would be present in adjacent receptors to be of much value.

[T. Singh]

This would seem to be true in a broad spectrum signal, but in my example of a monochromatic one, I doubt that enough information would be present in adjacent receptors to be of much value.

You can certainly design worst case scenarios that perform badly, but those are rare in practice and ignored in OLPF designs. Furthermore, the signal does not have to have a broad spectrum for alias cancellation to work since the red, green and blue filters let in each others frequencies, i.e. there is significant overlap in their response curves. The human eye responds similarly.

Regards,
T

[T. Singh]

I would be very interested to learn how a purely monochromatic point source would behave when it deviated from a mean wavelength of 510 nm. A test that I have considered would be a pair of slits driven by a monochromatic source, the tendency to alias could be defined at various portions of the spectrum. By simply changing either the magnification factor or the distance between the source and the sensor while searching for alias behavior.


This is not a particularly hard case for the green quincunx sensors. Vertical and horizontal lines are easy to reconstruct since there are several (height/2, width/2 respectively) samples along each line. The hardest case is tilted at 45 degrees, where alternating diagonals of sensors are "missing." Even this isn't too bad because a square grid of sensors has sqrt(2) greater diagonal resolution than vertical or horizontal (see Fuji's Super CCD). Missing alternating diagonals still gets you within 1/sqrt(2) of the worst case full sensor (worst case being along the vertical/horizontal directions) resolution. What's more all of this reasoning neglects the contribution of red and blue channels.

A tougher test would be a grid of points.

Best,
T

[Miltos Pilalitos]

I think that in the rare occasion that we will face such a subject and it will indeed create aliasing in one of the color channels , it will be minimaly percieved by the viewer due to the high resolution.

Since, i dont believe we might shoot for example an ultra saturated red ball in front of a super black backround every day, it should be easy to treat those isolated cases in postproduction where it is very easy to fix such an issue.

[Jannard]

The last time I looked, the Foveon was capable of 3 fps...

Jim

[Cail Young]

Curious.

To what do you attribute the "...necessity for brutal chroma noise reduction"? Are there transmissive losses to the deeper layers in the Foveon approach that impact the S/N performance?

Graeme can confirm, but from the last discussion on Foveon, yes.

[Deanan]

Graeme can confirm, but from the last discussion on Foveon, yes.

With foveon there's a significant amount of color color crosstalk and also problems with rays at oblique angles with respect to the layers. You
end up trading some positives for other negatives.

Deanan

[Evin Grant]

I've always been amused with the Foveon fascination. Just because it emulates the structure of film doesn't mean it will look like it. In fact I would argue the most perfect sensor is just one with enough pixels so that there is at least one R,G & B pixel per final recorded pixel. This is esentially how the Genisis and D20 work. But Bayer patterens let us extract even more than that. If your final image was 2K than ideally your sensor should be 3K or about 3x the pixels. But as Red has shown we would probably only need about 2.2K for that because the Bayer pattern is a very effective way to interperate color and luma at the same time. So the best 2K camera in the world is a Red 4K image scaled to 2K. And the best 4k might be a 6K camera scaled to 4K, but could you actually see the diference?

[Graeme Nattress]

Yup for full frame they have limited fps Jim. Not sure on the current number, but 3-6fps is where it's at. You can do 30fps at an incredibly reduced resolution, but that's so pointless as to eliminate any of the supposed foveon benefit as you're massively oversampling.

The noise reduction comes in as silicon is a poor filter, so the responses for r, g and b are very wide and need very heavy matrixing to bring in line for "accurate" colour. Luma is easy to extract as you sum R+G+B. Actually, it's almost totally misleading to call the three layers R, G, B and to colour them brightly on diagrams as a raw foveon image with the layers directly assigned to R,G & B is dull and colourless in the extreme. Hence the heavy matrixing.... So you sum to get luma, and then you must go aggressive on the chroma data to remove the noise caused by the need for aggressive matrixing. This is all in the foveon white papers if you read them. They also mention the need for OLPF, which Sigma ignores, making their cameras, IMHO, incorrectly engineered. What most people "like" about foveon is actually the removal of the OLPF, not the co-sited colour. Of course, with the co-sited colour you can remove OLPF and just get luma aliassing, but given that even luma aliassing is bad....

Graeme

[Kevin Halverson]

Aliasing in any sampled system (not just video) is always a bad thing as there is nothing that can be done to distinguish the alias from the intended quantity. Good engineering dictates that you always need a LPF, I can "see" no argument for its elimination.

[Graeme Nattress]

Yup, aliassing is bad. Perhaps we need to tell Sigma? :-)

Graeme

[T. Singh]

Aliasing in any sampled system (not just video) is always a bad thing as there is nothing that can be done to distinguish the alias from the intended quantity. Good engineering dictates that you always need a LPF, I can "see" no argument for its elimination.

I think Sigma wants to avoid the sharpening needed to restore the high frequency roll off of an OLPF. Sharpening will worsen their already low SNR.

[Graeme Nattress]

They do use sharpening though... On top of aliassed images it looks nasty. They've found that some people actually like aliassed images and that who they sell to. Weird.

Graeme

[T. Singh]

They do use sharpening though... On top of aliassed images it looks nasty. They've found that some people actually like aliassed images and that who they sell to. Weird.

Graeme

LOL! This is funny!