Oversampling--Why Does it Look Good?

[Caleb Heymann]

I've always struggled to explain why shooting HD for an SD output improves perceived image sharpness. Now I anticipate having the same problem with 4k for an HD or 2k master. It just doesn't make intuitive sense, at least not to me.

Does anyone have a theory of why oversampling works? All things being equal (lenses, DOF, color sampling, etc.), I'm focusing only on the sharpness and clarity of the image.

[Dylan Reeve]

I don't really know, but many formats undersample to begin with, so that probably has something to do with it.

Varicam is 960x720 at 4:2:2
HDCAM is 1920x1080 at 3:1:1
PAL DV is 720x576 at 4:2:0
PAL Digibeta is 720x576 at 4:2:2

So with all of those formats you start with an undersampled image anyway... What I don't really get though, if that is the reason, is how it remains 'better' when the oversampled image tends to end up on one of those formats at some stage anyway...

[John Brawley]

I've always struggled to explain why shooting HD for an SD output improves perceived image sharpness. Now I anticipate having the same problem with 4k for an HD or 2k master. It just doesn't make intuitive sense, at least not to me.

Does anyone have a theory of why oversampling works? All things being equal (lenses, DOF, color sampling, etc.), I'm focusing only on the sharpness and clarity of the image.

Try Nyquist.

http://en.wikipedia.org/wiki/Nyquist_frequency

jb

[Dominic Jones]

HDCAM is 1920x1080 at 3:1:1
Just for correctness sake, whilst most HDCam cameras have 1920x1080 CCD blocks, the recording format is actually 1440x1080 utilising non-square pixels - so spatially as well as chromatically sub-sampled.

But you knew that, didn't you Dylan?! :wink:

Not sure whether Nyquist is the answer here, as whilst the Nyquist frequency is higher with a higher resolution imager (or sampler, if you like), the entire "system" (once you take into account the delivery to intended format, say SD TV) also includes a stage resampling the image to the final format, and this step must also be subject to Nyquist's laws of sampling theory, surely? Or am I missing something?

I think Dylan might be more to the point, that the resample helps to hide deficiencies in the original, sub-sampled, image.

That's just off the top of my head though, and could be total BS! I'm sure Graeme and/or Stu will be along soon to put us right!...

[Mitch Gross]

In layman's; terms, all steps in any process have loss. Even in film, there is loss. Even from your eyes to your brain there is loss. And most important, you do not get to choose what you loose. So the more you start with the more you end up with.

That in a nutshell, is the case for oversampling.

[Graeme Nattress]

Let's not even think about colour, or a real camera, just a single black and white sensor with, say 1024 pixels across. The maximum detail that sensor can record is 1024 lines, or 512 line pairs. So, given a sharp enough lens, in focus, can you measure 512 line pairs? Sometimes yes, sometimes no - all depends on how those line pairs line up with the grid of pixels on the sensor - they might record all black or all white. Or you might get an aliased mess :-)

You should never try to record 1024 pixels worth of resolution on a sensor with 1024 pixels, or else you run into the problems above, and the result is not pleasant.

So we put in an optical low pass filter, which attenuates high frequencies in such a way as we avoid aliasing but also keep as much resolution from the lower frequencies as possible. Problem is OLPFs are rather slow filters, and need to be brought in early so as to filter enough of the high frequencies.

When you downsample, the anti alias filter can be made much steeper, and can cut in much later, thus allowing us to get much closer to our maximum detail limit without bringing in excessive aliasing. Over-do it, and you're back to an aliased mess though.

Now, say on RED we start with 4k samples, and we can get out to about 3.2k before we see no more detail. We're at about 78% of 4k, and negligible aliasing beyond that. When that is downsampled to 2k, you can aim for say 95% of 2k without significant aliasing beyond that, because of the benefits of electronic downsample filters.

So, if you start with a lower resolution camera, which doesn't have enough headroom for downsampling, you've got to aim for that 95%, or even 100%, but that means using too weak a OLPF or none at all (common in cheaper cameras), and that means aliasing hell. Or you can go for a sensible 80%, and apply electronic "sharpness" to "make up" for that other 20%, or some combination of the above.

Either way, it's much better to go for getting a good clean 78% of 4k than 100% of 2k which looks like alias hell.

It's worth remembering that downsampling is a sampling process, just as capturing with a sensor is a sampling process. The same rules of sampling theory apply, but in downsampling you're not limited to "physical" filters. That's why you have to be careful to avoid aliasing with downsampling, and why downsampling cannot remove aliasing that has already occurred.

Graeme

[number6]

Fascinating explanation, Graeme. Just wish you had included 3k in there in order to draw some practical usage implications, although I realize that wasn't the point of the original question.

[Dylan Reeve]

Just for correctness sake, whilst most HDCam cameras have 1920x1080 CCD blocks, the recording format is actually 1440x1080 utilising non-square pixels - so spatially as well as chromatically sub-sampled.

But you knew that, didn't you Dylan?! :wink:

Yeah, I did... God I hate all these different HD specs. It's staggering.

I defer to Graeme's knowledge on this one - it all makes good sense to me. More or less.

[Dominic Jones]

Ha ha, yep, ain't it fun!

Whenever I understand what Graeme's talking about it makes total sense, so I just have to assume that the bits that go over my head are true too!...

[laguun]

Just for correctness sake, whilst most HDCam cameras have 1920x1080 CCD blocks, the recording format is actually 1440x1080 utilising non-square pixels - so spatially as well as chromatically sub-sampled.

And to make things easier to remember: if you bypass the VCR in the camera and go through HD-SDI, the cameras will give 1920*1080p (750/790/900/900R) 4:2:2 10 bit YUV, 1920*1080i (730) 4:2:2 10 bit YUV, the F23 adds 1920*1080p at RGB444 and 1920*1060p at 60fps in YUV422 :)


But you knew that, didn't you Dylan?! :wink:

But you knew that, didn't you dominic?! :wink:

:)

[Graeme Nattress]

Another thing to remember, that offsetting the green chip from the red / blue chips in a 3 CCD and prism based camera is "good" for 4:2:2, it's bad for 4:4:4 where you can see that the colour components are no longer properly aligned, and that when you add, say, saturation to the video, you get fringes along the edges.

Graeme

[Dominic Jones]

But you knew that, didn't you dominic?! :wink:
:)
Ha ha, yup - sure did! :)

Interesting thread, thanks as always for the top-notch info Graeme...

[Joofa]

In layman's; terms, all steps in any process have loss. Even in film, there is loss. Even from your eyes to your brain there is loss. And most important, you do not get to choose what you loose. So the more you start with the more you end up with.

That in a nutshell, is the case for oversampling.

There is an other very important aspect of oversampling that helps you decide how you pick which functions to reconstruct your original signal. Somebody, mentioned Nyquist here. Of course Nyquist establishes a minimum amount of sampling rate, but it does not directly specify given the samples, how you reconstruct your original signal. Most textbook analysis assume a box filter in frequency domain, and hence, a sinc function in time domain to reconstruct the samples. But, of course one can use other functions.

Oversampling helped provide proof of the very significant work of the Hungarian electrical engineer for something that is sometimes loosely referred to as "Gabor wavelets" -- the proof that Gabor should have provided, but he didn't. Denis Gabor was the inventor of holography, for which he won a Nobel prize in physics in 1971.

In 1946, Gabor published an almost seminal paper that had the potential of revolutionizing the field of electrical engineering and signal analysis, much the same way when Shanon published his seminal and highly influential paper in 1948. Unfortunately, though Shanon's work was instantly embraced by people, Gabor's amazing work was hardly given any attention until after his death.

Could it be because just like Fourier, who just stated Fourier series expansion, and gave little insight into a formal proof, Gabor stated the expansion of his now famous Gabor functions, but never offered a proof that the expansion really exists (i.e., converges). In fact the assumption in Gabor's original work was the most pathological case in which an expansion can exist in time-frequency analysis.

However, later Baastians, Janssen, and Daubechies showed that under over-sampling conditions a Gabor's expansion does make sense.

Gabor's work has tremendous applications including the modeling of human vision system, biometric analysis for eye scans for security, finger print matching, signal and image compression, and the list goes on and on ...

[Tom]

The main thing I notice in downsampled CMOS images, especially in low light, is a reduction in visible noise.

[Graeme Nattress]

All images, when properly downsampled, should show lower noise.

Graeme

[lordtangent]

Can you explain why Graeme? Or the math behind it I mean. This is something I've been thinkgin about lately... basically, when you down sample you are combining samples though some sort of filter. So let's take a simple case of a 1/2 reduction thougha box filter, just an easy example. (i.e. four pixels become one) How many db of noise reduction would that provide? I'm asking because I don't trust my math and intuition on the actual result. (I was assuming 6 db less... but is that right?)

thanks!