Another CMOS question

[WILLIAM SPENCE]

I know that there is a mysterium upgrade happening in the future, and it got me to thinking about CMOS again. I know that right now it is prohibitively expensive to build a global shutter CMOS sensor. But I am wondering if you were to decrease the scan time short enough on current CMOS technology, if you would be able to essentially minimize rolling shutter problems like scew/partial exposure/wobble to the point that a global shutter would not be necessary? Any thoughts, or are there more effective ways than scan-speed to improve rolling shutter problems?

[Darren Orange]

Last I heard, this was not much of an issue as of Build 15.

[WILLIAM SPENCE]

Unfortunately, I don't even own a RED yet, so don't know how different builds have affected things. But from following this forum and viewing lots of footage, it seems clear to me that RED has a better handle on rolling shutter issues than Sony and Canon. So I figured with the high level of expertise available here, it would be a good place to pose the question. I know that Sony has been doing some sort of column summation, but I don't know if that is a more effective approach than simply making the scan time as small as possible - ?

[Uli Plank]

From our tests Red is doing very well in this field – it's not free of RS effects, but it's quite a bit faster than Sony's EX-1 ( which is showing exactly the same degree of RS as a cheap, but good Canon HV20 ).
We are currently doing excessive testing for 3D motion tracking with different cameras. I'll report our results.

Regards,

Uli

[WILLIAM SPENCE]

Nomad, that sounds like an exciting test. Please do report on that test as i am sure that many here would like to hear the results. Any other thoughts on how to improve RS effects without the cost of a global shutter? I figured with the new Mysterium X approaching, that would be one of the issues at the top of RED's priority list since they already have the finest image available(IMHO). I'm just curious if extremely fast scan times alone could improve the issue, or if it is being approached in the industry in some other engineeringly genious way :)

[DanWood]

In situations involving a lot of movement, RS may be noticeable. For example sports. In the current football series in Europe, the dominant broadcast camera is Sony HDC-3300. Sony uses 3-CCDs with 2/3" lens.

Dalsa Origin 4K and Sony F35 cameras have no RS. Arri D21 claims to have no RS. But look at their price.

For most scenes in movies, RED can be used without any noticeable RS. Ketch Rossi has made a feature length movie with RED. He has no complaints (only praises) for RED.

You have to look at the price advantage of RED. The quality you get with RED for $17K is amazing. (with lens, monitor, etc. < 50K).

P.S.: Comparing RED to consumer cameras under $6K, like EX1 and HV20 is unfair. RED should be comapred to F35, D21, Origin 4K, etc.

[David Mullen ASC]

Well, the Dalsa and the Arri-D21 have mechanical spinning mirror shutters, and the F35/23 uses CCD's I believe (as does the Dalsa).

Besides the RED, the other CMOS cine cameras with no mechanical shutters are the Phantom and the SI-2K.

[Ed Blythe]

I remember iBloom and some others discussing the possibility of an 'add-on' mechanical shutter in front of the actual lens... I think. Bit fuzzy on the details of how this would be effective in tackling the problem but that's the only example I can recall of "engineeringly ingenious" solutions.

[WILLIAM SPENCE]

I am certainly not comparing the RED to Canon or Sony, the EX1 and HV20 are not in REDs class. But I have read enough to believe that it may not be the best idea to shoot RED at a model catwalk with a million flashes going off - at least not yet. I just thought it might be interesting to open a technical discussion on how to improve the RS situation from a technology standpoint. During a pan, if it is fast enough, skew is induced because of the time it takes for the chip to be scanned from top to bottom. My thoughts are that if you could really increase the scan speed of the chip, that rolling shutter issues would just disappear and the expense of a global shutter would become unnecessary. What I do not know is if there is a bottleneck or physical limitation that would make the scan speeds necessary to accomplish this to be unattainable. But I do believe that the first chip manufacturer that comes up with a solution is going to have a big foot in the door and sell lots of them until the other companies can catch up. I personally am hoping that it will be RED. I am curious as to if anyone has any other theories such as column summation or other ideas on how to improve on the problem yet keep it affordable - ie keep global shutter unnecessary.

[DSPographer]

I think the addition of a mechanical shutter is the most obvious way to fix the rolling shutter issues for normal exposures (i.e. as long as the frame rate and shutter angle allow for it). Here is a presentation I just noticed on the issues involved in CMOS sensors with electronic global shutters and increased dynamic range:
http://www.cmosis.com/publications/slides-cmosis-vision-11062008.pdf

[DanWood]

Sony has developed a 35mm CMOS 24x36mm 24.81 megapixel imaging sensor.
It output all pixels at 6.3 fps with lower noise. This is achieved by “Column-Parallel A/D Conversion Technique” whereby each column of photosites is given its own Analogue to Digital Converter.

So far this is the best CMOS chip in this segment. It will be used in SLR cameras. If this chip is used without a shutter in a videocamera, the RS problem will be reduced but not eliminated.

Sony and Panasonic have half a century of R&D and they use CCDs for a reason.
Origin and ARRI have quietly spent years researching this stuff before using a shutter solution.
This is one of the resons for their higher price tag.

When a shutter solution is used, the price of the camera will be higher than $17K.
For $17K, you have to accept some compromises.
You may never get such a low price from Sony or Pani or anyone else. Because they all sell through middlemen.
Sony executive told me that the Sony factory gets only 25% or less of the suggested retail price.
The rest is all eaten up by middlemen.
So, to sell for $17K, sony has to produce it for $4.25K or less.

'no overhead' is the reason why Jim can give you a camera this cheap.
In the future, I hope, Jim might produce a more expensive camera with a shutter solution.

[Mike Smith]

I'm kinda confused about this whole mechanical shutter with a CMOS sensor thing... how exactly would that help the rolling shutter issue?

The issue is that a CMOS sensor cannot capture the light over the whole surface at one time the way a CCD can. How would covering up the CMOS sensor, partially or completely, help in this situation? Maybe it's obvious but it's not jumping out at me.

Thanks - Mike.

[Anders Holck]

The issue is that a CMOS sensor cannot capture the light over the whole surface at one time the way a CCD can.

Actually this is not the problem.
The CMOS can accumulate ligt across the whole sensor, its the readout and reset of the captured light in the photosites/gates that has to happen in a serial process, one line at a time. This is why a mechanical shutter will fix the problem, by covering the sensor while this part of the cycle is happening.

[Mike Smith]

Oh ok. Thanks Anders.

So basically, the CMOS chip stores the charge created by the light hitting it so even if it goes into total darkness, it still has this charge. Is that correct? So if any further light hits it, it changes this charge as time passes so that the image is changing before it can be read, is that it? Seems to make sense to me now. So a CCD must somehow freeze the charge at each photosite so it doesn't change over time, regardless of further exposure to light, until it is read. Hmm... I think I get it now! (?)

[WILLIAM SPENCE]

VMax, I was a bit confused by the mechanical shutter and what Anders said as well. But if your deduction that the mechanical shutter blocks the light and the charge is held and read off line by line during the time that the shutter is blocking the light, then this makes perfect sense. Which raises a few more questions regarding a mechanical shutter.

- How complicated is it to make a mechanical shutter that will do this properly? It would seem to be a simple solution, but then again things are rarely as simple as they seem.
- What are the limiting factors that a mechanical shutter introduces? Motor noise? Lowered frame rates? Shorter battery life? Higher failure incidences requiring maintanence?
- Would this be a cheap solution that could eliminate rolling shutter issues for manufacturers, or are there other solutions that make better sense?

[Jannard]

There is a bit of mis-information here. A mechanical shutter does EXACTLY what a rolling shutter does. It wipes the image from top to bottom (or side to side in some cameras). Just as a read-reset sensor does. Covering the part of a sensor that has already been read does nothing more than the sensor does itself. There is skew with a mechanical shutter just as there is with a read-reset sensor. The only difference is the time it takes to "wipe" the image plane vs. read-reset the sensor. In the past, the time has been much longer to read-reset (the more time it takes the more of a problem it creates). Those days are coming to an end. A mechanical shutter has much less advantage than people give it credit for... especially as CMOS sensor read-reset times shrink, and eventually are quicker than mechanical shutters.

Most people think RED footage has a "filmic" look. Much of that is due to the high, unsharpened resolution... but a lot of it is due to the "feel" of a fast read-reset sensor.

When we 1st announced this camera, we were told by the "top dogs" that it was mandatory for us to include a mechanical shutter. We haven't been asked for that feature by the feature film, ASC guys in a long time. I wonder if they aren't now used to seeing a "bright" image on an LCD instead of a "dark" image through an eyepiece in low light?

Jim

[DanWood]

I am not sure I underatand.
Many has pointed out the RS in RED.
F35, Origin 4K, D21 etc. claim to have no RS.
Are you saying, in the future, RED will have a solution for RS without a mechanical shutter ?

Sony claims to have the best 35 mm CMOS sensor at this time. Are you planning to use Sony sensor in RED?

[Jannard]

Not sure what part of this you don't understand...

The only "problem" with a rolling shutter is a long read-reset time. History will not look kind on early CMOS sensors because this time was too long. We have shortened the read-reset time considerably just in the last year. Next generation sensors (Epic, Scarlet and RED ONE upgrade) will equal that of a mechanical shutter. After that, they will be faster.

Try to wipe your mind of preconceived notions...

Jim

[Jannard]

My last rolling shutter comment for the night. A rolling shutter (with correct read-reset times) is a GOOD thing... not a bad thing. It accurately emulates a mechanical shutter if the wipe times are similar. We PREFER the look to a global shutter.

Jim

[Mike Smith]

Hmm... really interesting Jim. So, while I get what Anders was saying and I'm sure he's right about that, it seems my initial intuition that the mechanical shutter would offer little advantage with a CMOS sensor was not far off. In any case, an SLR with a focal plane shutter has always had the same issue yet no-one seems to consider it a problem. I cannot see it as a big issue with a motion picture camera either if you're trying for a filmic look. Even a fast paced scene like a car chase is going to have the camera following the action. Only the background will be at risk from rolling shutter artifacting and that will be motion-blurred anyway. If you're shaking the camera so much your "hero" character - vehicle or human - is warping, it's definitely not going to look filmic.

The only issue I see as insolvable with a RS sensor is flash photography in the scene which can be of extremely short duration; perhaps as short as a 30,000th of a second. No RS sensor can protect against that, at least not to my knowledge. But all photographic equipment has limitations you have to be aware of. Red is of course no exception. You just have to work around these issues as you would with a film camera. (With a film camera, you may miss a flash altogether and have to add them in post if that's what you're trying to capture.) What Red offers far outweighs these issues. If you bear them in mind, as you have to for any other capture system - film or otherwise - Red will do an excellent job, way beyond its pricepoint. An industry shaker indeed.

Regards - Mike.

[zak forrest]

Could someone give me a few scenarios where one would use the Red and a flash at 1/30,000th of a second?

[Mike Smith]

Zak, you wouldn't WANT to use a short duration flash with Red, and on a controlled set, you wouldn't have that issue. Some scenarios where you might be exposed to it... filming a concert, fashion catwalk, celebs at a red carpet event (Oscars)... in other words anywhere you'd be exposed to uncontrolled flash photography. But the worst that would happen is you'd have frames that were partially lit by the flash because the flash would be much shorter duration than the rolling shutter can go from top to bottom of the frame. In reality, I don't know if it would be that noticeable while watching the shot. If you were using this footage as "real" input to a narrative film, you may have to do something in post to fix it. In the past, I have seen films - long before Red came out - where they literally whited out an entire frame to represent the flash photography.

As I said, Red offers me far too much to worry about this issue. The non-flash issues such as warping from camera motion, I'm not concerned about at all as I don't intend on waving the camera about that wildly.

[zak forrest]

But do people even use 1/30,000th of a second when shooting a catwalk or red carpet event? I don't know anything about it, but it seems really excessive to me.

This just seems ridiculous and pointless to even bring up, like telling someone to avoid shooting helicopters or the wheels of a car because it will look like it's running backwards instead of forwards and that they would have to fix it in post. The rolling shutter on the Red is practically like any other motion picture film camera, and if you shot 35mm at the red carpet, well, whatever..

Also I think the motion is completely fine if you did decide to wave the camera around wildly.

I remember when some FX guys were on these boards and concerned with rolling shutter because it made their job a bit more difficult, I can understand that. But any other talk about rolling shutter on this camera just feels ridiculous to me. Sorry to fan the rolling fire, didn't mean to go off like that, I just wanted to know more about 1/30,000th of a second flash, and when people would actually use a speed that high...

[Mike Smith]

Yeah, I'm not sure why you're "going off" like that either. You asked the question and I answered it in as straightforward and unbiased way as I could. By all means feel free to wave your camera about while shooting. I didn't say you shouldn't - just that I don't intend to.

An intelligent flash gun does not vary its brightness for exposure but it varies the duration. With a tight shot, the feedback can make this a very short duration indeed. Remember, the shutter speed on your stills camera during flash photography has nothing to do with flash duration so you don't set flash duration as such.

[Erik Widding]

In any case, an SLR with a focal plane shutter has always had the same issue yet no-one seems to consider it a problem.

The only issue I see as insolvable with a RS sensor is flash photography in the scene which can be of extremely short duration; perhaps as short as a 30,000th of a second. No RS sensor can protect against that, at least not to my knowledge.

Focal plane shutter is a really good example of the issue, and the solution. Flash simply needs to be fired when the shutter is completely open. This is one of the reasons that flash sync on an SLR is limited to lower shutter speeds.


Could someone give me a few scenarios where one would use the Red and a flash at 1/30,000th of a second?

If you need to freeze motion, as in bullets, golf balls, water, etc. Probably not going to get enough speed out of RED ONE just yet for this, think Phantom rental until EPIC comes out... We are using 1/50,000 sec strobe to track golf balls moving 200mph, 18" away from the camera, for a project we are doing for a club manufacturer. Admittedly we are using a global shutter sensor, but you don't have to.

As long as the energy hitting your sensor from your strobe excedes that from the ambient light source by a 100x (or more, for more dynamic range and more freezing power), you can use the light source as the shutter. Unfortunately this can mean needing a strobe with a peak brightness 100,000 times ambient. Stated differently, ambient light hits the sensor for the entire exposure time, and the strobe for maybe 1% of that time, you need your light to be 100/0.01 times ambient.

So, think soundstage, not bright sunlight, as the opportunity to employ such a tactic. Though, in the first implementation of the golf system we did just this, double firing the strobe to get two frozen images in a single exposure:

http://www.birger.com/graphics/golfA225.jpg
http://www.birger.com/graphics/golfB225.jpg

For those that need to control lighting or effects, it would seem this would all become moot if the camera had a sync output that told you when the entire sensor was collecting light. Definitely can't do this at all shutter angles, and shutter speeds, but I would be willing to bet, if the sensor can go 60fps at 4K, you could do it with a shutter angle of 90+ degrees at 30fps.

The basic idea... sync your events to your camera, not your camera to the events.

[WILLIAM SPENCE]

Thanks for the insight Jim. Truly I am just curious as to how to improve CMOS, not to stir up debate on why or why it isn't an issue. For the way that I shoot, skew is not an issue, but flashes might be for the events that I cover. I think that everyone can agree that early CMOS designs had great image quality but rolling shutter issues. I think the best example is the HV20(and no, I am not comparing RED to that little Canon camera) From what you have said, making the read-reset times faster is all that is necessary to make this all but disappear - and without the cost, image repercussions and design complications that come with a CMOS global shutter. I think this is FANTASTIC news and is a solution that will make everyone happy. Good to know, and even better to know that this is something else to look forward to with Scarlet/RED/EPIC.