ISO Pics analyized

[CCIRandall]

Hey everyone,

Over at CamcorderInfo's blog we ran the RED pics through some testing software we use on all of our cameras, both camcorders and digital still cams. We realize (as we say in the post) that the images they put out weren't in lab-like conditions as most of the shots are underexposed and not evenly lit, so the color readings we got weren't any sort of scientific, but we did find some crazy amazing noise levels.

Check out the post here:
http://www.camcorderinfo.com/content/RED-camera-pics-out-our-analysis-Smooth-but-bad-32968.htm

[Cail Young]

Cool charts, but you don't seem to have taking into account that the camera was operating gained-up and yes, every single shot was deliberately underexposed...

[Laco Zamba]

Graeme said: "We've not calibrated over the full exposure range yet. Once that's done, you should not notice shifts."

http://www.reduser.net/forum/showpost.php?p=59031&postcount=97

[sander kamp]

Either I don't understand something or something went wrong when testing. When I look at the images in Safari (which shows color profiles) the RED images on this forum from look totally different than on camcorderinfo. Did you take in account that the test images were Adobe RGB on not sRGB?

[Graeme Nattress]

That makes total sense as what you're seeing is the quality of the sensor giving an image with as much lattitude as possible for post colour correction. Because it's raw, you can decode it however you like though.

Graeme

[Graeme Nattress]

Also, the colours are very accurate, but they're not in the same colour space that the test software was expecting, hence they look very under-saturated.

Graeme

[regista]

For noise and color accuracy, no less?

What a royal waste of time, IMHO.

:w00t:

[Craig Schober]

this is exactly the reason why jim and team red are sometimes hesitant to post day to day results that aren't just pretty pictures. now others who don't read carefully too might pick up this thread from google and wrongly assume the iso stills were intended to be properly exposed.

thanks for honesty and engineering excellence team red. keep it up.

[CCIRandall]

I hope people realize when we say these are preliminary images, we mean it. I mean the point is that the noise levels are CRAZY GOOD, and that's all we really wanted to point out. The color stuff is interesting, but yeah, it's just the run-of-the mill "We don't really know too much yet."

I think we're being fair when we say, "Since the shots were under exposed, the noise readings would likely score better than our current analysis. Noise free filmmaking in a digital world? Perhaps, although we're still waiting to see it in our labs. We talked with our friends at DigitalCameraInfo.com for a comparison to high-end still cams, and they said noise wise it blows away every camera they've tested. They told us while these images might have some color shifting going on, it's really impossible to gauge any sort of useful color data because all of the images are underexposed and we can't really give it a fair comparison to another camera till it's properly exposed."

Don't worry RED heads, we're not saying it's bad.

[wshultz]

They've said it can't be any kind of fair comparison for color but that the low noise blows anything they've seen away. If nothing else, it certainly vindicates Red's demonstration of low noise.

doh! Beat me to it above.

[Graeme Nattress]

As I said, if you test for sRGB for an image not in sRGB, you get funny results :-) Nice to get independent confirmation on the noise though.

Graeme

[Antoine Fabi]

Hi Graeme,

What is the real world difference between RGB and sRGB ?

[mdo]

I hope people realize when we say these are preliminary images, we mean it. I mean the point is that the noise levels are CRAZY GOOD, and that's all we really wanted to point out. The color stuff is interesting, but yeah, it's just the run-of-the mill "We don't really know too much yet."

I think we're being fair when we say, "Since the shots were under exposed, the noise readings would likely score better than our current analysis. Noise free filmmaking in a digital world? Perhaps, although we're still waiting to see it in our labs. We talked with our friends at DigitalCameraInfo.com for a comparison to high-end still cams, and they said noise wise it blows away every camera they've tested. They told us while these images might have some color shifting going on, it's really impossible to gauge any sort of useful color data because all of the images are underexposed and we can't really give it a fair comparison to another camera till it's properly exposed."

Don't worry RED heads, we're not saying it's bad.

It would be worthwhile to provide full data about the images on the site with the test results, including info on why the pictures were underexposed and the fact that the color space of the images was different than that of the testing software, so people don't get an incomplete impression.

Could save time playing catch-up with people who think you've identified flaws that don't really exist.

The point on the low noise is the real news here, and it seems that you feel this point is valid, even startling, notwithstanding the deliberate underexposure of the image. Why not clarify the confusions that muddy up the findings so people can get an accurate sense of what the picture represents?

[Graeme Nattress]

Well, the camera has it's own RGB space, unique to RED. sRGB is defined by specific numbers. If you know one and the other, you can construct a matrix that transforms from one to the other. Being video, we don't generally go to sRGB, but to REC709 instead. If you know the numbers, you can transform between any colour space you want.

Graeme

[Michael Schrengohst]

I think I would wait until the LART team gets some testing done.
You cannot perform any true test with the JPEG images that were
posted. I remember the Shoot out awhile back between the HDV
cameras and the HVX. They did not at all have the cameras tweaked
to perform at the best possible levels. They were knocking the HVX
but I think the HVX sells many more units than the comparable
Canon or JVC cameras. They should re-run those tests and compare
images with the Canon HV20. And the LART team should throw a Canon HV20
in the mix as well.

[Joel Kaye]

They were knocking the HVX
but I think the HVX sells many more units than the comparable
Canon or JVC cameras.

Just because I happened to go through this myself I don't think that's accurate. I believe JVC said they sold 12000 HD-100s before the HVX hit the market. I'm not sure how market share shakes out now though.

I ran my own tests for a month and tried every HVX setting there was. The HVX has it's points but the roundup tests by the pros were largely the same results I saw.

In the end we've all got to test for ourselves. I hold all outside tests with a grain of salt. Though I did see Jackson's short in Vegas and I pretty much trust that.

[Michael Schrengohst]

Just because I happened to go through this myself I don't think that's accurate. I believe JVC said they sold 12000 HD-100s before the HVX hit the market. I'm not sure how market share shakes out now though.

I ran my own tests for a month and tried every HVX setting there was. The HVX has it's points but the roundup tests by the pros were largely the same results I saw.

In the end we've all got to test for ourselves. I hold all outside tests with a grain of salt. Though I did see Jackson's short in Vegas and I pretty much trust that.

I see footage from all of these cameras. In the hands of a skiiled op they
can all produce great results. I have also seen some crap HVX footage.
I like the Canon but the HDV tape compression is a bummer.
I also shoot timelapse with a Nikon D200 and I hope the RED will
be able to produce moving (and timelapse) images that look as
good as what I am getting out of the Nikon.

[Jannard]

While the display of our low noise levels is nice, what is missing form this article is that it is a "cartoon shot" that we pulled off just to show the rubberband-like nature of our image. We certainly have not done any color matrix calibrations for that radical of under-exposure. And that fact seems to be missing in the article. If someone is going to take and use one of our images without permission, I think the least they could do is talk to us before publishing so we can make sure all the facts are there. I realize that Randall had good intentions, but this tells us we just shouldn't post anything more until we are done.

Jim

[Harmonica]

Hey everyone,

Over at CamcorderInfo's blog we ran the RED pics through some testing software we use on all of our cameras, both camcorders and digital still cams. We realize (as we say in the post) that the images they put out weren't in lab-like conditions as most of the shots are underexposed and not evenly lit, so the color readings we got weren't any sort of scientific, but we did find some crazy amazing noise levels.

Check out the post here:
http://www.camcorderinfo.com/content/RED-camera-pics-out-our-analysis-Smooth-but-bad-32968.htm

Just yesterday I was on CCI checking out your review of the Canon HV20, and wondering why we haven't heard from you guys over here on Reduser! Welcome, and thanks for the test results:biggrin:

[Joel Kaye]

In the hands of a skiiled op they
can all produce great results.

Thanks for adding that. I meant to say that. They are all fine and if you are good with any of them you'll get the good stuff.

RED will still kick their asses thorougly into submission soon. But they are all fine.

(Ok, I'm positive talented people will be using these and lesser cameras to tell better stories than some REDs will ever tell... but I'd still rather be shooting RED)

[Rocket]

this tells us we just shouldn't post anything more until we are done.

Jim

Damn, one guy always has to ruin it.

[CJ Roy]

While the display of our low noise levels is nice, what is missing form this article is that it is a "cartoon shot" that we pulled off just to show the rubberband-like nature of our image. We certainly have not done any color matrix calibrations for that radical of under-exposure. And that fact seems to be missing in the article. If someone is going to take and use one of our images without permission, I think the least they could do is talk to us before publishing so we can make sure all the facts are there. I realize that Randall had good intentions, but this tells us we just shouldn't post anything more until we are done.

Jim

Maybe a disclaimer explaining what RED has or has not done in regards to that particular test, attached to the released images, might help this misunderstanding. That way you don't have to put your customer base back in the dark as far as updates & image releases are concerned. For me, the new flood of image releases has been a great relief that real progress is being made. I'd hate to lose that again.

My .02

-CJ Roy

[jbeale]

Imatest is a sophisticated tool. In order to make the results useful, you need to use it properly. If you want to compare noise levels of Red and Brand X camera in a meaningful way, you need to shoot the cameras with the same lighting and the same exposure. You need to take output in the same color space and match the pixel resolution (since downsampled images show less per-pixel noise than native res images do). Then you can run Imatest on the output and get results that mean something.

For the test in question, I believe that none of these things has been done.

[regista]

Don't worry RED heads, we're not saying it's bad.

Randall, good or bad results, it's grossly inaccurate testing and equally irresponsible reporting, any which way you look at it.

Testing JPEGs off a website? Who are you kidding? I personally wouldn't accept the results of such an informal "test" no matter how positive or negative they would turn out to be.

Keep taking cheap shots like that and people who know better will stop relying on future CCI testing, no matter how serious, methodically disciplined, precise and sophisticated they may actually be.

:nuke:

[Michael Hastings]

Randall, good or bad results, it's grossly inaccurate testing and equally irresponsible reporting, any which way you look at it.

Testing JPEGs off a website? Who are you kidding? I personally wouldn't accept the results of such an informal "test" no matter how positive or negative they would turn out to be.

Keep taking cheap shots like that and people who know better will stop relying on future CCI testing, no matter how serious, methodically disciplined, precise and sophisticated they may actually be.

:nuke:

ditto

[Matt Setnes]

While the display of our low noise levels is nice, what is missing form this article is that it is a "cartoon shot" that we pulled off just to show the rubberband-like nature of our image. We certainly have not done any color matrix calibrations for that radical of under-exposure. And that fact seems to be missing in the article. If someone is going to take and use one of our images without permission, I think the least they could do is talk to us before publishing so we can make sure all the facts are there. I realize that Randall had good intentions, but this tells us we just shouldn't post anything more until we are done.

Jim

I agree Jim. Good intentions used off incomplete examples which ends in a complete waste of time. I'm confused that even though the article states the images weren't shot in lab-like conditions, why run a whole image testing and write a whole article on it?

[Tom Lowe]

These tests are too technical for me to understand. Can someone give a summary? Are they complimenting the RED on low noise?

[squinonescolon]

I think you should still post pictures, but make sure to mention in every one of your posts that the images are copyrighted. This way any legitimate media has to ask for permission or expect the wrath of Red's lawyers.
Steven

[Eugene]

The party's over, Honey.

[John V]

I agree...photos from a website do no justice to anything. Some testing

[Adrian T.]

Randall, good or bad results, it's grossly inaccurate testing and equally irresponsible reporting, any which way you look at it.

Testing JPEGs off a website? Who are you kidding? I personally wouldn't accept the results of such an informal "test" no matter how positive or negative they would turn out to be.

Keep taking cheap shots like that and people who know better will stop relying on future CCI testing, no matter how serious, methodically disciplined, precise and sophisticated they may actually be.

:nuke:

Ditto.

Randall, if you take your work seriously, you should put this report offline asap or at least add a big disclaimer that you used the wrong color space, that it's based on a low res JPEG, and that the camera was not calibrated for colors at this level of under-exposure. This was a noise test after all, and not a color test!

:ranting2:

Jim, please keep posting your test shoots and try to ignore such ignorant bastards. :matrix:

[albert rudnicki]

CHEAP SHOT !
Is this a Sony ad at the end of his analysis ?

http://www.camcorderinfo.com/content/RED-camera-pics-out-our-analysis-Smooth-but-bad-32968.htm

[Greg Voevodsky]

I think Randall should pull his unauthorized, copyright infringement, incomplete, misleading and poorly done article (with jpegs in wrong color space). He should also apologize to Red company and Red users for his misuse and abuse of red images.

[Adrian T.]

I think Randall should pull his unauthorized, copyright infringement, incomplete, misleading and poorly done article (with jpegs in wrong color space). He should also apologize to Red company and Red users for his misuse and abuse of red images.

Exactly!
Jim should force him to do so. :gun:

[Jay A. Kelley]

You guys are over-reacting. He did what he did, it will have NO EFFECT on sales or anything else. Also, he will be the first of THOUSANDS who will post images and punch holes in the picture. RED will hurt a lot of products, they will try to fight back.. Won't help, but they will try.

We better start getting a stronger stomach for this stuff, cause this is only the beginning.

Look at it this way.. Say his test stays like it is, then the camera comes out and future tests make his look wrong and unprofessional, that's the end of it. Us going bonzo like a bunch of "Red Heads" simply makes us look like we can see no fault with the camera. Like we are mindless worshippers here to grovel for all things RED.. I don't know about all of you, but that's not me. I'm here cause it's a cool camera and I am excited to get one. However it is still unproven technology in that all the promised features have yet to be operational. Do I think they will be? You bet your undies I do. However, until they are, the camera is not finished. We cannot argue that point and must respect those that say it.

Truely confident people have no need to yell for they know they have already won.

Jay

[Greg Voevodsky]

Truely confident people have no need to yell for they know they have already won.

Jay

It's funny how upset Hollywood gets when its copyrighted movies are illegally copied, manipulated, and released to the public before they come out.

It has nothing to do with a positive or negative review, rather a proper and professional one. It was not done legally with permission nor well with fact checking and it hurts the reputation of RED and others with misinformation. That's why it should be removed with an apology.

[Joel Kaye]

It has nothing to do with a positive or negative review, rather a proper and professional one. It was not done legally with permission nor well with fact checking and it hurts the reputation of RED and others with misinformation. That's why it should be removed with an apology.

I'm no attorney but it seems to me this would be covered under "Fair Use" because it's a review that's adds value (as commentary - accurate or not). You can use all kinds of stuff.

Were they crappy journalists? Possibly. They didn't contact RED for a response and they appear have posted some bogus conclusions. And why anyone would base any test of a camera that shoots compressed RAW based on a jpeg BEFORE the shipping camera is released is way beyond me. Guess they want cheap traffic - but my guess is a whole lot of us won't bother going to them for the scoop in the future.

The best response is for RED to post their side (which they've done) and forgetaboutit.

I think Peter Jackson and Soderbergh have a little more influence than whoevertheyare.com

[Keith Alan Morris]

Jim, rest assured no one is going to be influenced by a faulty test done with jpg's!

That link should definitely should be taken down for all concerned.

[Gregory Karydis]

Jim thank you for giving us something to chew on every little while and I know my heart jumps a beat when I see a new sample from the camera but I think it's a good idea to hold off from any future tests shoots going public.
At least you could have a forum locked ONLY to registration holders where you can post anything you like and we'd be under strict NDA to do as you please :)
While at the same time you could keep posting "public-safe" samples on the general forum.
I know it's a bit late for the RED One as the cat's almost completely out of the bag but for any future projects I think it'd be worth thinking about.

I for one will be equally happy if you do not post anything else until the cameras come out of production (I know some people might hate me about saying this but what can you do, eh?) Better safe than sorry.

[Keith Alan Morris]

Um, that's crazy talk. Post video, your kids giving me the finger, whatever you want!

[John V]

Not a test. Period..no control..not actual output from camera..downrezzed photos. Now if you have the correct information then perhaps.

[Vincent Rice]

As it becomes clear that RED is real and what's more 'rocks', a lot of web sites will be using any excuse to post RED related news. RED is the new SEX. It sells baby and we are going to have to get used to it.

In many ways I'm a little sad that the journey is nearly over. Soon RED will be mainstream and everyone will know all about it; its no longer the sacred secret of the enlightened few.

But then again, I'll get to make my short with it next year!

[Antoine Fabi]

I think Peter Jackson and Soderbergh have a little more influence than whoevertheyare.com

he hee... possibly.:nerd:

I just wonder WHY someone makes a ton of detailed analysis based on a heavily compressed picture, wrong color space, from a Beta camera...

...and publishes it...

[Oskari Sipola]

That analysis is just stupid in so many ways.

Take it down, Randall.

[ChristopherKenworthy]

he hee... possibly.:nerd:

I just wonder WHY someone makes a ton of detailed analysis based on a heavily compressed picture, wrong color space, from a Beta camera...

...and publishes it...

Maybe for the same reason that people join as Junior Members and post threads with provocative titles like, "Red Team beat up students for wearing Sony badges."

Read the wiki quote on disinformation. It's quite informative, but leaves out one of the key disinformation techniques, which is; 'Tell a lot of rational truth, with just enough false information to cause doubt'. Every time I see these murky posts and "articles" about Red, it leaps out at me as classic disinformation. Not that I'm accusing anybody directly, of course. It could just be incompetent misinformation, but that's almost as bad.

Much as I'll miss the images, I think Jim's justified in freezing things for now.

[Gavin Greenwalt]

http://i111.photobucket.com/albums/n134/im_thatoneguy/RED_Image_Quality.gif

[mdo]

I always thought gremlins were to blame.

[Tom Lowe]

LMAO, Gavin!! :)

[Greg Voevodsky]

Here we go again - with another REDiculous study with incomplete information from im.thatoneguy with "Red image Quality as a function of Magical Creature Proximity"

Where the heck are the Aliens and Big Foot? Reds 'AU' Awesome Units and 4k resolution with low noise in low light would surely capture these creatures and prove their very existence - unlike film of Big Foot and Alien Autopsy that was blurry, grainy, and hard to see that they were real... now, we can prove the so called expert scientists all wrong!

They exist! RED will capture and prove IT. Hale to the MAD SCIENTIST!

Now, excuse me, I have to take my medication now - the pink elephants are circling me. :-)

[Tonaci Tran]

What the hell is up with the URL??
"RED-camera-pics-out-our-analysis-Smooth-but-bad-32968.htm"

smooth but bad?

[Anders Holck]

Oh. Boy. Just remember the old saying:
"One fancy thermometer does not a doctor make"

I have a really hard time to grasp what the article tries to prove, or dispute. But I'm actually more stunned that they haven't pulled it yet, as it clearly serves no purpose and shows utter incompetence in both it's intended goal and procedures.

Well, guess it's another low blow for the so called internet reviewers/experts....

[Gavin Greenwalt]

I can answer all of your questions.

What is the article trying to prove: Studies look more important with lots of charts.

Why hasn't it been pulled: Had you heard of CCI before this post? Didn't think so.

[Brook Willard]

Guys... enough has been said here.