So how does RedRay work?

[Chris Park-Kennaby]

I've got no solid or official information to contribute however I was thinking about the leap that the RedRay codec makes from any previously available compression. Of course as time goes on compression only gets better but the massive difference between RedRay and older compression seems like there must be another other clever idea being implemented by Graeme...

Could it be possible that RedRay works by the same principle as RedCode? Most codecs encode three colour channels (RGB or YUV) but RedCode only has to deal with a bayer pattern greyscale image because it keeps the information RAW. As we know this image is then debayered resulting in an image that resolves about 80% of the original RAW resolution.

Would it be possible to blow up an image (any 3 channel image for instance) by 125% and apply a digital bayer patterned filter to create a greyscale image for encoding which would then be decoded and debayered on playback to form the original image?

Would this process be beneficial in achieving extremely low data rates such as demonstrated with RedRay?

[Graeme Nattress]

I can't say how the codec works, but I can tell you that getting good quality dilithium crystals in commercial quantities has turned out to be trickier than I thought.

Graeme

[Steve Wake]

I can't say how the codec works, but I can tell you that getting good quality dilithium crystals in commercial quantities has turned out to be trickier than I thought.

Graeme

Why didn't I see the Scotty/Jim connection before? Brings new life to old lines like:

"Jim, I canna change the laws of physics. I've got to have thirty minutes."

[Chris Park-Kennaby]

Ha! Love the response Graeme :) I've got no idea what dilithium has to do with RedRay but I'm liking the extremely vague honesty.

Sorry if this thread came across as an attempt to get you to spill the beans - not the intention at all, just thought it was an interesting idea worth mentioning. Though I'm still not sure whether it would be feasable or worthwhile...

As always looking forward to hearing more, when the time is right.

[Richard Foster]

Of course. It's like the transporter. It takes all the pixels apart and reassembles them perfectly. And, naturally, requires a lot of dilithium crystals.

[Keith Walters]

Leaving aside arguments from ignorance (you know; invoking some fantastic new top-secret compression technology, that I know nothing about, and neither does anybody else, apart from RED), and only discussing what is actually possible, I think I can make a fair stab at guessing of how Red Ray might actually work.

It seems to be fairly common knowledge by now that Redcode recording is based on JPEG2000 (or JPEG2000-like) compression of streams of four separate half-sensor-native-resolution images: one red, one blue and two pixel-offset green images. That is (for the RED One at least) there are basically one 2K red, one 2K blue and two 2K Green images per “4K” frame. The Green images are diagonally offset by distance of about 1.4 pixels, which allows the de-compression/de-Bayer process to produce an output resolution approximating that of a hypothetical 3-Chip camera that uses three “3K” sensors. Thus, although the fully decoded output from a RED One is routinely described as “4K”, it is more like 3K in a 4K container, that is, the 4-gallon 4K bucket never has more than 3 gallons in it. (The higher resolution MX and Epic sensors apparently do produce the full four gallons).

In an idealized RedCode Post-Production environment, the process would start with copying all the RedCode camera files into the editing computer (or making them accessible to it at any rate).
In line with all other non-linear editing systems, the computer then makes lower resolution copies of these files - the so-called “Proxies” - which are what are actually used for the editing process. In most cases, all the editing decisions are made using these lower resolution files, which allows the editing computer to work in real-time, or close to it.

At the end of the editing session, the computer generates some sort of Project file, which is basically a “script” of instructions that tell it how to duplicate all the editing decisions made using the Proxies, but this time doing it to the original camera files, which is generally known as making a full definition “conform”. (This is pretty much the same principle as using workprints for film editing, or “off-line” editing using videotape, in use long before computer-based editors became available).

In just about all non-linear editing systems, the higher resolution of the original camera files can make the high definition conform take a very long time, which is why most large production houses set up “Render Farms”. In a Render Farm, a large project can be cut up into smaller sections and “farmed out” to a large collection of identical dedicated computers, all working in parallel, to “render” the original camera files into the final full-definition output frames, which are then re-assembled to produce the final output file.

A great advantage of cameras and editing systems that store and work directly with Raw format, as opposed to more conventional ones based on fully decoded camera video, is that (in theory at any rate), the first time any image transcoding takes place is when the original camera files are rendered directly to the final output format by the render farm.

The same Post Production “script” can be also set to produce 4K, 2K, 1920 x 1080 HD, or standard definition video, or even 4K optimized for burning to a master negative for film duplication. In all cases, the output file is directly rendered from the original camera files, in more or less a single step.

In practice, none of this seems to happen very often. The vast majority of RED camera footage seems to get converted to the final output format (most often either 2K or 1920 x 1080 HD), PRIOR to the editing process, throwing away a considerable amount of the original sensor information. To make matters worse, quite often only a single format is rendered at the end of the process, and this is then re-transcoded by external apparatus to produce any other formats required.

What I suspect Red Ray is actually for, is an extension of the principle of RedCode, where the original sensor data (G1, G2, B and R) is directly compressed as if it was a series of four monochrome images, without any attempt at image processing.

In a live camera situation, JPEG2000 compression of the original sensor frames is about all that is possible in a current-technology real-time environment. There is, however, no reason why further compression of the Raw files would not be possible, using MPEG4 technology or some variant of it, but only in a post-production environment, where multiple compression passes are possible.

What I think happens with Red-Ray is that, instead of the render farm rendering down to one of the industry standard delivery formats, it only makes a simple assemble-edit of the original RedCode files, applies MPEG4-like inter-frame compression to the G1, G2, B and R data streams, and then folds the “Script” for the final image manipulation of the images (colour correction, gamma curve, brightness, contrast, sharpening etc), as metdata into the Red Ray data stream.

The output from a Red Ray unit (in a cinema for example) would then be decoded more or less directly from the original camera files, all of the Post “massaging” being done on the fly by dedicated high speed processors.

The overwhelming advantage of this approach is that it would minimize the amount of pre-render degradation of the original camera images. Even though they would undergo considerable data compression, the de-compressed files would still represent relatively untouched sensor data with its dynamic range essentially intact, which would produce the best results possible from the available data. This contrasts with the current situation where video parameters are routinely “baked in” right at the start of the editing process.

The Red Ray output could also be set to precisely match the resolution of the display device, which would eliminate yet one more stage of re-mapping.

Where this would fall down is in the area of CGI intensive features, since I doubt anything as compact as what has been proposed would have anywhere near enough processing muscle to produce convincing real-time 4K images.

However in that case, the CGI rendering computers could output the images in the same G1, G2, B and R Bayer format as comes from the camera. In that case, the images would most likely need little or no processing in the Red Ray unit, apart from being routinely de-Bayered to the desired output format.
Also, if you look at the vast majority of TV shows and Movies, editing other than simple cuts and a bit of gamma manipulation is rarely used anyway.

The major advantages would be:

* Minimum transcoding degradation between camera and display. (It would be analogous shooting on Kodachrome and projecting the original camera stock).
* A single low-data-overhead delivery format that could be configured to precisely match just about any display system on a relatively inexpensive player.

Anyway, that’s how I imagine RedRay works. And if it doesn’t work that way, it damn well should… :cheers2:

[M Most]

Also, if you look at the vast majority of TV shows and Movies, editing other than simple cuts and a bit of gamma manipulation is rarely used anyway.

Oh, if only that were true. Size and position transforms, speed changes, visual effects "fixes" (remove a mic boom, change a sign, replace a sky, etc.), and monitor burn-ins are very common and very plentiful in television series today, and even more so in motion pictures. If you think editing of these programs is that "simple," you most likely aren't involved in doing it.

As for Red Ray, I don't know any more than anyone else here. But I do know that it's not limited to RedcodeRaw input material (it can work with anything - and has to, because I've yet to work on a project in which every shot in the show was done on Red and not touched by any visual effects) and it's very unlikely that it is utilizing a reverse-deBayer approach. My impression is that it's basically a very well tuned JPEG2000-style wavelet encode coupled with an equally well tuned interframe compression scheme that may or may not be similar in approach to MPEG and H.264. If done correctly, the rates being touted for RedRay files are certainly possible, even if a bit surprising.

[Keith Walters]

Oh, if only that were true. Size and position transforms, speed changes, visual effects "fixes" (remove a mic boom, change a sign, replace a sky, etc.), and monitor burn-ins are very common and very plentiful in television series today, and even more so in motion pictures. If you think editing of these programs is that "simple," you most likely aren't involved in doing it.

The vast majority of which are well within the capabilities of a custom hardware solution. As I said, not everything would be amenable to such a process, and in that case a "Follow Copy" flag would be invoked, and pre-processed images would be output, with an unavoidable introduction of transcoding losses.





As for Red Ray, I don't know any more than anyone else here. But I do know that it's not limited to RedcodeRaw input material (it can work with anything - and has to, because I've yet to work on a project in which every shot in the show was done on Red and not touched by any visual effects) and it's very unlikely that it is utilizing a reverse-deBayer approach. My impression is that it's basically a very well tuned JPEG2000-style wavelet encode coupled with an equally well tuned interframe compression scheme that may or may not be similar in approach to MPEG and H.264. If done correctly, the rates being touted for RedRay files are certainly possible, even if a bit surprising.

Yes, but you're now appealing to explanations based on new technlogies that you don't actually know exist, which is a classic "argument from ignorance".

It just seems bizarre to me that if they really have developed some new ultra-efficient compression CODEC, they would be limiting its use to a relatively small-market product like Red Cameras

It's entirely possible that the original RedCode concept has been overtaken by other technological advances. What would a few years ago have been considered outrageously large capacity Hard Disks are now available that can be run and powered from a USB socket. Distributing 4K movies on portable USB hard drives is now a perfectly practical option, but not just for RedCode. Without cheap optical media support, I don't see it as having the impact it might
once have.

A means of projecting more or less directly from the original camera files would have clear advantages.
A more efficient CODEC that simply allows you to store conventionally processed/degraded files in less storage space no longer does


"If done correctly, the rates being touted for RedRay files are certainly possible, even if a bit surprising"
To paraphrase one of your own statements:

If you think massively "re-tuning" the algorithms of a horrendously complex system like MPEG4 is something than can be successfully tackled by a relatively small team like RED's, you most likely don't know the first thing about it.

[Emery Wells]

Also, if you look at the vast majority of TV shows and Movies, editing other than simple cuts and a bit of gamma manipulation is rarely used anyway.


Keith, I appreciate your idealized vision of post production, but this is clearly written by someone with virtually no insight into the everyday realities of post. I wish the post process was as clean as you describe. Where I could have one magical 'script' that could contain the sum total of everything that has been done to my image. If that were to ever exist, it would require major cooperation and standardization across vendors. It would also hinder innovation over night.



If you think massively "re-tuning" the algorithms of a horrendously complex system like MPEG4 is something than can be successfully tackled by a relatively small team like RED's, you most likely don't know the first thing about it.

Mike Most knows some things.

[Petr Dvorak]

I want to see RedRay in action! :w00t:

[Mark L. Pederson]

It just seems bizarre to me that if they really have developed some new ultra-efficient compression CODEC, they would be limiting its use to a relatively small-market product like Red Cameras

Small market product ...?

Keith ... sorry to tell you ... but it IS true. They have developed a new ultra-efficient compression CODEC.

I've seen it. Up close and personal.

Seen stuff my company has shot and/or mastered - encoded on it and projected on a huge screen.

It is, in my opinion, one of the most significant achievements in Digital Cinema in history.

Stick around ... you will get a crash course in "small market".

[Stuart English]

It just seems bizarre to me that if they really have developed some new ultra-efficient compression CODEC, they would be limiting its use to a relatively small-market product like Red Cameras .

The codec in RED RAY is optimized for 4K content distribution via the internet, not camera usage.

It is our intention to demonstrate that to be a "significant" size market.

[Stuart English]

It's entirely possible that the original RedCode concept has been overtaken by other technological advances. What would a few years ago have been considered outrageously large capacity Hard Disks are now available that can be run and powered from a USB socket. Distributing 4K movies on portable USB hard drives is now a perfectly practical option, but not just for RedCode. Without cheap optical media support, I don't see it as having the impact it might
once have.

The concept of physical inventory - be that hard drives or optical media - is in our opinion obsolete.

It leads to too many inefficiencies and logistics overhead. The future of distribution is over a network.

[Shawn Booth]

If you think ____ is something than can be successfully tackled by a relatively small team like RED's, you most likely don't know the first thing about it.

I put a blank in your quote.... Fill in with whatever you like. The answer will always be "yes", "why not?", "if they really want to"...

It was once said about the same "small team" and building a camera... They now have 3. Three cameras that have thousands hooked... (One of which is named Epic. Which it simply is.)
I wouldn't doubt this "small team".

Just saying...

[Christian Edwards]

so Redray will still be needed as a proprietary device for encode/decode ?

The concept of physical inventory - be that hard drives or optical media - is in our opinion obsolete.

It leads to too many inefficiencies and logistics overhead. The future of distribution is over a network.

[Stuart English]

...so Redray will still be needed as a proprietary device for encode/decode ?

The RED RAY player and the RED distribution codec work hand-in-hand to deliver 4K resolution information, yes.

[PatrickW]

How does RedRay work for live content, if at all?

Can I connect a Epic to the internet and deliver live video/audio to Red Ray players in theatres?

[Anthony Rinaldi]

How does RedRay work for live content, if at all?

Can I connect a Epic to the internet and deliver live video/audio to Red Ray players in theatres?

Or to set-top RED Ray players even? Perhaps a RED Ray 4K live-streaming option?

[Petr Dvorak]

Or to set-top RED Ray players even? Perhaps a RED Ray 4K live-streaming option?

And what about 4K streaming from SPACE? That's something.
Stuart is NASA hookep up?
;-)

[Wayne Morellini]

I can't say how the codec works, but I can tell you that getting good...

Graeme

Graeme, is this codec your idea, or is somebody else's?


Thanks.

[Graeme Nattress]

Graeme, is this codec your idea, or is somebody else's?


Thanks.

I like to think the clever parts are mine, but you may need to wait for biography for the fuller version of the RED Ray story :-)

Graeme

[Dan Hudgins]

two 2K Green images per “4K” frame. The Green images are diagonally offset by distance of about 1.4 pixels,

A great advantage of cameras and editing systems that store and work directly with Raw format, as opposed to more conventional ones based on fully decoded camera video, is that (in theory at any rate), the first time any image transcoding takes place is when the original camera files are rendered directly to the final output format by the render farm.

Rather than have two green images its better to interpolate the two green images before compression and make a single 4K green image, the reason for that is that the wavlets disturb the two green images, so they do not quite combine as in True RAW, and interpolating the green as one plane makes a much sharper image anyway.

Your plan will not work in so far as it does not allow for time consuming area filters needed, although you can do "single pixel" color adjustments with LUT "on the fly", any complex area filters take too long to process "on the fly" by the player at 4K (well maybe it could be done but would require many complex image processing chips and a large memory size and power use and so cost too much for a low cost consumer player), they need to be done before encoding for release format (mostly in non real time). An example of that issue is that I think it has been said that RED ROCKET does not produce "finish" results the same as REDCINE-X, it seems maybe because extra filters were not implemented in the RED ROCKET hardware, mostly it seems RED RROCKET just de-compresses the images and re-sizes them, but does not ant-OLPF sharpen (un-sharp mask) or do heavy area filtering?

When I get my de-Bayer/image processing program out, it will do that kind of workflow in that the edit list can go back to the True RAW data to re-render for each output format, it saves quite a bit of render farm time that way because only the selected frames need to be rendered and the True RAW frames do take less diskspace than making RGB conversions of all frames. But in that case, the area filters would be part of the rendering before the result files were compressed for release formats.

I would think that RED RAY ingests RGB frames like any other compressed distribution format does like H.262 etc.

[Kaku Ito]

Greame,

I'm sorry that I couldn't meet up at NAB, got too busy color grading on a feature and also I got called to teach RED ONE at an University from this month. Anyways, what software would provide outputting to REDray format?

[Graeme Nattress]

Kaku, we will provide the software necessary for a REDray encode.

Graeme

[Petr Dvorak]

Kaku, we will provide the software necessary for a REDray encode.

Graeme

hurrah

[Jeremy Torrie]

That is awesome news Graeme...are you working with Avid for smoother integration? it would be great to be able to add RedRay as a device, but as of now it would be some manner of export to SSD rather than a traditional playout like we now do on Symphony Nitris or other...

[Tim Hole]

Did you read far too much Arthur C. Clarke, Asimov and E.E 'Doc' Smith growing up Graeme?

[Stuart English]

That is awesome news Graeme...are you working with Avid for smoother integration? it would be great to be able to add RedRay as a device, but as of now it would be some manner of export to SSD rather than a traditional playout like we now do on Symphony Nitris or other...

You just need to hand our encoder a sequence of industry standard DPX or TIFF files for the image and WAV or AIF files for the audio channels, Jeremy...

[Ryan Patch]

I think something that most of these posts are missing is the idea that REDCODE, as shot by cameras, and the REDCODE that will be used as the distribution codec are different things. REDRAY is not about decoding a live Epic signal, it's about encoding RGB values (because that's what you get after you modify, VFX, color, finish, title, conform, etc your project) into a codec that can be used for distribution. They are two different situations: one is for capturing light and storing sensor data to be decoded/interpreted later, the other is to take a finished RGB signal and compress (and presumably protect/encrypt to assuage studio's fears) it for network distribution. Although they are probably based on the same ideas, they are not the same thing.

[Stuart English]

Correct there are two versions of the REDCODE codec - REDCODE RAW for camera (which is obviously RAW) and RED delivery codec (which accepts RGB)

[Graeme Nattress]

Thanks for the clarifications Stuart.

Graeme

[Andy Jarosz]

I was told the short shown at NAB was played off a RED Ray. Is this accurate?

[Mohammed El Sharqawy]

You just need to hand our encoder a sequence of industry standard DPX or TIFF files for the image and WAV or AIF files for the audio channels, Jeremy...

two questions about the encoder, they were asked before on another thread..
will the encoder software be free?
how fast is it? and is there a hardware option to speed up the process?

[JD Holloway]

The codec in RED RAY is optimized for 4K content distribution via the internet, not camera usage.

It is our intention to demonstrate that to be a "significant" size market.


Yeah!!! so my hopes/predictions are true!
This product has taken so long to come to market...and still looks like it has some way to come.
I feel that all the work is being done, not in the player per-say, but with the content distribution network.
This will be a HUGE breadwinner for RED if the battle plan goes well.

I personally hope to have one of the first players, even if its to view 1080/2k material.
I know you have said its optimized for 4k delivery, but I hope you see the potential for 1080 too.
Much of the country will lag internet speeds and "on demand" has a catchy ring to it...<5 mb/s...'nuff said.
Add to that price/MB charged by some ISPs. As well, all the damn 1080 tvs that people have...and watch SD tv on.

[Stuart English]

Yes, but read again what I said about how 1080p benefits from being sourced from a full 4K resolution 10-bit image.

[Kaku Ito]

Kaku, we will provide the software necessary for a REDray encode.

Graeme

Super!

[JD Holloway]

Yes, but read again what I said about how 1080p benefits from being sourced from a full 4K resolution 10-bit image.


True enough. Down-sampling is fantastic; like the difference between artists and students oil/acrylic paint.
But to implement REDRAY quick to the mass market online (not theatrical), RGB catalogs will have to be used, no?
They are surely 1080/2K at best. There is little to the 4K catalog at this point.

Of course its obviously a fantastic way to distribute RED derived footage or 4K film scans.
I just see more potential in the market while "the 4Ks" catches up to us.

[Jeremy Torrie]

This thread seems to get forgotten, but I don't know how.

Stuart -any updates on RedRay Pro? Release date, in Red Store date?

[Wayne Morellini]

I like to think the clever parts are mine, but you may need to wait for biography for the fuller version of the RED Ray story :-)

Graeme

Thanks for that Graeme. Sorry for the late reply, been sick a lot again lately. You have given me enough infornation. If you are the only significant new or internal contributor then I guess it is based on existing codec or codec work. Which brings it down to a handfull of released codecs that it maybe based on, if not other codec/research work. My own ideas are not based on existing work, so it is unlikely to be too simular. They are pretty technical different and advanced. Look forward to seeing your spin on the field.

Thanks.

[Wayne Morellini]

I think something that most of these posts are missing is the idea that REDCODE, as shot by cameras, and the REDCODE that will be used as the distribution codec are different things. REDRAY is not about decoding a live Epic signal, it's about encoding RGB values (because that's what you get after you modify, VFX, color, finish, title, conform, etc your project) into a codec that can be used for distribution. They are two different situations: one is for capturing light and storing sensor data to be decoded/interpreted later, the other is to take a finished RGB signal and compress (and presumably protect/encrypt to assuage studio's fears) it for network distribution. Although they are probably based on the same ideas, they are not the same thing.

Never the less, there is a great need for something to replace h264 in consumer cameras and probably only one company that could effectively implement redray on consumer cameras in the near time, ambarella. Even for locsl video production and eng work, if rr is as good as people think it would be suitable. The reason is very simple, storage costs. Prices of sd cards stagnated forva few years. If Redray can do 4k at today's consumer fhd costs with better top end quality on eng and consumer work that is a big saving. It also presents great quality compared to hdtv, so serves well for hdtv aquistion. It would also offer much as a hdtv codec replacement, even fitting in the presnt channel structure. I notice the delay since these issyes were raised years ago, so hopefully it is some indication that some of this might be being considered.

[Graeme Nattress]

Wayne, there's a whole team of codec experts at RED. I just do my little bit..... I get to work with some rather clever guys.

Graeme

[Bruce Allen]

Thanks for all of the info! Sounds great and just what we need :)

One question remains: 48fps stereo 3D? ...in 3K? ...in 4K? Please?

And a projector that can handle those specs too would be lovely!

You know Peter Jackson will thank you!

Bruce Allen
www.boacinema.com

[Wayne Morellini]

Wayne, there's a whole team of codec experts at RED. I just do my little bit..... I get to work with some rather clever guys.

So, we have been misled, you are not the architech ;-)

If I haven't asked this before, how far off are we to getting gpu based acceleration on mac and pc for all things realtime on native redcode footage? Please, please, the level of mac i need is crazy under cpu processing. Gpu processing would also extend the life of the system.

I understand that old Gpu architechures faced constraints, but now most new mac systems are availsble with dx11 level gpu architechture with proper general purpose processing features. Open CL also has some crossover with existing C code for it. So any possibility of word from upstairs Graeme?

Thanks again Graeme.