Compression, Math & the Internet

[Steve White]

Obviously RED RAY sounds amazing on paper:
2 hours of 4K on a ~9 GB disk.

People are crunching numbers and pushing out fascinating results, like 8 hours of 1080p and upteen hours of 720p. This math is wrong, and here's why:

The RED RAY codec is going to be an algorithm that compresses real world images. The amount of available detail in a given frame does not scale linearly with the number of pixels. There's only a finite amount of observable detail in any given real-world image, plus noise.

Even at 1080p, focusing errors and softness appear in most current films on Blu-Ray. Going to 4K is not going to make these films sharper - the information simply isn't there. Going to 4K won't make the picture worse, but it can't make it better either.

Moreover, there are no details out about the sampling of the RED RAY 4K. Your average Blu-Ray disc is 8-bit 1920x1080 4:2:0, giving a 3.11 MB/frame (uncompressed). 540 GB for a 2 hour film.

Suppose RED RAY uses a Bayer pattern encode at 4096x2304 and 12-bit, which gives 14.2 MB/frame. That's 2.45 TB for a 2 hour film.

Obviously neither Blu-Ray nor RED RAY can sustain these data rates with their 25-50 GB and 9 GB capacities. It therefore follows that you can break any of these codecs by feeding material that actually has bandwidth approaching the theoretical data rates of the uncompressed image stream.

Anyway - we'll just have to wait until the RED RAY codec is available to see under which scenarios it behaves. By that time, I don't think many people will care that RED RAY fits on a 9 GB disk - because that kind of bandwidth is almost feasible over the internet.

If the RED RAY codec becomes a viable internet distribution codec for HD and 4K video, I think that's how it will kill Blu-Ray, but I don't expect the images coming out of it to be significantly better than Blu-Ray currently provides. If they are, I will be pleasantly surprised.

[Dj Joofa]

Obviously RED RAY sounds amazing on paper:
2 hours of 4K on a ~9 GB disk.


Your average Blu-Ray disc is 8-bit 1920x1080 4:2:0, giving a 3.11 MB/frame (uncompressed). 540 GB for a 2 hour film.

Suppose RED RAY uses a Bayer pattern encode at 4096x2304 and 12-bit, which gives 14.2 MB/frame. That's 2.45 TB for a 2 hour film.



Spiff, actually if you look at the numbers for Red Ray, it appears they are manageable to some extent. Using your numbers for HD 540 GB for 2 hour film crammed into a 9 GB disk gives a compression ration of 9/540 = 1:60. Firstly, 1:60 is decent sized compression, not a whole lot, considering that many HD video conferencing systems do a decent image at around 1:300 compression ratio. Of course, for a video conferencing environment the image is assumed to have little motion. However, if you take a ratio of 300/60 = 5, that tells us that our HD movie can have about 5 times more motion on average than a typical video conferencing scenario, which is ok with me.

Now look at Red Ray's data. 9 GB / 2.45 TB ~= 1:270, which is approaching the figure for 1:300 for HD video conferencing. However, as happens frequently in image compression, the data rate does not scale the same way as the image size increases. I would estimate that compression ratio for Red Ray to be around 1:150. Hence, compared to HD video conferencing it is about 300/150 = 2.

[Steve White]

Keep in mind that I limited the data rates by chroma subsampling for the Blu Ray, and assumed a "RAW" compression for the RED RAY. This 270:1 image compression we're talking about is pure data reduction based upon redundant information within successive images.

Perhaps I was being unfair with my 12-bit resolution. So instead lets give RED RAY 8-bit at 180:1 compression. Your average mathematically lossless compression ranks between at 4:1 and 2:1. And I don't know about you, but I can sure see the artifacts in Blu-ray at 22:1 (assuming 2 hours in 25 GB).

My main point however is that this 180:1 data reduction raises eyebrows... but is not the relevant number. The relevant number is 9 GB, and whether or not enough information can be fit into 9 GB that the average person can't tell anything is missing in 2 hours.

[Dj Joofa]

Keep in mind that I limited the data rates by chroma subsampling for the Blu Ray, and assumed a "RAW" compression for the RED RAY. This 270:1 image compression we're talking about is pure data reduction based upon redundant information within successive images.


Of course, even my figure of 1:300 for video conferencing is for 4:2:0 sampled data.



Your average mathematically lossless compression ranks between at 4:1 and 2:1. And I don't know about you, but I can sure see the artifacts in Blu-ray at 22:1 (assuming 2 hours in 25 GB).


4:1, or 2:1 is for lossless compressions such as zip/bzip, etc. as you already point out. Lossless figures don't concern us here. As far as seeing artifacts in Blu-ray, may be that is because you are seeing it on an HD TV at home. For theatrical projection, the discerning power for signal variation reduces a lot because of many factors, with the contrast reduction due to the presence of flare in a theater not being the least of them. Hence, Red Ray can take advantage of that fact and smooth and blur the image to some extent as some detail is not going to be noticeable in a theater, which is otherwise visible on a computer monitor. That can help Red Ray fit more data in on a 9 GB disk.

[Steve White]

That can help Red Ray fit more data in on a 9 GB disk.
Just to be picky, I don't like this phrase. You can fit 9 GB of data on a 9 GB disk. What we're concerned about is how much data you can to represent with 9 GB.

It's obvious that you can stick a whole movie onto 9 GB - it's been done for years. What RED RAY offers is an opportunity to make that 9 GB yield a prettier image than the DVD MPEG-2 codec.

I'm awfully tired of picking out compression and edge enhancement artifacts from video. However, I'm not interested in blaming monitors for showing me compression artifacts. I want the display makers to try their best to accurately display their source, and if that source is loaded with errors, I'd like to know.

I hope RED RAY doesn't come with a whole slew of compression artifacts. I assume we won't see macro-blocking and mosquito noise. I assume softening and blur are "better" since they are less noticeable.

Ultimately however, I'd like my eyes to be the limiting factor with my motion picture displays. Right now, that doesn't appear to be the case. As a result, I still haven't purchased an HDTV.

[Dj Joofa]

I hope RED RAY doesn't come with a whole slew of compression artifacts. I assume we won't see macro-blocking and mosquito noise. I assume softening and blur are "better" since they are less noticeable.


A whole lot of noise artifacts creep into "video" because of block-based nature of compression. H.264 is typically applied to blocks in an image. I think blue-ray is also H.264 based and perhaps block based.

Red Ray has an opportunity to do full frame compression such as wavelets and hopefully such noise should be more limited.

[Bing Bailey]

the DCI Master spec is based on wavlets and jpeg 2000 codec

[Zach Nelson]

I'm awfully tired of picking out compression and edge enhancement artifacts from video. However, I'm not interested in blaming monitors for showing me compression artifacts. I want the display makers to try their best to accurately display their source, and if that source is loaded with errors, I'd like to know.

amen.

[Steve White]

Some of the Blu-ray discs out there are MPEG-2, and those ones look like crud. The better disks are H.264. Both of these algorithms are DCT and motion estimation based, and hence both feature mosquito noise and macro-blocking.

From what I understand of wavelet encoding, images are usually full-frame encodes, which does eliminate mosquito noise from the artifact list. Instead, overcompression causes softening in the picture. I'm not familiar enough with bad wavelet encoding to know what other artifacts one can expect.

Either way, in order to obtain the 22:1 (let alone 140:1) compression ratios we're talking about, temporal compression methods (usually group-of-pictures (GOP)) must be used. For low motion recording motion estimation and macroblocks do a great job, since the difference between successive pictures is minimal. However, since noise differentiates any two otherwise identical successive (real-world) frames, some amount of noise rejection is required for GOP to be effective. The difference between noise and the image is contextual however, so if your GOP codec is intelligent, noise rejection doesn't have to be a brutish attenuation of high frequencies.

I recall that Cineform uses something like a 2-frame GOP in their wavelet codec to boost compression efficiency. Their range is usually somewhere around 6:1 and 10:1 to achieve their "visually lossless" moniker. RedCODE RAW seems to be batting between 10:1 and 12:1 using an I-frame only approach.

[Bing Bailey]

what RED are about to attempt is amazing and the work of a mad genius lunatic. they are taking over acquisition with the RED ONE and RED CODE, They are taking over the Lens Market with their Lenses. They are doing most of their own accessories at lower cost. now they want to take over the post production and playback side of things with REDRAY, 4k Displays and Projectors.

I hope jim has bodyguards and armor because he's going to ruffle feathers in about 6 different industries. he may need a RED army and Im not talking about the one in CHINA

[Dj Joofa]

Some of the Blu-ray discs out there are MPEG-2, and those ones look like crud. The better disks are H.264. Both of these algorithms are DCT and motion estimation based, and hence both feature mosquito noise and macro-blocking.


A small correction. H.264 is based upon an (integer) transform derived from DCT. It is not exactly DCT, however, it does have the property of being perfectly invertible. In addition, H.264 standard allows for a de-blocking filter to reduce some of the blocking artifacts.




Either way, in order to obtain the 22:1 (let alone 140:1) compression ratios we're talking about, temporal compression methods (usually group-of-pictures (GOP)) must be used.

Yes, Red Ray has to be spatio-temporal to get compression ratios like that. In addition, there are only so many known ways of doing compression, and notwithstanding the mystery/hype surrounding Red Ray, it might be a variation on the known ways to mix spatial wavelets with temporal compression.

[Radoslav Karapetkov]

It'd be super cool if Jim resurrects HD-DVD for RED RAY and focks Sony good.

[Dj Joofa]

It'd be super cool if Jim resurrects HD-DVD for RED RAY and focks Sony good.

It is not easy to shatter a known standard such as H.264 based coding even if Red Ray comes up with its own known notion of spatio-temporal based compression comprised of wavelets. I think such compression will be designed from the point of view of projector/displays only, but it will suffer when internet streaming is considered. Wavelets do have some advantage over block-based coding, however, on the other hand, block-based coding does have certain advantages when it comes to internet and streaming. Can't have your cake and eat it too always.

[Steve White]

A small correction. H.264 is based upon an (integer) transform derived from DCT.
Thanks for that! I haven't read enough literature about H.264 to accurately reference the standard. Likewise, I'm not sure Blu-ray is technically H.264, though it certainly belongs to the same family as H.264, WMV-HD, and AVC-HD.

block-based coding does have certain advantages when it comes to internet and streaming.
Can you list or refer to some of these advantages? I was under the impression that the block-based encoding schemes won out previously due to their ability to be realized in hardware. Most of the wavelet stuff I've heard about has been CPU intensive calculation made possible by the likes of modern x86 and 64-bit architectures, and I expect RED RAY to have steep decode calculation requirements.

It doesn't seem like bitrate would be a problem though, which I thought would be the dominant issue with internet and streaming distribution.

[Radoslav Karapetkov]

No, what I meant was to just use the media [and its capacity], not the codec.

HD-DVD discs + RED RAY Codec.

[Dj Joofa]

Can you list or refer to some of these advantages? I was under the impression that the block-based encoding schemes won out previously due to their ability to be realized in hardware. Most of the wavelet stuff I've heard about has been CPU intensive calculation made possible by the likes of modern x86 and 64-bit architectures, and I expect RED RAY to have steep decode calculation requirements.

It doesn't seem like bitrate would be a problem though, which I thought would be the dominant issue with internet and streaming distribution.

Without going into the issues of fine/coarse grained parallelism offered by data partitioning and data locality for block-based vs. (full frame) wavelets; for purely streaming applications, the loss of data packets on the Internet, or even delayed receipt of a packet can cause severe problems because of time constraints maintained between the live streaming applications such as video conferencing. These issues are not faced with a dvd player or a projector, because data is never lost. However, for online streaming with image partitioning even if certain partitions are lost, portions of images can still be reconstructed. The H.264 standard went to great length to even suggest a Flexible Macroblock Ordering (FMO), where even if half of the data blocks making an image are lost, the error can be concealed! I am currently not aware of FMO being implemented in an online system due to its complexity, though. However, such options are there.

Additionally, internet streaming imposes a certain size on optimal MTU (~1500 bytes) for packets (Final Cut Pro / Quicktime users must have seen this number when adding a hints tracks to their movies). It is much easier to partition an image into blocks to fit into such MTU-sized packets as opposed to breaking the set of compression co-efficients for full frame image compression techniques.

H.264 (and to some extent H.263 and H.261; (H.262 is MPEG-2)), were designed with such issues in mind and even uses a Network Abstraction Layer (NAL) methodology for easier manipulation on networked environment.

[Steve White]

Ah. So would it be reasonable to say that a full frame encode scheme would be okay for internet downloads, but not so much internet streaming?

Moreover, can you see some kind of similar (in result, different in implementation) packet management for a wavelet image stream?

[Dj Joofa]

Ah. So would it be reasonable to say that a full frame encode scheme would be okay for internet downloads, but not so much internet streaming?


If temporal compression is also used for a full frame encoding scheme (which it will have to to reach those compression ratios), after the first frame, successive frames will have to offset the data content in the first frame (which block based do by differencing). What type of frame differencing may be used here is something that I have not thought about before and even if I did that would mean I would be doing the work for Red free of cost :) So can packets from the 2nd frame onwards be partitioned into optimal MTU sized units with the stipulation of good error concealment even if they are lost is something that will require deeper analysis of this issue than I can think in a few minutes here.

But it appears something has to be devised to get to the cleaner approach of block-based schemes for such issues.



Moreover, can you see some kind of similar (in result, different in implementation) packet management for a wavelet image stream?

Again, I shall have to think about it. Wavelets co-efficients are spread scale-space. How to use them in a temporal scheme in addition to their usual spatial domain mechanism will require some thinking at least on my part.

[Dj Joofa]

Spiff, you have had the honor of receiving the 500th message on this forum from me:)

[Steve White]

Woot! Now I'm at 50.

It's always encouraging for folks in R&D when the "future work" slide comes up. :)

A scholar.google.com search for "temporal compression wavelet" produces some 13,800 hits... so clearly some folks have been thinking about such things. I wonder if RED RAY will be the first iteration to percolate to the market.

[Dj Joofa]

Woot! Now I'm at 50.

It's always encouraging for folks in R&D when the "future work" slide comes up. :)



If it is not private information, you like to share what/where R&D work you do?

[Steve White]

My M.Sc. (physics) was in hyperpolarized 129Xe nuclear magnetic resonance imaging. My Ph.D. (engineering physics) is in progress, and has to do with magnetic non-destructive evaluation.

During my spare time in my M.Sc. program I took up video and visual effects, through which most of the work was done in MatLAB using the Image Processing Toolbox. In my search for more versatile tools I ended up moving to After Effects.

Since I'm generally more interested in the technical aspects of image production, my curiosity led me through the available technologies from miniDV to RED.

[Dj Joofa]

Wow! Engineering physics is a complex field. You are a heavy-duty guy. I better have some respect for you!

[Peter Majtan]

It'd be super cool if Jim resurrects HD-DVD for RED RAY and focks Sony good.

I am sure Toshiba will GLADLY sell the entire HD-DVD division with all IP and manufacturing plant to Jim for peanuts. You might be onto something here! :clown2:

[Radoslav Karapetkov]

Me hopes.

If they can cram 2 hrs. of 4K in a single DVD9[how da hell?!!!], the 25Gb of HD-DVD would be paradise.

But hey, I already got a DVD Burner ;).

[Peter Majtan]

Me hopes.

If they can cram 2 hrs. of 4K in a single DVD9[how da hell?!!!], the 25Gb of HD-DVD would be paradise.

But hey, I already got a DVD Burner ;).

That's the great thing about HD-DVD - the authoring process is almost identical to DVD and the drives are backwards compatible. Moreover, unlike Blu-Ray, the manufacturing cost of HD-DVD is cheap as hell!

Notice that DVD Studio Pro (from Apple, originally Spruce Technology's DVD Maestro) had a HD-DVD option build in even before HD-DVD came onto market!

Now this would actually make sense to resurrect it (as all the BRAND NEW infrastructure is still there available) and re-brand it Red-Ray!

You rock Karapetkov!

[Christoffer Glans]

Blu-Ray has been expensive due to the discwar and there's no difference between authoring for HD-DVD and Blu-Ray

Also, Blu-Ray already won so why bother?

[Stephen Gentle]

No, what I meant was to just use the media [and its capacity], not the codec.

HD-DVD discs + RED RAY Codec.

Sure fire way to make RED RAY fail... Having a format that everyone can already burn is what makes it incredible. Even with Blu-Ray it would be OK (well, it would be excellent - 50GB disks...), because like DVD burners, every computer will have one in two or three years.

Using a dead format makes zero sense. Who would want to scour ebay to try and find an HD-DVD burner so they could use RED RAY? I doubt there would be enough demand for Toshiba to keep making burners...

[Peter Majtan]

Guys! Chill out! We are clearly just having some fun on the account of SONY - can You blame us? :innocent:

[Peter Majtan]

Blu-Ray has been expensive due to the discwar and there's no difference between authoring for HD-DVD and Blu-Ray.

But if You wanna get started - here is why You are wrong:

HD-DVD was so much more popular with the DVD factories as it allowed them to re-use much of their current infrastructure. Not so with Blu-Ray. DARKKELT have made the same point: "500 dvd's with 4 colour artwork and case only costs about $1000. if you wanted 500 blu rays right now you're looking at 30 to 40 grand. not so much because the individual replication costs a lot but because the optical master costs a lot." HD-DVD is less then half of the Blu-Ray for both glass master and individual media duplication...

[Radoslav Karapetkov]

Well, I dunno, a lot of people already seem to have HD-DVD players [do these have upgradeable firmware?!].

And RED-RAY seems to be a pro application, not a consumer one like BluRay 1080p movies.

It's 4K! I can't see 4K becoming a consumer thing anytime soon [but maybe Jannard has a different opinion on that :) ].

And it's a digital cinema specification - 2K DCI, 4K DCI... - not a home video specification: 720p, 1080p.

30 Gb of RED-RAY encoded 4K, that's three times the capacity of DVD9 [2hrs. of 4K].

And they can rename it "4K-DVD" :), cause it would actually become exactly this.

Or simply "4K", which is even easier and cooler to say than "BluRay" - say BluRay 10 times, and then say 4K ten times :)... well they're almost the same, but 4K sounds more straight and also sounds like "more" :). People always like to have "more", you know? Lol.

So, 1080p consumer Sony BluRay and 4K [HD-DVD] RED-RAY pro application.

But maybe the difference between home video and digital cinema formats will disappear.

Actually, I think that the future belongs to Digital Cinema Wars: Attack of the Codecs.

And not so much to physical media differences.

Lol.

[Steve White]

Hm. No one has latched on to the end of my initial post.

Does it really matter if RED RAY burnable to DVD? A lot of people think that broad-band or re-writeable flash distribution will be the death knell for disc-based formats.

The data rates RED RAY proposes are already reasonable for internet downloads, as there are a number of services that routinely enable 5-10 GB downloads (Steam for example). Furthermore, if RED manages to address the streaming issues Joofa discussed earlier in the thread, one could see RED RAY becoming both a broadcast and internet streaming codec.

As for RED RAY and digital cinema projection... I'm a huge sceptic on this. Cinemas need to keep the quality of their presentation higher than can be achieved in the home. One way to do this is to go to higher fidelity distribution codecs. I can't really see digital cinemas going for anything but a visually lossless, high chroma codec. Based upon my limited knowledge of compression, I don't see how this is possible with the current RED RAY specifications.

[Peter Majtan]

Don't forget multi-channel audio issue for cinema distribution... But I could see the Red-Ray suitable for private cinemas, post facilities and as alternative source for "normal" cinemas to screen indies and other alternative programs...

[Dj Joofa]

A scholar.google.com search for "temporal compression wavelet" produces some 13,800 hits... so clearly some folks have been thinking about such things.

Yes, I would not be surprised by this fact. There is an over saturation of publications these days. There is a widening rift between the engineering departments in US universities and where the industry is headed. These people don't talk with each other much. People in academia try to solve problems (sometimes known as "manufactured problems") which have not applicability or need in the industry. People in the industry try to solve problems without knowing that solutions to such problems already exist in academia.

We were on the subject of Wavelets. I am always surprised that how many times that subject is resurrected. Daubechies and others seemingly "re-invented" wavelets perhaps realizing much later electrical engineers were working on Quadrature Mirror Filters (QMFs) a long time ago and they are deceptively close to wavelets. Time-frequency analysis that forms the backbone to wavelets can be traced all the way back to Hisenberg uncertainty principle and the coherent states in Physics. Gabor's famous work was forgotten until over-sampling resurrected it.

In computer graphics some of the seemingly amazing findings were known to electrical engineers for ages long ago.

Parochial vision, narrow focus and "publish or perish" philosophy so prevalent now has resulted in all this mess.

[Steve White]

What you say of academia is true.

In the end a lot of this stuff is just applied mathematics. However, each application of the math usually comes associated with its own jargon. I frequently find myself looking through publications and catalogues, but don't have the benefit of knowing the correct search term.

I work on something for a couple of months, and then come up with a 'new' way to describe it. I then type those word combinations into the search engines, and a whole new set of papers pops out of the aether.

Would that academic institutions rewarded quality over quantity. The only problem is that quality is much harder to estimate.

[Sindre Saebo]

Hi! I don't post here often, but have followed the Red forum almost from the start. I see people discuss the data compression of Red Ray and wonder how the Red Ray will work its magic. I have another question: If two Red Rays can be linked for 3D projection (which I read some speculation about in another thread), won't the compression artifacts from the high compression ratios you guys are mentioning distort and destroy the 3D image?

Sindre Saebo
Oslo, Norway

[Vincent Rice]

Blu-Ray has been expensive due to the discwar and there's no difference between authoring for HD-DVD and Blu-Ray
?

This is completely incorrect.

[conrad gaunt]

Just to be picky, I don't like this phrase. You can fit 9 GB of data on a 9 GB disk. What we're concerned about is how much data you can to represent with 9 GB.

It's obvious that you can stick a whole movie onto 9 GB - it's been done for years. What RED RAY offers is an opportunity to make that 9 GB yield a prettier image than the DVD MPEG-2 codec.

I'm awfully tired of picking out compression and edge enhancement artifacts from video. However, I'm not interested in blaming monitors for showing me compression artifacts. I want the display makers to try their best to accurately display their source, and if that source is loaded with errors, I'd like to know.

I hope RED RAY doesn't come with a whole slew of compression artifacts. I assume we won't see macro-blocking and mosquito noise. I assume softening and blur are "better" since they are less noticeable.

Ultimately however, I'd like my eyes to be the limiting factor with my motion picture displays. Right now, that doesn't appear to be the case. As a result, I still haven't purchased an HDTV.


Lets assume it uses a wavelet compression system. Wavelet compression doesn`t tend to create blocking errors because it doesn`t break images into blocks and apply different quantization levels to different blocks. What you`d get with tiny data rates is jpeg2000 style blurriness I think.
Wavelet compression systems are the best we have at the moment, even for lossless data compression. If someone created an intra-frame wavelet compression system, image data could get seriously squeezed, and artifacts minimized. I don`t know if the RedRay compression will be wavelet based or not, or whether it will use temporal compression, but I hope both, because the image will be packed with only relevant visual detail because wavelets are better at throwing away noise, and the data stream could get very small. 9gig could hold a lot of 4k visuals. I don`t see any reason why LCD, Plasma or whatever screens can`t increase in size?

[lordtangent]

If someone created an intra-frame wavelet compression system, image data could get seriously squeezed, and artifacts minimized.

Some one already has. And it's free...
http://www.bbc.co.uk/rd/projects/dirac/index.shtml

I wonder if Red is just recycling Dirac as "RedRay codec" the way they recycled JPEG2000 as "Redcode"? I for one would very much like to see a RAW flavor of Dirac and a company pushing for it's adaptation. Honestly I'm surprised Red didn't just go with Dirac from the start.

[Ryan Sims]

A lot of people think that broad-band or re-writeable flash distribution will be the death knell for disc-based formats.

Hmm, I looks like Sony is starting to think this way now. Hence, BD-live (http://www.sonypictures.com/homevideo/bdlive/).
Link your Blue-Ray disk to downloadable content. It won't be long before the Blue-Ray players just become another apple-TV box.
Can someone say Blue-TV.

[Matthew Lochman]

They should push cf card implementation as much as possible to increase demand and drive down the cost. That should be the distro for films, as well as the capture medium. if the cost of cf was a little lower than they'd be as disposable as cd/dvd and all RED shooters would be better off for it.

hey man- that's how film did it.

[Lachlan Ward]

he may need a RED army and Im not talking about the one in CHINA

They do a pretty good job of blowing stuff up and hitting people with sticks. I'm sure they'll do.

[Zack CC]

Just throwing some discussion points in.

http://blogs.zdnet.com/storage/?p=281

or

http://blogs.zdnet.com/storage/?p=313

= P

the day i netflix a chip, ill be happy.