Rocket or fast storage ?

[Mike N]

Hello,

I want your opinion about which is best to buy, Red rocket or a fast RAID storage in a situation in which I don't have money for both.

Maybe a silly question, why would you need a Rocket if you can't grade (don't need to) in native 4k resolution, since as far as I know at the moment only Scratch can take advantave of both native R3D grading accelerated by Rocket.

Wouldn't I be better of buying a fast RAID storage which gives me 600-700mb/s + read write speeds ? Wouldn't this also allow me to view full 4K playback in Redcine-X or other programs which work natively with red files ?

Would using a Rocket and outputting DPXs or TIFFS be slower if I don't have a fast storage which can handle all that amount of data ?

Ultimaltely what makes this whole process so speed hungry regarding storage speed ? I thought the codec is around 42 MB/s for the highest setting. A decent SATA drive easily tops that ? Is that during the debayering process file sizes increase exponentially ?

Would using an alternative codec like Cineform also spare me from owning a Rocket ? As far as i know Cineform preservs the RAW data but keeps the low data rates.

Thanks

[Jeff Kilgroe]

You will need fast storage to make full use of a RED Rocket, but you don't need 600-700MB/s. With refurbished Rocket cards available for $3K and relatively affordable RAID solutions out there, perhaps there is a compromise solution that would allow you to own both? What is your budget?

The RED Rocket accelerates the debayer process as well as provides hardware scaling and transform of your full resolution REDCODE images. So in that regard, it's much more than just for grading. It's really a power-house for transcoding and processing REDCODE.

You should also consider what application tool sets and which workflows you will most commonly use. Many will benefit from a RED Rocket, some will not as there is no support for the Rocket in some applications -- Adobe CS5, for example.

The REDCODE codec can be more than 42MB/s, it really tops out a bit over 50MB/s in some cases. But it seems that you will want a drive system capable of 100MB/s or more, SUSTAINED, to ensure reliable playback of R3D media. Typically, any two or more decent hard drives in a proper RAID-0 set or 3 or more drives in RAID-5 will handle this with ease. Many individual drives are up to the task these days, but a good host controller or RAID controller running that drive is also important.

Your speed requirements for storage will grow considerably, depending on your needs for transcoding or rendering your footage. In many peoples' workflows, their storage is often a bigger bottleneck than they realize. If you are reading your source data from the same volume that you are writing to, then you will need that 500MB/s or faster RAID when writing 2K, and especially 4K DPX files or other formats that take up a lot of space.

The Rocket is capable of a full resolution and quality debayer / demosaic of your RED footage at real-time or faster. But we find that the speed of transcodes is limited by a couple factors -- primarily, the speed of the storage, as we have been discussing. And also by whatever codec you are using to encode your footage. For example, the Apple ProRes codec is still bound mostly to a single CPU and is itself a real bottleneck.

[Tom Gough]

Jeff you are a diamond geezer :)

So generous with your time and experience - thank you

All the best,

Tom

[Mark Phelan]

Good question, not a fun decision because everything is a compromise. Since there was no Rocket card available when I purchased my system, I spent the money on a good RAID at the expense of not getting an LTO also. Plus, I wasn't convinced which LTO was the right choice, so I demurred. Now, there are better options and I am considering which route to take.

If it were me, I would still get a strong RAID first, then get the Rocket, if it has to be one versus the other.

[Anson Fogel]

You NEED fast storage for every major step in the workflow. You don't NEED a rocket. I did just fine for 18 months posting RED doing renders overnight without a rocket, but I could not have done much without fast storage in any part of the workflow, not well. You don't need a rocket to render, you need a rocket to render FAST. You don't need a rocket to view redcode, you need a rocket to view redcode at high res full framerate. Finally, a rocket without fast storage is problematic, the storage will limit the render speed even with a rocket. As Jeff notes you don't need crazy fast, just fast. But from where I sit, this is not even a debate.

[Jeff Kilgroe]

To add to my above, since I didn't come right out and say it, I would also go for the storage before the Rocket. Can't fully use Rocket without it anyway. Concentrate on adequate storage for your needs, both capacity and performance, also make sure you have a good backup strategy in place. Then you can augment your workflow(s) with other tools like the Rocket, more software, etc..

[Dan Kanes]

My question:

Are you primarily a camera house or post house oriented person?

If you are camera oriented I'd say buy the rocket! Make clients pay for storage. If you make most of your money doing color correction or editing then I'd agree with everyone else and say focus on storage first. Which tool will help you make the money to get the other piece of the puzzle faster?

[sander kamp]

As long as your drive is not fragmented even a USB 3.5" drive will do for playback. Add another drive on another bus and that's about as fast as it will get for rendering. As said Red footage tops out at 40-something Megabytes/s so that and some headroom is all you need.

Now for archiving huge amounts of footage or Epic, speed will come in handy for sure.

(Yes, I have a Rocket and render footage all the time)

[Jeff Kilgroe]

USB is painfully slow, not sure how anyone could recommend it for RED workflow. I guess if that's all you have, then that's it. But IMO, it's worthless for use where time is a factor, offloading, etc.. USB3 is gaining some ground, but frankly I would prefer it just go away. It's still no faster than eSATA overall and has much higher latency. USB2 tops out at about 45MB/s maximum throughput, which is not enough to guaranty real-time playback of some of the higher REDCODE modes we have now, let alone what's coming up with Epic. FW800 is twice as fast and most of us here still feel it's too slow for any performance or time-critical task. We offload 16GB RED CF media via eSATA readers as fast as the cards will physically allow us. USB2 is not up to the task, FW800 typically is a bottleneck for full CF card images. For doing verified copies, every second counts. Even when sitting at my desk on my own time, I don't want to wait a second longer than I have to.

As for transcode write speeds, depends what you are transcoding to. In addition to the read requirements, you need an additional sustained 250MB/s if you want to write 2K DPX in real-time. If you're transcoding to 2K ProRes, 50~75MB/s sustained writes will be sufficient on even the fastest Mac Pro right now because of the inefficiencies of the codec itself and you won't get all the way to real-time performance, even on the latest CPUs, due to these inefficiencies.

But what I'm really saying here is that USB is a bad way to go for storage. If clients are giving me storage, I tell them up front that if they're giving me 800GB of R3Ds to transcode and they hand it to me on an external USB drive, it will take about 5 hours just to read all the data off that drive, so don't expect same-day turn around on the job. Whereas if it's a decent 1TB eSATA drive they give me, I can offload all the data from it in about 90 minutes. And yes, we offload and log all such client jobs into our systems before going to work, having both online masters as well as work files. If the client hands you a 1TB USB2 HDD and you do the transcode straight from that, you're making a serious noob mistake.

There are lots of considerations here. We're not even sure what requirements the OP has or what he would like to do. Different people have different priorities, I know several who don't have an interest in the RED Rocket or faster storage. They don't mind their transcodes taking all night or even a couple days. To each their own....

[Mike N]

Thanks for the answers guys

I'm basically trying to figure out what are the data outputs of the Red workflow and see if I can get away with it in most cases by owning a good RAID system and bypassing the Rocket...for now.
Or if owning a Rocket would make a fast storage somewhat redundant, since you can transcode your footage super fast to some low data rate codec and then your default storage will be able to handle the problem. But that, I think, would limit my choices only to compressed lossy codecs, right ?

After reading your answers I think my assumption was correct. As i see even if you own a Rocket there s not much you can use it outside a Redcine-x or Scratch for now.
I will probably use CS5 or Avid for cutting and AE beefed up with some plug-ins for color correction. After things get moving i'll probably want to move to more high end software.

So to sum this up. For playing back realtime at full res only the R3D files in a program like Redcine-x,Avid or CS5 i would need about 100MB/s sustained read speeds.
To transcode realtime to DPXs or TIFFs I would need over 200-300MB/s sustained ?

The CS5 and AE Cuda support only works for playback or also for transcoding and effects processing ?

Which RAID systems do you recommend. Can I find some decent internal solutions, like the controller included with the motherboard ? or should I go with separate PCI controllers or external solutions ?

I'm new to the storage stuff if you could recomend a few brands so I can do some research it would be great. A company that I liked is CalDigit ar they at the top of the game or somewhere in between ?

thanks