Disk Storage recommendation

[Stephen Townsend]

I'm currently running a Mac Pro with just the one HD and I'm editing some RED footage from a USB drive which is unsurprisingly running quite slow - particularly at rendering.

I understand I should really have a minimum setup of 3 disks: 1 for RAW footage, 1 for working on (i.e. running software) and 1 for rendering out to. Is that correct? And that these should all have the fast connections possible?

I'm in the process of transferring the RED footage to another MacPro which is booted up as disk and connected to my main MacPro through firewire (I'm not sure which type so I'd be interested to find out if anyone can tell me how?) in the hope that it will speed up my workflow.

What I'd like to know is what sort of disk setups other people currently have and what might be recommended for me in the future? I understand RAID is something to look in to but there seems to be a lot of different options and since I'm not an actual technician I don't really know a lot about RAID.

I work in a creative agency as a Motion Graphics designer so my role entails a large spectrum of work so although editing isn't my only role I do enough of it to require a faster setup but not at a huge cost.

Any sort of general information to help me understand it all the better really as I can hopefully make my own more-informed decision if I know what's what a bit better.

[martinnoweck]

Hi Stephen, are you editing, archiving and / or transcoding red footage?

anyway - you should def. think about a raid solution, it depends on the answer to my question which kind of raid ...

to start you can look for an macpro internal raid solution.
best wishes and kind regards,
martin

[Stephen Townsend]

It's mostly for editing purposes. I imagine we will just archive to USB drives like we have done for past data so it's more for editing and rendering.

That's something I was curious about actually - whether it's more typical to setup a raid solution on the main editing Mac Pro or have something external that other computers can access. I'm the main editor/animator here but sometimes one of the juniors here works on some of the small jobs so I'm thinking it might be ideal if we can keep all RAW footage on one main storage which can be accessed by both of us easily. That would save a lot of time if I wanted to just pass on a project and all the file structure would remain the same and there would be no need for transferring files (which is particularly time consuming when dealing with red footage of course). My question about that would be, how would the RAID then be connected to both our Macs to maintain a good data rate?

Cheers Martin

[martinnoweck]

Hi Stephen,
it depends on your budget, reading your post i would recommend / think about a fibrechannel (or at least 10-giga bit ethernet) SAN setup, but this is not a cheap solution.
will you and your juniors work with RAW "native" or with transcoded material, for example tiff / dpx sequences, prores or dnxhd files?
best wishes and kind regards,
martin

[Stephen Townsend]

Hard to say at the moment. We're still in the early stages of working with RED which is why our current setup isn't geared to cope with it and it's hard to say what the future will bring. Since we're a creative agency it could be that the next few projects are more animation based anyway so our future usage of RED is undermined yet. We've also done some shoots using dSLRs like the Canon 5D Mark II which can achieve pretty good results.

I shifted the RED footage over to a firewire drive and the renders were still very long. I have a feeling it may be more due to the layering in aftereffects choking the processor than lack of disk speed though but we certainly need to look in to a better disk setup for the future anyway. The company is used to sharing all files over the network (usually just image-based files though) so I imagine they'll be keen to continue that with video footage. Maybe I can persuade them to get a 10-giga bit ethernet network setup when we move to our new studio in the summer!

[Stephen Townsend]

Would I be correct in suggesting that Thunderbolt could well be the future of disk storage? Obviously this isn't going to be suitable for my current MacPro since it doesn't support it but rumors have it that the future MacPros will?

[stu aull]

Tbolt will kick some serious 4K @ss I expect, but you/we await the MacPro version - this after dropping rather large coin on the CURRENT Pro in anticipation of RED. Ah, well.

I think you are prob pretty seriously under-powered to be evaluating RED files in your current setup, which is a shame. Be sad to see you swear off RED w/o a proper Mac to pump stuff thru with! An intermediate step should be considered so you can leverage the $$ you have in your current MacPro (which hopefully is at least the newest Intel version?). I would go thru some of the older posts archived here!! and look at E-sata card/drives or USB 3s. At least these will speed things up for you I would think. Lots of RAM never hurt either. Having to evaluate options here myself jumping from SD to RED, so understand your concerns.

cheers
Stu Aull
Alaska

[Stephen Townsend]

Cheers Stu

It's a good MacPro - 12core 32GB RAM and pretty new but like you say it's more than likely to be the rest of the setup that's the issue! I've got a DIT coming in today so maybe he can help suggest some good solution options.

[stu aull]

Stephen-
following up - what did you end up doing??

Stu Aull
Alaska

[Brian Merlen]

in order to really answer your question we need to know what editing software you use, because although storage is your current concern, it might not be your only bottleneck. a redrocket card, or a new gpu might help. that being said throw some internal hard drives in your computer and use those to store data and render transcodes to. since your using red you might want big drives, say 3tb+. internal will be faster then firewire 8. thunderbolts not an option so dont worry about it. you might as well use internals since you got the bays free. maybe even consider getting an ssd as your main drive for OS/APPS and use others for storage. the answer really depends mostly on your needs, what program you use, and what budget you have in mind. raid solutions are good if you want redundancy and data protection

[Josh Beadle]

2 x G-Speed eS Pro enclosures, Raid 6 with Atto 680 dual port board for 800mbs on the cheap. No need for fibre except in your diet. Thunderbolt is not the answer for R3D's:

http://www.g-technology.com/products/g-speed-es-pro.cfm

. . . but until The Kilgroe speaks - you do not have an answer

[Jarek Zabczynski]

Thunderbolt is not the answer for R3D's

I've heard good things about people using the Pegasus TB unit.

[Stephen Townsend]

Just wondering if anyone has some further input on this and if anything might have changed in the last few months (as it easily can do with todays ever-changing technology)?

The G-Speed eS Pro enclosures that Josh recommended seem like a good option. Just wondering why I might go for that over an internal solution? At first I thought it might because I could connect it up to another machine easily but the only machine I'd be likely to connect it up to would be an iMac which that particularly enclosure wouldn't support. I then looked at the G-Speed Qs which support firewire and usb (http://www.g-technology.com/products/g-speed-q.cfm) but obviously I'd lose a lot of speed when connecting to my MacPro.

[Brian Merlen]

Mac pro ports are just too damn slow is the problem. Esata. SAS, USB3 are your best options if you want to go external, but like you said, if you want to hook it up to an Imac, that is going to be a problem. Unless you get some sort of thunderbolt adapter to hook it up with (if the Imac has TB even)... Internal is a good option speed wise, but not if you want to hook the drive up to other workstations occasionally, which based on your Imac issue seems to be what your trying to do.

This is basically why everyone hates Apple now, forget about the CPU speed bump, how do we hook up the old workstations to the new devices when non of the damn ports match machine to machine... Its a total nightmare right now port wise...

Until the Mac Pro gets thunderbolt added, or the Imac gets USB3 added, its going to cause issues trying to go from one to the other while using fast external storage devices...Hopefully the Imac gets its Ivy upgrade soon, not that it will help you if you got an older one...

Personally I like the new Drobo 5D, since it has USB3 AND thunderbolt I think that will be my new storage device, I like the legacy support that offers compared to thunderbolt only externals like the R6. Plus I believe the Drobo 5D supports 4tb HDD's, which is also a plus storage wise since I can pack a ton of storage in a relatively small device. I also was looking at the rack mount USB3 OWC storage rack, but that's a pain for mobility reasons...