10Gb storage solution

[Kaku Ito]

Today I got the Small Tree GraniteStor storage solution working with help of Chris from Small Tree.

I briefly tested that it can allow playing 10bit 422 uncompressed HD from two Mac Pro clients with 10Gb connections at the same time. Also, the configuration allows 24 1Gb ethernet connections aside from two 10Gb connections, so I'm hoping that iMac and Mac Book Pro can do some offline level editing using this storage system.

It seems quite simple where you connect the GraniteStor RAID to Xserve via ATTO R380 SAS connection, a dual channel 10Gb Small Tree card, then specially configured x24 1Gb switch with x2 10Gb connections in the back (although one of them is connected to the xserve, but the second port on the 10Gb card on Xserve can act as a connection to a client, thus it provides two 10Gb connections with two clients), then the client can either separately connected to the 1Gb connection of the switch with the second 1Gb ethernet port on Mac Pro (the first port can be normally connected for your infrastructure) or install 10Gb card to your Mac pro for uncompressed workflow. 1Gb ethernet connection on Mac Pro provides multistreams of full HD Prores422HQ realtime playback.

I received everything the day before, then received the Xserve yesterday. We connected everything in the day today, and Chris helped me using iChat and screenshare to setup the switch and the clients right, but only took less than an hour to setup.

I will give more reports on this system especially with R3D editing in few days.

[Kwan Khan]

Thanks for sharing,
looking forward to see....

[Kaku Ito]

We were using it simultaneously from two Mac Pros with 10Gb connections doing uncompressed 422 HD and a couple of Mac Book Pros with 1Gb connections doing DVCPRO HD (1080i) and this thing holds up. You can also do M size R3D proxy editing no problem.

Only thing is it is based on AFP so I had problems rendering larger than 2.15GB file from ROCKETcine-X. This is due to ROCKETcine-X using older QuickTime API. Deanan mentioned that it might be able to work around somehow so I'm going to get on this problem with Small Tree people. Since the same thing can be done with P2 to QT out in FCP and other apps, I'm pretty sure something can be done.

[Kaku Ito]

About AFP 2GB file limitation out of REDLINE can be worked around by mounting the volume as NFS. I suggested that to Small Tree and they think it will work. I'm going to ask Chris to help me setting this up so we can report something here.

[Nook Kim]

Kaku,

Could you share the price point of the system you're testing as well as some specs of the unit?

Can you test HDCAM tape out from a 1080 10bit Uncomp on one machine and Rocketcine-X 1080 10bit Uncomp Export on the other machine at the same time? How about 29.97fps on both?

How many 1080 10bit Uncomp tracks can you do in FCP?

Does it allow you to work in Color with R3D RAW while playing in realtime?

Sorry for too many questions, but I'm looking for a storage solution. If all of my questions are yes'd, I'd just hook up a LTO drive to it and live a couple of years peacefully.

Thanks,

[Kaku Ito]

Hello Nook,

That's exactly where I'm going so let you know how it performs.
I'm waiting for Chris from SmallTree to help me setup the NFS share right now.

[Nook Kim]

Kaku,

Thanks for your contribution :)

[Kaku Ito]

Nook,

I haven't checked that simultaneous test, but Chris from SmallTree helped me to set up the SmallTree GraniteSTOR on Xserve with 10Gb ethernet connection work with NFS, so we don't have the 2GB limitation on rendering. Yeah!

[Jeremy Neish]

We have a similarly configured editing server. Works great up to a point, but if you push too hard it can become unstable. After a lot of experimenting, we found the weakest link to be the built-in AFP server in Mac OS X Server. It lacks robust multi-threading, and would conk out or become unstable if we pushed it too hard. It turns out the AFP daemon is actually a relatively old and inefficient open source project that apple modified and incorporated into Mac OS X many years ago.

After some searching we ended up purchasing HELIOS EtherShare. This product was originally designed for the print industry, but the server software is VERY robust and supports multi-threading multi processing well.

We can do simultaneous Prores editing on 6-8 machines with no delays or dropped frames over 1Gb ethernet. In addition we have around 15-20 non-video editing machines (designers using Photoshop, and Developers) all connected to the same servers. Also, Ethershare is just a server side solution, you just use the built-in AFP client on your editing machines.

Come to find out, Apple uses HELIOS internally quite extensively.

And yes, the 2GB AFP file size limit on all of RED's software is very aggravating.

[Kaku Ito]

Jeremy.

Thanks for sharing your configuration. Hope the AFP issue gets fixed, but for now we can work around with NFS. I had clients prefer using HFS anyways, so we'll see how it goes. SmallTree recommends 1GB RAM for each active clients and separate ethernet connections from your infrastructure. Good to know there's other solution to it, too.

The configuration I purchased includes the ability to have two 10Gb ethernet connections. So far 10Gb client was able to playback 10-bit 422 uncompressed HD/1080i and Prores4444.

I have metaSAN running with Infotrend RAID drive, too, but it's quite costly in the beginning and also to expand, so I'm hoping 10Gb/1Gb ethernet solution will be serving us well.

We have a similarly configured editing server. Works great up to a point, but if you push too hard it can become unstable. After a lot of experimenting, we found the weakest link to be the built-in AFP server in Mac OS X Server. It lacks robust multi-threading, and would conk out or become unstable if we pushed it too hard. It turns out the AFP daemon is actually a relatively old and inefficient open source project that apple modified and incorporated into Mac OS X many years ago.

After some searching we ended up purchasing HELIOS EtherShare. This product was originally designed for the print industry, but the server software is VERY robust and supports multi-threading multi processing well.

We can do simultaneous Prores editing on 6-8 machines with no delays or dropped frames over 1Gb ethernet. In addition we have around 15-20 non-video editing machines (designers using Photoshop, and Developers) all connected to the same servers. Also, Ethershare is just a server side solution, you just use the built-in AFP client on your editing machines.

Come to find out, Apple uses HELIOS internally quite extensively.

And yes, the 2GB AFP file size limit on all of RED's software is very aggravating.