Storage

[Stacey Spears]

This might belong in the main workflow thread, but I am going to post it here for now.

What is everyone using or planning to use for storage in their Adobe workflow? Medea, San, Fiber, SCSI, SATA II, etc...

I am looking at a Burly rack mount myself: http://www.burlystorage.com/ccp0-pro...Burly9RPM.html

I am looking at a 3ware RAID controller: http://www.3ware.com/products/serial_ata2-9650.asp

As far as drives, I will hold out until I get closer to receiving the camera and hope that the cost of the 1 TB drives comes down. If not, I will get eight 750GB drives.

[Kevin Halverson]

Are you asking about archival or working storage? For archival I use DLT. For working, a combination of SATA, SCSI and NAS. I haven't purchased any 1 TB drives yet, but I have been using 750's for about 18 months now without any complaints.

[Edgar Pitts]

We are going with an all Dell solution. For us the reliability and support are worth the slight premium in price.

3 x MD1000 with 500GB Sata II drives in Raid 5 yields 18TB usable:

http://www.dell.com/content/products...=04&l=en&s=bsd

Powerconnect 6248 Switch:

http://www.dell.com/content/products...=04&l=en&s=bsd

PowerEdge 2970 Server:

http://www.dell.com/content/products...555&l=en&s=biz

PowerEdge 24U Rack:

http://www.dell.com/content/products...=04&l=en&s=bsd

Total Cost: $35,000

[Kevin Halverson]

That's an impressive storage solution. I am wondering how many users you are supporting and what types of applications they will be using. Are you a post house?

[Ben Feuer]

I'm probably gonna rock some combination of these;
http://www.g-technology.com/Products/G-SPEED-es.cfm

You get a 15 percent discount if you're a Digital Cinema Society member, which is extra nice.

[RivaiC]

For GRaid - But it's OS X only ? no XP support ?

[Mike McCarthy]

Wow I am dissapointed that the rest of this thread is gone. It was a really interesting. Luckily I have a txt backup of my really long post.

[Edgar Pitts]

Wow I am dissapointed that the rest of this thread is gone. It was a really interesting. Luckily I have a txt backup of my really long post.

Please repost!!! I too was disappointed. I found some of it in the Google cache, but not all. Thanks.

Edgar

[Kevin Halverson]

I was told that there was a HD failure on the reduser server. The backup is apparently not as up to date as one would have hoped for. If you do have a .txt backup, it might be worth posting, there was some great information in this thread.

[Edgar Pitts]

This is what I could pull from the Google cache. It is missing some from the 1st and 2nd pages.

-----------------------------------------------------------

Quote:
Originally Posted by McCarthyTech
I guess it would be good to answer the original question of the thread as well.

I use a number of solutions on 6 different stations.
4TB Huge Raid from Ciprico (10 400GB IDE drives in RAID50 connected with Dual SCSI 320 via ATTO card)
5TB ProAVIO Array (14 400GB SATA Drives in RAID 50 connected with Dual SCSI 320 via LSI card)
2x 2TB Internal SATA Arrays (8 250GB SATA Drives in Raid 5 connected to Broadcom RaidCore PCI-X card)
2x 1.2TB Internal Arrays (3 400GB SATA Drives in Raid 0 to integrated NForce Raid)
1TB Internal Array (4 250GB SATA Drives in Raid 5 to Promise 4200 PCI-66 Card)
Next purchase: 8TB Fibre array, with 4Gb SAN infrastructure (16 500GB SATA Drives, ATTO and QLogic HBAs)

McCarthyTech, you have a very interesting case there with different configurations under the same roof. Could you provide a read/write benchmark for each of these configurations? I would be interested to know both the burst and sustained abilities of each.

khmuse

--------------------------------------------------------------------------------

Quote:
Originally Posted by khmuse
McCarthyTech, you have a very interesting case there with different configurations under the same roof. Could you provide a read/write benchmark for each of these configurations? I would be interested to know both the burst and sustained abilities of each.

I have found that Blackmagic has a great benchmarking tool, but requires their hardware in system. Other benchmarking tools require you to set a block size. I usually get the best performance withabout 4MB blocks, but I have no idea if that emulates real world performance. Anyone know the average I/O block size when editing AVI files? Or DPX? If I can get an answer to that, I can benchmark them all for you guys. I assume you don't just want the optimal transfer rate if that is at an unrealitstic setting.

As a rule of thumb, my dual SCSI solutions provide 400MB/s, and my 1TB SATA arrays around 200MB/s. I am anticipating my dual channel 4Gb fibre SAN to be around 500 MB/s, but that will be shared to at least 3 systems.


McCarthyTech

--------------------------------------------------------------------------------

Quote:
Originally Posted by Edgar Pitts
Could you please provide companies that you would recomend?


Rorke, 1Beyond, Apple (heaven forbit) and a few others, will provide workstation SANs, while Dell, HP, or IBM only provides SANs for servers and datacenters. Some of those products have requirements like "Unix only" or whatever, making them useless to us. There are probably 20 companies that make shared SAN solutions that are optimized for media production. I can't think of many more at the moment.


Quote:
Originally Posted by Edgar Pitts
So if the FC switch has 12 ports, a quad link to the array would leave 8 available links to the workstations (2 dual links and 4 single links). What do mean by connected with faster interface?


I meant more channels for the array than the workstations. Ideally as many channels for the array as the workstations total, butthat rarely is feasible. Your situation seems best served by initially using single links to stations, and dual linkto the array. The first upgrade past that would be to double the array performance by adding another whole array, which would be 2 more channels on the array side, balancing the 6 workstation channels. Only after that would I worry about dual channel to any of your workstations.


Quote:
Originally Posted by Edgar Pitts
If you are smart about the dual channel array you purchase now, it the future you should be able to purchase another identical one, stripe them together, and boom, you have quad channel solution, with twice the capacity, making use of all of your original investment, but now supporting uncompressed 2K to 4 or 5 stations at once. Could you please elaborate on this?


You can use Raid 0 to aggregate the performance of multiple arrays. Usually large arrays are split that way already. I am buying an 8TB array, which will be divided into to 4TB halves, one on each channel of fibre. My workstations will stripe data between the two halves for double the performance of a single half. If I buy another physical array, I will have four 4TB parts to stripe together, doubling the performance again. That would be 16TB equally devided over four channels of Fibre, returning about 1600MB/s if I am lucky. A single workstation with a quad channel HBA could recieve this, or 4 workstations with single channel HBAs will never be effected by each other. If I have 4 workstations each with dual channel HBAs, they each have a peak rate above 800MB/s, but not all four can achieve that at once. See how that works?

Basically buy extra ports on your switch, and make sure your specific model of Fibre array is going to be available for a while. You may want to buy dual channel HBAs even if you are only going to initially use single channel. Otherwise single channel HBAs will be obselete to you once you ungrade.

McCarthyTech

--------------------------------------------------------------------------------

McCarthyTech,

Do you have an opinion on RAID 5 vs. 6 in this scenario?

Any opinion on SATA brands? Maxtor, Seagate or WD?

Stacey Spears

--------------------------------------------------------------------------------

I've heard good things about Terrablock...any thoughts?
http://www.facilis2.com/

zeke

--------------------------------------------------------------------------------

Quote:
Originally Posted by McCarthyTech
I have found that Blackmagic has a great benchmarking tool, but requires their hardware in system. Other benchmarking tools require you to set a block size. I usually get the best performance withabout 4MB blocks, but I have no idea if that emulates real world performance. Anyone know the average I/O block size when editing AVI files? Or DPX? If I can get an answer to that, I can benchmark them all for you guys. I assume you don't just want the optimal transfer rate if that is at an unrealitstic setting.

As a rule of thumb, my dual SCSI solutions provide 400MB/s, and my 1TB SATA arrays around 200MB/s. I am anticipating my dual channel 4Gb fibre SAN to be around 500 MB/s, but that will be shared to at least 3 systems.


Thanks for the offer. I would be interested in three measurements. Say a 1 MB, 10 MB and 50 MB block sizes. Both burst (say 10 - 50 blocks) and sustained (at least 30 seconds or more) would give a great sense of how much real world performance can be expected. I have also had great success with SCSI configurations on both PCs and SGI workstations.

khmuse