Cheap eSATA RAID solution

[Rick Darge]

So this is what I'm thinking about doing...

Buying the 4bay Firmtek + 4host eSATA PCI-X card $650

http://firmtek.com/seritek/seritek-2eEN4/

Going to newegg and picking up 4 500GB SATA drives at $140 a pop, $560 for 4

http://www.newegg.com/Product/Product.asp?Item=N82E16822144016


Then I would stripe these 4 together as a Raid 5 - Roughly 1.5 Terabytes of space..

1.5TB = 15 HOURS of RAW 4K if I'm not mistaken

Nice thing about this setup is after I fill up 1.5 TB, I can shelve all 4 drives and drop another $560 for 4 more drives + the firmtek cages at $22 each, making the price roughly $650 for an extra easily swappable 1.5 TB of space.


Crap, this is going to get expensive fast shooting 4k

1.5tb=15hours= $43 per hour = $0.72 per minute +

Still cheaper than film :alien:

[Jeff Kilgroe]

So this is what I'm thinking about doing...

Buying the 4bay Firmtek + 4host eSATA PCI-X card $650

OK, sounds good. Why PCI-X? Considering REDCINE won't run on non-Intel Mac systems and will require a more current PC... Most everything has gone PCI-Express these days.

The 500GB Maxtors are good drives... So are the Seagate and Hitachi units. All perform about the same. Can't buy from a better place than NewEgg either... I love those guys.

RAID5 would be the way to go for reliability. I would consider a straight RAID-0 array for performance though if you plan to do any editing from this array. However, you mention shelving the drives for storage. IMO this is a bad idea. ...I know we've been over this in other threads, but hard drives (even if they are components of a RAID5 configuration) are not designed to sit idle on a shelf for any extended period of time.


Crap, this is going to get expensive fast shooting 4k

This is the other reason swapping out those drives for archival isn't a great idea. LTO-3, DLTS4, and other large format tape solutions are far more reliable for archival and are intended for that very purpose. They are also cheaper than hard drives by a good margin. The tape unit itself is expensive, but once it's cost is recovered, it's far more economical. If I had to buy an archival solution today, I would go for the Exabyte or Dell (ADIC) autoloader units. I can buy brand new Exabyte 12-tape LTO-3 autoloader systems off ebay for less than $3K. Figuring $3500 for the tape system with SCSI interface and cables and $0.15/GB tapes... If you compare that to hard drives ($0.33/GB plus swappable enclosure, interface, gables - $650). The tape system makes a lot of sense.

Compare 20TB of backup capacity -- why I chose 20TB will be immediately clear.

LTO-3: $3500 + $3000 (bulk buy of 50 tapes) = $6500

HDD: 40 * 500G drives = $6500

So the break-even point between the two solutions is about 20TB at about $6500. Lets look at the cost for the next 10TB. Oh, and we'll assume that the price of 500GB HDDs has dropped another 25% in the mean time.

LTO-3: $1500

HDD: $2435

With the amounts of data we'll be working with in 4K, 20TB isn't going to be archiving that much considering original "negatives" or RAW files, plus intermediate data, projects, finals, etc.. 1TB of drive storage is roughly 10.5 hours of REDCODE RAW 4K @ 27MB/s. The first 20TB (210 hours) of archived footage equates to about $31 per hour. From that point forward, LTO3 or DLTS4 would equate to about $7.15/hour.

We also have new tape formats coming up like DLTS5 and LTO-4 which will double capacities and keep tape prices roughly the same. So expect tape offerings to go from $0.15/GB to aout $0.09/GB within the next few months.

[istvanttt]

well you can also consider that the LTO-3 tapes occupy less physical space and have less weight when you have to ship them.

[Curran Giddens]

http://firmtek.com/seritek/seritek-1en2/

I got 2 of these FirmTek SeriTek/1EN2 Dual-Bay Hot-Swap External SATA Enclosures a while ago. My PCI-X card is a Sonnet Tempo 4+4. I use Seagate Barracuda 7200.8 400 GB drives. I would not buy this same setup knowing that Redcine won't run on my G5. I will wait until the new dual quad-core intel Mac Pro's are available and get a PCI Express card with RAID array.

I brought this up on the old dvxuser RED forum:

If the RED-RAID (20-40 drive mini-fridge) had drives that were user-replaceable (doesn't need to be hot-swappable), you could use it as your desktop RAID. When you need to use the RAID for 4k 60 fps or 2k 120 fps, you could just swap out the drives for another set. Just make sure you put the drives back the same order.

I hope the Holographic storage comes down in price quickly. It is supposed to have a long shelf-life, not to mention random-access, high throughput, and high-capacity.

[MikeCurtis]

1.) You'll actually get less than that - 1387GB formatted capacity

2.) Nowhere NEAR fast enough for uncompressed RAW

3.) It makes no sense for you to work in uncompressed RAW if you're on that tight a budget - you're shooting/transferring compressed, so roughly 100GB/hr, and THAT would be a datarate you could do.

4.) Are you Mac or PC? If you're Mac, you have no RAID 5 options with that card. There's no software based RAID 5 for Macs.

5.) Yeah - gotta be on a PCIe based Intel Mac system for Redcine

6.) and if you had 4K footage, how would you monitor and output it? Shoot 4K compressed RAW, then downconvert to HD for lovely results that you could potentially output and watch in a meaningful way. But your 4 drive RAID still too slow for that, probably even at RAID 0.

Not meaning to sound discouraging, just realistic.

-mike

[Rick Darge]

I should have been more clear. I'm not looking to edit in 4k by any means, even when I get my 8-core 8GB Intel-Mac at the end of the summer. I'm just looking for a practical solution of where to store all this data coming in. That's why I was thinking temporary storage would be on a RAID 5 1.38 TB - When that filled up after projects were finished, I would archive to tape or something else. The more I think about it though, the more this whole post idea scares me. we're talking HUGE amounts of data

What's your setup like or going to be like after you get the RED Mike?

[Sean Michael Johnston]

I'm looking at this:

http://www.macgurus.com/productpages/firewire/Burly1FirewireHS.php

or similar for backup. We'll be buying a few of these and lots of $20 sleds. HDs are about $100 for 250gig and $200 for 500gig.

I was ready to jump on the LTO path. But after talking to a few people who have been using these and researching the software required to run the hardware I realized that even though the capacity has grown, the limitations of a linear storage medium have not. These solutions are not much better than they were 10 years ago when we used DLT to backup our AVID.

A hard drive costs a little more per gigabyte, but when a client calls and asks[demands] that they want to revise that project you did last year and they need you to send a low res quicktime for them to review now. Do you really want to load up an LTO tape[or what if you striped a project across 4 tapes in an autoloader?], run Retrospect and spend an hour or more resurecting an entire project?
Wouldn't it be faster to pull a hard drive off a shelf, plug it into a hot swap drive bay and access that file immediately?
There are also a few good disc catalogging programs that are shareware or freeware that do a good job of keeping track of which hard drive I need to pull down off that shelf. We've been using one of these with DVDRs for years.

[Jeff Kilgroe]

I was ready to jump on the LTO path. But after talking to a few people who have been using these and researching the software required to run the hardware I realized that even though the capacity has grown, the limitations of a linear storage medium have not. These solutions are not much better than they were 10 years ago when we used DLT to backup our AVID.

Well, there's a few things about tape that are being ignored here. Autoloaders don't run striped volumes. They're spanned from one tape to the next. Firewire isn't all that fast and individual hard drives aren't going to be much (if any) faster than reading tape. If you have proper backup procedures in place, you can pull the proper tape (or tapes) from your archives and load them just as fast as you could from individual hard drives. Sure, tapes are linear, but a proper tape system with good backup software will tell you which tape you need and it will fast forward right to the data it needs.

I know that a lot of people have a dislike of tape systems... I know several IT guys that detest tape just because it's tape. Their opinions have nothing to do with how reliable or practical of a solution it is and they continuously go on and on about how other storage media is far superior. I've got a buddy that preaches holographic storage as if it's going to save the world. Fine, great, sounds good... Where can I buy it? But there's a real flaw with that logic. Other than hard drives, no other media provides equivalent capacity and nothing touches tape in terms of TCO and ROI numbers when you start considering large volumes of data. Personally, I would feel very comfortable if I have to pull data from an LTO-3 tape 5 years from now. I would not feel so good about pulling data from a 500GB hard drive that has been boxed up for that long. And in a year or two after I have been using my RED One and I've got over 100TB of archived data. I know that I'll be happy I don't have $30K invested in hard drives, but rather $12K in a format that is designed from the ground up to sit on shelves with fewer moving parts and less chance of magnetic dispersion and corruption of the data itself, which tends to happen over extended periods of time on hard drive platters. Not to mention the motors, internal lubricants, head mechanisms, etc.. that have just been sitting there for 5 years or more... Before the next great backup format comes along (holographic?) and I can revise and condense all the tapes to something more efficeint and hopefully something more reliable.

I do consider tape to be more reliable than HDD... I've been using both for years and have managed large SANs and helped run datacenters. While I love the idea of hammering that final nail into the proverbial coffin for all tape formats, that day just isn't here yet. When it comes to long-term archivial of tens and hundreds of terrabytes, even petabytes and zetabytes of data, the world still operates on tape. Large companies have various levels of backup solutions and ways to get at their data. But some of the biggest out there, like the library of congress for example, still uses tape as their final archival step and will do so for the forseeable future.

For people planning to shoot a lot of video with RED (200+ hours per year), they would be best served to have a SAN in place for short term backup, project workspace, etc.. Archival to a common tape format, perhaps consider secondary backups of the really important stuff to additional tapes and/or smaller projects to optical media as well... If the price of optical media ever comes down. Right now a 50GB BD+RE disc is the same price as a 400GB LTO-3 tape.

[Sean Michael Johnston]

Thanks Jeff, do you have an LTO system in house now?
I'm still on the fence about which solution we're going with. We'll probably be living with it for a long time. Any user feedback is much appreciated.

[Jeff Kilgroe]

No LTO system here now, but I have used them and some of the other sytsems like DLT-S3. Currently everything here is on RAID-5 storage accessible via SAN. My SAN consists of an XServe RAID and two other PC nodes each with a RAID-5 array. I'm using a VXA-2 tape system for archival (160GB uncompressed per tape) for archival of my animation and video - mostly DVCPROHD via HVX200. A single slot VXA-2 drive isn't going to cut it for RED and its data requirements and I plan to buy something new when my RED ships. I'm planning on either LTO-3 or DLT-S4, but hopefully LTO-4 or DLT-S5 will be available later this year in time for my RED. As for workstations, I have a couple G5 quads, a C2D Macbook Pro and a few PCs. My latest PC is an 8-core 2.33GHz system I just built about 6 weeks ago. I plan to buy a new 8-core Mac Pro and retire one of the G5 quads to render farm duty sometime later this year. Most of my workstations are configured with a system drive and two additional drives in a RAID-0 stripe for workspace, etc.. My primary G5 quad just has two 500GB drives in a RAID-0 stripe as its primary and only HDD volume. I keep a few external fw/usb2 drives handy as well.

[Nik Manning]

I think the perfect storage system would be a 3.5 inch drive version of the iomega rev drives. Then they could hold up to maybe 1tb per platter.

[Stokestack]

Good posts, Jeff. I've wondered about the economics of tape vs. hard drive. For personal backup and long-term storage, I'm going with cheap hard drives all the way. But your math on the tapes (assuming it's right) is pretty informative.

You bring up some things you worry about with hard drives, but without studying the matter in detail over time, I have alternate worries about tape.

1. The degradation of the binder that holds the oxide to the tape, compounded by tape's flexible nature and its travel over curved surfaces. Audio tapes made during a certain period later started to shriek when played back because of (the absorption of or loss of) moisture in the binder. It was probably absorption, because the only way to recover the recording was to bake the tapes at low temperature for hours or days, and then you had one opportunity to play the thing out and copy it. We have better compounds today, but the concept inspires concern.

2. Exposure to magnetic fields. Hard drives (even today) are housed in stout metal enclosures. Maybe these tape cartridges are shielded, but if not, I'd worry about magnetism more than I would with a hard drive.

3. Physical damage, like stretching.

In a tangent point, someone mentioned products like this:
http://www.macgurus.com/productpages/firewire/Burly1FirewireHS.php

This product suffers from a glaring design omission that is rampant and aggravating in this product arena: no eSATA connector! I'm using SATA drives for a reason! Given that my computer has SATA, the drives are SATA, and this standard is likely to grow in popularity (despite the baffling failure to include POWER in the connection standard)... why would you omit a SATA connector from the enclosure?

I've been shopping around for a long-term strategy for hard-drive-based archiving, and vendors' failure to recognize the importance of having all three connectors (eSATA, Firewire, and USB) on any SATA-drive enclosure kills lots of otherwise promising products.

[Charles Perkins]

i'm looking into raids atm and at the low end, i'm looking at this: http://www.macgurus.com/productpages/sata/Burly8RHS.php filled with either 500 or 750gb HDs.

has anyone used these, i've never heard of them before but it looks ok.

[Rick Darge]

that looks pretty nifty chas - haven't used it, I use the firmteks currently and love them

[Simon Blackledge]

Any reason your going port multiplied and not multilane?

These are good.
http://www.proavio.com/eb8pm.html

Get them in the UK here.
http://www.scan.co.uk/Products/ProductInfo.asp?WebProductID=480110

[Charles Perkins]

no reason for port multiplied over multi lane. whats the diffrence between them?

also, could i use two of the sonnet 500p's together to creat a 10 drive raid?

[Andrew M.]

I've been shopping around for a long-term strategy for hard-drive-based archiving, and vendors' failure to recognize the importance of having all three connectors (eSATA, Firewire, and USB) on any SATA-drive enclosure kills lots of otherwise promising products.

Why you will not consider Blue-ray Disc as a good long term archive.
At this moment double sided BD has 100GBytes of capacity. Next year it will be 400GB
I agree, it requires segmenting, but you have to load it in to the computer for processing any way so I bet you can load 800GB data from 8 BD faster then from the tape?

[Ivan G]

At this moment double sided BD has 100GBytes of capacity.

I thought a double sided BD was 50 gigs and not 100. I have heard of hologram discs that hold 300 gigs

[Andrew M.]

The dual layer BD is 50GB the two sided dual layer is 100GB
Don’t mix dual layer with two sides, normally you have label on the second side but for archive purposes you record on both sides, put 4 or 8 of them in the box and you label the box. Don’t forget to put the discs numbers on the inner, unrecorded part of BDs since you do not have room for label anymore on two sided. Also make sure that you include label in the title on the disc data.

http://en.wikipedia.org/wiki/Blu-ray_Disc

http://www.supermediastore.com/blue-ray-blu-ray-dvd-media-center.html
http://www.tdk.com/procommon/press/article.asp?site=con&recid=140

[Jeff Kilgroe]

Why you will not consider Blue-ray Disc as a good long term archive.
At this moment double sided BD has 100GBytes of capacity. Next year it will be 400GB
I agree, it requires segmenting, but you have to load it in to the computer for processing any way so I bet you can load 800GB data from 8 BD faster then from the tape?

The problem with BluRay as a long term archival solution (right now) is that cost per GB is very high. Additionally, the shelf life or reliability of BD+RE media has not been proven. Even up to a few years ago, we were hearing all these claims about optical media having a shelf-life of 100+ years. However, many are now discovering that most common CD-R media that made such claims 12+ years ago are now failing. Most manufacturers have changed their tune about shelf-life and only claim 25-30 years with current CD/DVD media. Sony claims 25 year shelf-life with their current BD media.

Realistically, even a long-term archival solution only needs to last 8 to 10 years. This allows adequate time for data storage solutions to increase several times over and archived data collections can be re-organized and consolidated onto more current media.

Anyway, back on the subject of BluRay... It may make sense as an archival solution in a couple years, right now it doesn't. And I'm not sure what you're saying about 400GB. Not going to happen. The BluRay spec can only support up to 12 layers per disc. That's just how many they can physically pack in there and even the Sony reps I've talked to at various presentations expressed doubts (based on what their tech guys have told them) that BD will actually reach 12 layers and still be readable from one side of the disc. Dual-sided, 6 layers per side should be do-able. But either way, BD media tops out at 300GB, which translates into 291GB of real-world storage.

Holographic discs will probably replace BD media in a few years anyway. Current low-density proof of concept devices pack 800GB on a standard CD/DVD size disc without using multiple layers. But we're still at least a year away from being able to buy a holo-writer.

But to revisit the cost per GB for BD media... Currently there's 100GB discs (dual layers on each side of the disc). But I can buy a 400GB LTO3 tape for the same price. It's actually cheaper to just buy the 50GB single sided BD discs as they're under $30 now in bulk.

[Andrew M.]

go to my last link TDK news.
200GB per side gives you 400GB for two sides.
I agree with you that we need maximum 10 years.
If you take all the tapes you have that are 10 years old and you put it on the BD probably 10 BDs will hold all your 10 year old library.

[Mark B.]

More important than the storage space, in my mind, is that the holographic storage is going to be designed with an exterior casing around the disc surface (like the old 3 1/2 discs from the past). Scratches and dust on the data surface won't be much of a problem that way.

[Jeff Kilgroe]

go to my last link TDK news.
200GB per side gives you 400GB for two sides.
I agree with you that we need maximum 10 years.
If you take all the tapes you have that are 10 years old and you put it on the BD probably 10 BDs will hold all your 10 year old library.

Er... I did click the link and I've already read that a few weeks ago anyway. Mentions nothing in there about 400GB discs or layer configurations. 200GB discs (like these from TDK -- TDK is the primary maker of Sony branded BD media too) were demonstrated at CES with Sony's new quad-layer BD writer. They can put 200GB on a BD by using 4 layers on each side, so you still have to flip the disc after you record the first 100GB. 8-layer readers/writers, or what will be needed to put 200GB on a single side have not even been constructed in a test lab scenario yet. Sony has even stated that they doubt the ability of BD writers to expand beyond 4 layers at this time with the current diode lasers they're using, they will probably need a more powerful laser system.

BluRay DOES NOT allow for 200GB per side and never will unless Sony, Pioneer, Philips, Samsung and Matsushita all agree to revamp the standard and allow for a thicker disc. 12 layers is the maximum number of layers in a disc. So any way you slice it, that's 300GB per disc, not per side. Once all 12 layers are there, you can't magically bond another 12 layers to the other side of the disc. Now you can make double sided discs, but with the physical space constraints and substrate thicknesses, you're still limited to 12 layers... Divide them up however you want, 8 on one side, 4 on the other or 6 and 6, 2 and 10, etc.. So if you have a BluRay disc with 200GB on one side, the most you can put on the opposite side would be another 100GB.

Anyway that said, it's still early in the game and Sony and friends could decide to add another 0.475mm of thickness to the disc spec and jam another 4 layers in there to give you a 400GB disc... All accessible from one side or as 200GB on each side or whatever.

[Jeff Kilgroe]

More important than the storage space, in my mind, is that the holographic storage is going to be designed with an exterior casing around the disc surface (like the old 3 1/2 discs from the past). Scratches and dust on the data surface won't be much of a problem that way.

I don't know about the casing/caddy... Sony dropped the caddy design for BluRay (they kept it for XD media, BD was the same, just larger) because it would require more parts to support both BD media and current DVD/CD media on the same device and because of manufacturing costs.

Of the holographic media types demonstrated publicly thus far, some have a case/caddy some do not. Some aren't even based on the rotating disc paradigm.

[Andrew M.]

Yes, in the article TDK do not mentioned if it is single side or dual sided 200GB disc.
I presumed single side 6 layers at 36.7 GB per layers.

[Stokestack]

Why you will not consider Blue-ray Disc as a good long term archive.

Because the organic dyes in writable optical discs don't last. I would never archive anything on these things. I had some name-brand CD-Rs that were totally worthless after about eight years. And who knows when they actually went bad? Probably long before that.

With the ever-falling cost of hard-drive storage and the low capacity, poor reliability, and slow speed of optical discs, I don't see that there's really any contest currently.

The optical-disc market has been riddled with blunders from the beginning. Multi-session CDs never really worked. CD-RW never worked as promised. After all that, the release of DVD without a working recording strategy was inexcusable, compounded even further by this asinine DVD+/-R fiasco. And I don't expect recordable "HD" discs to be any different. And now we hear talk of adding layers, but where is the discussion of how current players will support this? If they don't, then it's worthless.

[Andrew M.]

I didn’t know that!
But I have to tell you that I moved to optical storage only after I couldn’t access some of my old hard disks that I never touch for 5 years or so.
I was told that for hard disks you have to re-write every bit, including fill in bits and sector marks every now and then to make it last.
I guess I could design refresh/read/write program to do it on the magnetic storage.
For tapes though, rewinding it from end to end used to help but I don’t think it is the case with the current tape technology.
We better do some serious research on this subject.

[Jeff Kilgroe]

Hard drives are not made to sit on a shelf. The data will go bad, it's just a matter of time 1 year, maybe 5... Optical media is far more vulnerable error. Scratches if improperly handled, the dyes degrade over time - especially if exposed to light. That laser that writes to the discs isn't that powerful, leave a CDR on the dashboard of your car for a day or two and you won't have any data left.

For long-term shelf storage, tape is the best format... But it's far from perfect as well. Most archival tapes like LTO, DLT, AIT, etc.. Are well shielded and constructed very well. That's why they're $60+ They do hold up very well long term and can handle some decent temperature and humidity variations. You still have to watch out for moisture with tape, but this is still less of a concern as it would be with hard drives.

Anyway, whatever format we pick, I think it wise to not assume that it will sit on a shelf indefinitely and for anything truly important, more than one backup copy would be a wise decision.

I plan to keep all current projects as well as everything I've worked on for the last year in online SAN/RAID storage. Regular backups will be made to tape -- and I will probably make duplicate archival backups to store off-site for a portion of the data, or all the stuff that I really don't want to lose. If I had to buy an archival solution today, I would just add more RAID storage to the SAN and I would get an LTO-3 autoloader drive for archival. But I will re-evaluate this situation continuously and I'll see what comes of all this by July/August when my RED is close to shipping. Hopefully LTO-4 and DLT-S5 will be available by then giving tape a much higher capacity. Perhaps BluRay will be commonly available in 4-layer (200GB per disc side) for cheap. I wouldn't trust optical media for a primary backup, but as a secondary it wouldn't be bad. Besides, if BD+R media can drop in price to $5 for a 100GB disc, then that would be a great option.

[Andrew M.]

Yes. Looking back, the tape was serving us well, so we trust it.
Tapes though should be re-winded once a year, but as I said check it with the tape handling procedure since my information is 10 years old when I was dealing with university networks. We did use tapes for backups in the computing center. And we had fair share of mechanical or other soft error experience often. I will not recommend tape backup though.
Now I tend to trust optical storage since I didn’t have any bad experience with it yet. It is just my personal experiance.

[Jeff Kilgroe]

I will not recommend tape backup though.
Now I tend to trust optical storage since I didn’t have any bad experience with it yet. It is just my personal experiance.

Understandable. I personally can't recommend hard drives. I've had dozens of drives fail on me over the last 10 years, even lost a 500GB SATA-II drive last week. Motor froze up. Yes, dozens sounds like a lot, but not really. I'd put my failure rate with HDDs probably at about 5% if I had to guess. I've had far more defective pieces when ordering memory sticks and other components. With tape, I have had the best luck. Not to say that I have had no failures. I went through data archive hell with an HP 10GB tape drive about 8 years back.

[Andrew M.]

What I like about tapes is that if the header of the whole tape has error you just loose the volume info of the tape and direct access data information, however all the data is in tact.
Also any error on the tape will just affect the sector with the error and all other data is safe. Wit the hard drives or optical storage, if the header info of the disk has error you can’t continue, you have to use special utility to recover the data and then try to assemble it from the pieces, real puzzle job. Also if the media is not removable, any problem with the motor or electronics makes the whole unit dead. Again you can replace PCB or even the motor but it is data recovery job. Tape has the nice ability to recover from all these entire problems but in turn it is easy to mangle it in a bad tape drive.

BTW what problems do we have when storing rolls of films for long time in the archives?

[zak forrest]

Understandable. I personally can't recommend hard drives. I've had dozens of drives fail on me over the last 10 years, even lost a 500GB SATA-II drive last week. Motor froze up. Yes, dozens sounds like a lot, but not really. I'd put my failure rate with HDDs probably at about 5% if I had to guess. I've had far more defective pieces when ordering memory sticks and other components. With tape, I have had the best luck. Not to say that I have had no failures. I went through data archive hell with an HP 10GB tape drive about 8 years back.

i can relate jeff, because i have had about a half dozen drives fail on me in the past year and im currently in the middle of trying to figure out the best way to archive when i have 2 red cameras soon. i am forever scared of hard drives, even though i think the problems i had were mainly horrible parts in lacie enclosures (anyone else have this problem?), and might not be so bad with other drives. i hope you continue to keep us posted with whatever you decide on upgrading to when you get your red camera.

could you tell me: do you think the same situation you had with the tape/tape drive of 8 years ago could happen today, or has this problem(or whatever happened) been fixed for good in the past 8 years of tape technology development?

only one hellish experience with tape (and so long ago) seems like a pretty sweet track record to me. i think im going to go the tape route. this is a new thing for me, i am not sure what format, or brand drive etc to get, and i dont know what an autoloader drive is either. any suggestions from the crowd?

the archival solution i come up with will be used on a feature film with 2 red cameras and a high shooting ratio, i do not have any clients, i wouldnt be shooting 12 hours straight every day or anything like that. just the film.

[Andrew M.]

From my experience, most long term archives are stored on the tapes.
We are talking large institutions here….hmmm….
I am tempted to use Optical storage though. I use two copies anyway for the archives storage location diversity. On site and off site copy.
If you check the copies every two years, it is very unlikely that both copies will go bad at the same time. If they will go bad, just single one, then we do scrap the idea of archiving on optical media all together.
Alternatively we can move to the optical media with the proven track record then.

Anybody has personal bad experience with optical storage going bad after one year or so?

[Jeff Kilgroe]

I've had CD-Rs go bad after 8+ years... As I mentioned in another related thread around here. They dyes just don't last that long. Manufactured discs and even some types of CD-Rs with metallic compounds in them can last longer. Be wary of those with metal foil layers (manufactured or writable) though as moisture can get inside and corrode that layer.

The only optical media I've had go bad after a year or so are discs that I've mistreated. Like direct sunlight on the dashboard of a car can easily wipe a CD-RW if left for too long. DVD-/+R/RW|DL, etc.. are basically the same thing as CD-R, mostly the same dyes and manufacture processes, just developed for and written with higher density data. This in turn means more chance for error with small scratches and whatnot. Compound that even further for BluRay and HD-DVD.

My big concern right now over optical media is that it's expensive and time consuming to use. BluRay is showing some promise with capacity -- double-sided 100GB discs are available. But even single-sided 50GB (dual layer) discs are still $35 each. And write times are very slow compared to just about everything else out there.

[feb31films]

i'm looking into raids atm and at the low end, i'm looking at this: http://www.macgurus.com/productpages/sata/Burly8RHS.php filled with either 500 or 750gb HDs.

has anyone used these, i've never heard of them before but it looks ok.

I am actually going to be ordering the 2 Bay version of this next week (with a couple of Seagate drives) for some external editing/storage. Got a short project shot in DVCPRO HD that I'll be editing and I will be more than happy to post my results on this forum. I've been looking into this for a few weeks and if you are thinking of going the same route, one word of caution. There are SEVERAL manufacturers of these types of enclosures (Sonnet, Otherworld Computing, MacAlly, etc..) and though nearly all of them will mount the new SATA II drives (3.0 Gb/s) MOST of the enclosures limit the data transfer across the cable to 1.5 Gb/s. That will create a serious bottleneck in your workflow. These BURLY enclosures will allow up to 3 Gb/s transfer speed which is why I am going with them. Make sure you do your research. Happy editing!