CF as Permanent Storage

[Bing Bailey]

Im wondering what are the economics of using CF Cards as permanent Storage for RED Files

if it costs 400+ for the average 35mm can, then what would it cost to use CF CARDS only once and use them as our permanent storage for the files rather than buying an LTO or TAPE solution. at least we'd be guaranteed that the sizes would go up over time and the prices would come down.

right now its $199 for 4.5 mins of 4k footage, so if you have say an optimistic ratio of 11 to 1 for a 90 min feature. thats 990 mins, 220 8gb CF CARDS would cost

$43,780 which would be your tape stock. but you'd have no processing costs or tape transfers. by the end of this year that amount will have halved with 16gb cards and maybe halfed again in 2009. so end of 2009 with that ratio we'd be looking at about $11,000 for storage. an LTO drive costs in the Region of $4k for the latest single tape drives. maybe another $2k for a decent raid too offload the footage. how long are lto3 o 4 tapes rated to live. LTO tapes cost about $80 a piece. but it takes time to backup. its about 350mb a minute. 2 Terabytes of data on 220 CF CARDS, that would take at least 6 LTO maybe more if you don't use compression. thats about $500

what is the life of a CF CARD you've only used once. does anyone know their rated life. would it give us peace of mind about what is essentially our original digital negative.

[Chris Parker]

in my opinion, LTO-3 is a better choice. WAY cheaper per GB. And it doesn't take that long to download to them. You could easily keep up with a feature shoot, and have the LTO-3 tapes in the hands of the producer by the next day, or at the very worst, weekly deliveries.

[Bing Bailey]

my only concern is I've been in IT for 15 years. I've seen a lot of backup tapes become unusable. in a server you can get over that because you always have the server in its present state or at least an older backup. for footage it could be devastating.

[J.D. Frey]

I'm pretty sure that most tapes are rated at 50 years. You could also go the hard drive route. There are obviously lots of horror stories about drive crashes, etc. but recovery techniques on hard drives are much better than tape recovery in the event of a failure. The only issue with hard drives as along term storage is compatibility. However tape backup is pretty standard, and tapes are also regularly used in HDDVD and BluRay production as a transfer media standard.

With tapes there is always the option to make two copies. I think the price for LTO is around 50 bucks a tape, so not bad.

J.D.

[Chris Parker]

yep. would seem to me there is a HUGE potential for companies to offer 'digital vault' services to store digital masters (like LTO-3 tape) and keep them upgraded and in a state that they can be accessed by their clients. Like a bank.

A client calls up, and orders some old footage. This company delivers it to them on a hard drive. To do this, they pull the footage off the LTO-3 tape, verify it, and deliver it. The client, of course, pays a fair price for this.

[Michael Hastings]

http://www.amazon.com/Sandisk-Performance-SDCFH-8192-901-Retail-Package/dp/B00065ANY2

8 gb ultra II speed cf is 90 and 16 gb are $149 so you could back up to these slower cards for long term storage at 1/3 the price of the high speed cards needed for the camera.



Im wondering what are the economics of using CF Cards as permanent Storage for RED Files

if it costs 400+ for the average 35mm can, then what would it cost to use CF CARDS only once and use them as our permanent storage for the files rather than buying an LTO or TAPE solution. at least we'd be guaranteed that the sizes would go up over time and the prices would come down.

right now its $199 for 4.5 mins of 4k footage, so if you have say an optimistic ratio of 11 to 1 for a 90 min feature. thats 990 mins, 220 8gb CF CARDS would cost

$43,780 which would be your tape stock. but you'd have no processing costs or tape transfers. by the end of this year that amount will have halved with 16gb cards and maybe halfed again in 2009. so end of 2009 with that ratio we'd be looking at about $11,000 for storage. an LTO drive costs in the Region of $4k for the latest single tape drives. maybe another $2k for a decent raid too offload the footage. how long are lto3 o 4 tapes rated to live. LTO tapes cost about $80 a piece. but it takes time to backup. its about 350mb a minute. 2 Terabytes of data on 220 CF CARDS, that would take at least 6 LTO maybe more if you don't use compression. thats about $500

what is the life of a CF CARD you've only used once. does anyone know their rated life. would it give us piece of mind about what is essentially our original digital negative.

[steadimoose]

The workflow in the edit room seem well covered, but the workflow from the set, to the edit room on a daily basis, is still somewhat unclear. I had an idea about using XDCAM disc as backup, but Sony dosn't support the data format anymore.
"Effective March 31, 2007, "Professional Disc for DATA" products have reached end of life."
http://www.sony.net/Products/MO-Drive/ProDATA/index.html
Nobody in the post-industry in my region, that i have spoken to, uses the LTO solution so far, so, you would at least need one LTO at each end of the pipeline.
The XDCAM is a widely adopted standard in the TV-industry, it's in a nice case and is quite stable, from my experience. It holds 50 GB - enough to hold a decent amount of shooting time. You can archive it, and you don't end up with all your eggs in one basket.
But so far, it's a dead end, unless someone out there knows something else. Please pitch in.

[Bing Bailey]

What about dual layer Blu Ray , thats 50gbs , about $20 a disk , probably sturdy enough to send to the post house as long as you made your own set of backups to disk first

[Bing Bailey]

2 of those would probably cover most days shooting

[Jeff Kilgroe]

My vote is for LTO-3 -- cheapest $/GB and widest acceptance as a data archival format.

Oh yeah, don't make just one copy. Don't store all your copies in one place.

[Dylan Reeve]

the CF media should be stable on a shelf for a fairly long time - but there could be problems just getting the quantity that's requried. I think it could be tricky getting 220 cards in one go.

Plus the logistics of restoring data from 220 CF cards is painful. Not complicated I guess, but tedious to say the least.

For the duration of the production I would suggest online drive-based storage (nearchive) for all camera files. Once the production is finished, all can be backed up to LTO3/4 tapes which are rated at 30 years storage lifetime - LTO is always backward compatible and cross-manufacturer compatible - meaning the LTO3 tape you write in a Tandberg drive today, can be read in a Quantum LTO6 drive in 10 years.

I'm skeptical about optical discs - I have CDs and DVDs from less than 5 years ago that are already just about unusable.

[Robert Berger]

the CF media should be stable on a shelf for a fairly long time - but there could be problems just getting the quantity that's requried. I think it could be tricky getting 220 cards in one go.

Plus the logistics of restoring data from 220 CF cards is painful. Not complicated I guess, but tedious to say the least.

For the duration of the production I would suggest online drive-based storage (nearchive) for all camera files. Once the production is finished, all can be backed up to LTO3/4 tapes which are rated at 30 years storage lifetime - LTO is always backward compatible and cross-manufacturer compatible - meaning the LTO3 tape you write in a Tandberg drive today, can be read in a Quantum LTO6 drive in 10 years.

I'm skeptical about optical discs - I have CDs and DVDs from less than 5 years ago that are already just about unusable.

I have been told that CF are no good for back-up, its about slippig away from data in a short period of time (not in weeks ore months) and LTO4 or 3 is better and cheaper.
Does somebody have a full CF card witch is years old with data on it?
My 8 GB CF card with the red 0044 shots from october 07 left daily in the car (cold/warm/moisty) and showing everywhere, is still oke!
I wil try not too used it and see what happens!

Robert

[Wade McDonald]

For myself and my company, we've decided to keep an "online" archive of all projects' original source footage with a RAID 6 array. You can easily get a 8-12TB raid for ~$10k-15k or so. We'll be expanding it as our storage needs increase... it's easy, cheap(er), and relatively low maintenance... plus the data's online! I've got a Xserve blade running host to it that doubles as a cluster-able render-box for Compressor/Shake.

[Nathan Buxton]

its not going to lose it's data unless it encounters strong electromagnetic fields. Didnt Mr. Jannard post something about a CF card found in a lake that still had its contents intact?

[Oliver Skibbe]

I'd like to see some kind of WORM-CF, just like this:
http://www.dpreview.com/news/0702/07022601sandiskwriteonceflash.asp

:w00t: (http://www.dpreview.com/news/0702/07022601sandiskwriteonceflash.asp)

[Jeff Kilgroe]

Write-Once FLASH is coming. If it can be made fast enough and cheap enough, then to me that seems like the perfect media for RED. You shoot and when you're done you have the equivalent of your digital neg right there. Backup just in case the card gets lost, stolen or run over by a truck. But the upcoming WORM memory products are intended for such uses and as archival media. At least from most manufacturers.

[Bing Bailey]

I've never heard of RAID 6 , I know RAID 0 , 1 and 0+1 , 5 and 10, 12TB , I guess thats SATA Drives in a chassis. I think given the average flick is going to take about 2TB's to store , are we talking about all the footage or just the finished piece. 12TB gives you 6 movies worth of space and only if your shooting ratio is about 11:1

Computers can't get fast enough , DUAL GPU's cant come soon enough and Harddrives can never get big enough for me. ROLL on 16core , QUAD GPU, 16GB Memory, 100TB RAID STORAGE all for $12.50 :)

[Wade McDonald]

I've never heard of RAID 6 , I know RAID 0 , 1 and 0+1 , 5 and 10, 12TB , I guess thats SATA Drives in a chassis. I think given the average flick is going to take about 2TB's to store , are we talking about all the footage or just the finished piece. 12TB gives you 6 movies worth of space and only if your shooting ratio is about 11:1

Computers can get fast enough , DUAL GPU's cant come soon enough and Harddrives can never get big enough for me. ROLL on 16core , QUAD GPU, 16GB Memory, 100TB RAID STORAGE all for $12.50 :)

RAID 6 is pretty much RAID 5 only with "double striped parity" so that up to 2 drives can fail and you can still rebuild. (because nothings worse than a second drive dying during a rebuild except an idiot pulling the wrong drive)

Yep, I'm talking SATA raids with fibre hosts.

RAID 6 info:
http://www.intel.com/technology/magazine/computing/RAID-6-0505.htm

I only mentioned the RAID idea since to me, it seemed like a good solution for archiving, and a lot cheaper and easier than CF cards. I imagine that most features' total footage in R3D would be 3-4 TB but you can always add more storage and it's online ready to be accessed at any time.

Besides, can you imagine something more nightmarish than restoring your feature's 3TB of footage from hundreds of little 8-16GB CF Cards? That's a Sisyphusian (http://en.wikipedia.org/wiki/Sisyphus) task, IMHO.

W

[Nathan Buxton]

I've never heard of RAID 6 , I know RAID 0 , 1 and 0+1 , 5 and 10, 12TB , I guess thats SATA Drives in a chassis. I think given the average flick is going to take about 2TB's to store , are we talking about all the footage or just the finished piece. 12TB gives you 6 movies worth of space and only if your shooting ratio is about 11:1

Computers can't get fast enough , DUAL GPU's cant come soon enough and Harddrives can never get big enough for me. ROLL on 16core , QUAD GPU, 16GB Memory, 100TB RAID STORAGE all for $12.50 :)

There is such thing as dual and quad gpus it's called SLI... you should look into it. I don't know if it would actually benefit REDCINE but it may be worth a shot

[Bing Bailey]

There is such thing as dual and quad gpus it's called SLI... you should look into it. I don't know if it would actually benefit REDCINE but it may be worth a shot


yes Im aware of it. I'm a gamer too . but its not as nice a solution as having multiple gpu's on the same card. which I believe amd/ati are rolling out soon. anyway SLI is a fudge. it wouldn't be as fast as GPUS on the same die

[Nathan Buxton]

ohhhhh gotcha... sorry i figured youd know about that heh... didnt they have a few boards with dual gpus already?

[jbeale]

what is the life of a CF CARD you've only used once. does anyone know their rated life.

I've never seen a "rated lifetime" figure for CF cards. I guess they aren't expected to be used for long-term storage. I suspect that storage at low temperature is better than high temperature. But anyway the anecdotal evidence is good:

"The memory cards in most cameras are virtually indestructible, found Digital Camera Shopper magazine. Five memory card formats survived being boiled, trampled, washed and dunked in coffee or cola. [...]"
http://news.bbc.co.uk/2/hi/technology/3939333.stm

[filip kovcin]

The XDCAM is a widely adopted standard in the TV-industry....... It holds 50 GB - enough to hold a decent amount of shooting time.


correct me if i am wrong but XDcam holds just around 25GB on disk. NEW xdcam HD format - (read: dual layer bluray cartridge disk) is cappable to hold 50GB.

i think that this path - with xdcam HD (maybe not the cheapest one) is pretty ok. xdcam hd is more and more accepted as format - so it's not difficult to rent a XDCAM HD drive/vtr for your purpose. copy your footage and store it - no need for LTO drives and tapes.
AND - you can use that XDCAM HD VTR on the set also as preview backup (XDCAM HD compression) for realtime footage thru HDSDI out - when it comes alive on red :) and after the shooting your assistant/data loader can use the very same machine to backup your data - as data only. so again - if i am NOT WRONG - this is quite unique solution.

but maybe there is also possible to RENT LTO drives, but i never heard about that possibility. (at least not in poland where i live)

filip

[Lachlan Ward]

the CF media should be stable on a shelf for a fairly long time - but there could be problems just getting the quantity that's requried. I think it could be tricky getting 220 cards in one go.

Plus the logistics of restoring data from 220 CF cards is painful. Not complicated I guess, but tedious to say the least.

For the duration of the production I would suggest online drive-based storage (nearchive) for all camera files. Once the production is finished, all can be backed up to LTO3/4 tapes which are rated at 30 years storage lifetime - LTO is always backward compatible and cross-manufacturer compatible - meaning the LTO3 tape you write in a Tandberg drive today, can be read in a Quantum LTO6 drive in 10 years.

I'm skeptical about optical discs - I have CDs and DVDs from less than 5 years ago that are already just about unusable.

Dito. A tape library that's Automated really does look like the best option by far. I heard rumblings about a new HDD build, some kind of tricked out sci-fi laser cube, anyone know anything?

[Dylan Reeve]

Dito. A tape library that's Automated really does look like the best option by far. I heard rumblings about a new HDD build, some kind of tricked out sci-fi laser cube, anyone know anything?

There's been talk of three-dimensional storage for quite a long time. I'm fairly sure I remember reading a 'it is just around the corner' article nearly 10 years ago.

The one I remember is a bit like DVD in theory, but uses two lasers, and the data is recorded within a cube where the two beams intersect.

Other 'holographic' storage options use a disk... In fact, according to CNet (http://www.news.com/Maxell-focuses-on-holographic-storage/2100-1015_3-5973868.html), Maxell will release a 300GB holographic disc in September 2006... Oh... Wait... Although lots of people are working on this stuff.

Here's a 2001 article from Slashdot about 1TB in a cubic centimeter:
http://slashdot.org/articles/01/04/26/1253237.shtml

InPhase Technology actually makes some holographic storage, but it's not exactly a widely used technology:
http://www.inphase-technologies.com/
$18,000 for the drive, $180 for 300GB media.

[Hans von Sonntag]

The avantage of tape is that it can be repaired in case it got ripped. Additionally, tapes are pretty sturdy. I never got a videotape broken. Even they fall on the floor, how often happened that?... Do this with a HD. And third, tapes have a certain physical size. CF Cards are prone to get lost, tapes like Betacam for instance not.

The most widely used standard is LTO3 which is compatible to LTO4, which again will be compatible to LTO5 as LTO2 is to LTO3, you get it.

I suspect that LTO will be the tape standart that will be widly used like Betacam was. Exchanging footage will be easy.

Hans

[John K]

I have been told that CF are no good for back-up, its about slippig away from data in a short period of time (not in weeks ore months) and LTO4 or 3 is better and cheaper.
Does somebody have a full CF card witch is years old with data on it?
My 8 GB CF card with the red 0044 shots from october 07 left daily in the car (cold/warm/moisty) and showing everywhere, is still oke!
I wil try not too used it and see what happens!

Robert

I've got an early 80s computer that has its operating system on an EPROM (which uses basically the same storage technology as flash RAM) and it still works. I know in the early days they only guaranteed EPROM storage for 5 years, but it seems to have a much longer life than that. Once a FLASH RAM module is assembled, there's not a lot that can go wrong with it, and unless it gets zapped with a lot of static electricity, theoretically it should last a lifetime. On the other hand I wouldn't want to be the guinea pig for this.

Why not just add an IDE type hard disk bay to the front of your PC, buy some cheap 300GB or so drives, and just copy your flash cards onto those, more or less using them as giant floppy discs? Modern PCs will allow different hard disks to be swapped around with no trouble, as long as you do it with the computer switched off.

[Bing Bailey]

would be nice if RED could make a device that had a hi speed CF CARD Reader and a Red Drive combined that would offload the footage and do a verification check on the data without the need of a computer in between

would make for faster offloading. then you could replicate that to an external drive to send off site

[cinemano]

Hard drives are so cheap nowadays.. why not just store them in hard drives?
A 1TB drive in a supermarket in Portugal is like 200 euros.. thats 20 cents per gigabyte.. so each RED Drive emptied onto it would be like 66 euros or something.. Three Red drives per 1 TB storage, its not bad.
And here things are expensive. cant imagine how much cheaper 1tb drive is in US or Uk.. :)

[Dylan Reeve]

Drives don't always survive long-term storage when not connected. They are not a great 'on the shelf' option.

[BASSAM MSSALATIE]

i will vote for cf cards if no need for long storage time. otherwise RAID 6 :matrix:

[combatentropy]

Simple and straightforward:


Remove CF card from camera.
Insert CF card into PC, copy footage to PC hard drive for editing.
Put CF card on shelf.



I've got an early 80s computer that has its operating system on an EPROM (which uses basically the same storage technology as flash RAM) and it still works.

Besides stories like this, I have no proof of the life of flash. But I'm sure if you Google long enough you will find scientific tests. It should last at least as long as any tape: magnetic storage with no moving parts vs. magnetic storage with dozens of moving parts.

And the cost of flash is falling like a rock. At this moment, we are on the cusp of affordability, where it is still just a little painful. But by the time you get your tape-backup system running, flash will be cheaper.

LTO, IMO, is just complication. The device itself is more complex: lots of moving parts. Therefore, lots of mechanical points of failure.

And not only is the equipment more complex, but it makes your workflow more complex. Which means more human points of failure (forgetfulness, botchedness).

[Dylan Reeve]

It's nice in theory, but we're talking about cards that are smaller than a DV tape, they're fairly easy to lose.

Also, while it seems like a good idea, it all falls apart if you need to restore the data later on (for high-res finishing, or in reviving and old project, or after a hardware failure) when you have to load and copy each card individually - the time taken at that point will be huge and painful.

Think how unfun it is redigitising a program from source tape - then consider that each of those tapes holds 6-10 times more than a current 8GB flash card.

LTO is a backup device, manufactured as well or better than most of the professional tape formats we've been using until now. It is high capacity and very fast. You'll be able to store 500 minutes of footage on single LTO4 tape. An entire feature could be stored on one or two tapes. The restore process in that case is really simple and efficient.

Storing CF is an alright idea for the duration of active production maybe, but would be a huge nightmare for longer-term archival storage. You'll have a library of cards numbering in the hundreds or thousands very quickly. And you can't easily put a nicely numbered spine label on them or anything.

[Jeff Coatney]

Kodak really has the horsepower to be relevant in the field of archival storage solutions. I'd like to see them come up with a non-holographic, non-magnetic method of storing and retrieving digital data on thousand year time-scales. They've always been the "memory people". Capturing images in time, held in silver on acetate substrates. The company that could deliver such technology cheaply could carve out a huge market share for themselves in an all-digitized world. I bet that there's a long-dormant defense program that's already done all the heavy-lifting for such technology. I'll bet it's right under their noses. A simple, fail-safe, vibration and extreme shock-resistant method of holding vast amounts of ones and zeros for thousands of years and yet requires a simple reader to extract.

[Jeff Coatney]

So, what are we looking for? Needs to be non-magnetic for long term stability and data integrity. Hard drives are out. Current tape formats are out. Flash memory looks promising but how well do they hold up to moisture? Heat? Can they endure high humidity climates? How stable is the metallurgy? Will it disintegrate or corrode after a couple of hundred years?

Retrieval of archival material doesn't need to be fast but it should be easy. A common aspect of most available technologies is the reliance on motors, capstans, bearings and other "nineteenth century" parts that rust, wear, fuse, breakdown. We see this in hard drives, tape cassettes, tape players.

The ideal retrieval technology would be a solid state or optical reader that is wireless, easy to build from ubiquitous components. The archival medium should be "archival" to begin with.

A robust substrate, with very few components, perhaps even an elemental, non-corrosive metal than cannot break down or be damaged by most fires, immersion in salt water or the introduction of most chemicals. It should withstand high doses of radiation.

The assumption in current tech is that there is a baseline level of civil infrastructure that will support any archival effort. I think Katrina, the Tsunami and 9/11 should revise that thinking. We should disaster-proof our models for what passes for "Archival".

Anyway... just thinking out loud.

[Bing Bailey]

It's nice in theory, but we're talking about cards that are smaller than a DV tape, they're fairly easy to lose.

Also, while it seems like a good idea, it all falls apart if you need to restore the data later on (for high-res finishing, or in reviving and old project, or after a hardware failure) when you have to load and copy each card individually - the time taken at that point will be huge and painful.

Think how unfun it is redigitising a program from source tape - then consider that each of those tapes holds 6-10 times more than a current 8GB flash card.

LTO is a backup device, manufactured as well or better than most of the professional tape formats we've been using until now. It is high capacity and very fast. You'll be able to store 500 minutes of footage on single LTO4 tape. An entire feature could be stored on one or two tapes. The restore process in that case is really simple and efficient.

Storing CF is an alright idea for the duration of active production maybe, but would be a huge nightmare for longer-term archival storage. You'll have a library of cards numbering in the hundreds or thousands very quickly. And you can't easily put a nicely numbered spine label on them or anything.

yah but there is no digitising with RED. the flash cards are going up in size and the price is going down. which pretty much means in a very short period we won't be talking about thousands of cards. if we get 32 or 64gb cards within a year we're looking at maybe 50 to 100 for a whole production. each labelled with the day/date production name. would be great if theres a better solution but they all involved lots of time , lots of money or lots of paranoia about losing your work

[Dylan Reeve]

I've look at a lot of current and emerging technologies and I think that LTO is probably the best available at the moment. It is tape, but tape can last a long time, it relatively easy to read even if all the drives cease to exist, the mechanics of reading tape aren't too complicated.

In the future, I think a 3-dimensional holographic storage (which doesn't rely on a foil substrate and polycarbonate laminate - the areas where DVDs and CDs tend to fail). But the chances of an archivist being able to read these 'data cubes' in the future without the right hardware is unlikely.

Tape really does seem like the best option for now, and probably the next 5 years or so at least.

[mezmo]

Clients need to be handed a LTO style tape(even if they can't read it) it's
more a signing off on your responsibility at some stage of the production.
Having said that Red drags you into processing footage post shoot and this needs to be considered also. A Simple archival box with RAID or more complicated SAN/Server to hold your clients material untill the project is finished or released to the clients responsibility.
I'd tend to go for BOTH to be on the safe side. I would hesitate in saying this
is a disadvantage with shooting tapeless, just another expense and an imporatnt part of the workflow that needs to be considered.
I've always thought of Red as a whole system, cameras just one part of it.
Mezmo

[Mark K.]

Compact Flash as a long-term storage medium does have a lot of appeal. CF cards are very small, very sturdy and data can be accessed from them quickly and easily (all of which makes a compelling argument for CF as an archive medium).

As it stands, the single most frustrating element of solid-state recording in video cameras at the moment is archiving your footage after you've uploaded it to your editing system. Hard-drives become expensive in RAID configurations and are comparitively fragile; Burning to optical discs like blu-ray is a slow process and data read speeds are quite slow; tape-based storage (i.e. LTO-3) is probably the most cost-effective option (per Gb) however it requires further investment in equipment and maintainence costs.

Perhaps it's just a hang-up left over from the days when film and tape were our only options, but I think that most people simply prefer to shoot on archival mediums - it's reassuring to know that your original footage exists as something you can hold in your hand (rather than something that will eventually have its Ones and Zeroes transferred to something else entirely). Unfortunately, until the cost per-gig of CF cards drops substantially, they will remain a less-than-ideal for this sort of task.

[Dylan Reeve]

I still think the physical factors with CF are the more limiting factor in their usability. They're really easy to lose, and the small capacity and relatively long physical turnaround time in copying from them makes it a pretty unappealing when it comes to restoring from them. The handling/mounting/unmounting time alone in copying from 20-30 CF cards in one go would be quite substantial. Not to mention the copy process has to be initiated once for every card - in restoring from a tape or even disc based system there is very little handling time, and only one copy/retrieval process.

On a big job (which I haven't yet had to face with a tapeless workflow really) I'd be inclined to make two datatape backups - one would stay here with me, the other would go to the client. The tapes are pretty inexpensive, the biggest issue is the capital cost of the drive, which isn't cheap, but is substantially lower than any sort of professional video tape deck.

[Storyline]

LTO is the standard for governmental and banking archiving in the US - and as such is far more widely supported than any particular videotape system. It's a safe bet for mid-term storage - and anything longer than that raises the issue of just how any editing system will deal with a video file system that may be years out of date. How long with Apple or Adobe or someone else support a given codec in their NLE?

Long term storage of source material has always been dodgy and catch as catch can in the film biz - things tossed into dumpsters, no climate control, elements not catalogued, large physical space requirements. I think digital cinema archiving will be less prone to the pitfalls of archiving source materials.

As for edited masters/release copies, it's good to clone digital final shows to new formats as they come along (or even to the same format every couple of years) to preserve the movie in top form. We've done that going from BetaSP to Digibeta for old edited masters. Digital cloning makes things even better without quality trade-offs.

We put LTO3 into DataSafe's media storage (climate controlled) - and those tapes take up less room per GB (or movie equivalent running time) than even DVCProHD tapes.

[Jeff Coatney]

I think that when the subject of "Archiving" comes up, particularly with respect to Red footage, many of us assume that all the materials we shoot will be archived so we can perform a type of "Apocalypse Now Redux" or a "Blade Runner: Final Cut" in some future circumstance. I don't think it will be practical to truly "archive" the entire Camera "Negative". I think the best strategy is to archive the final, and or Director's cut, and relegate the rest to an "archival tape" format (D5, LTO, The-thing-that's-not-been-invented-yet, etc.).

For RED projects that will be seen via an actual 35mm print, the release print itself can be a stable archival form for many years if not decades, if properly stored.

The best motion picture archival method is having CMY separations made of the conformed negative (or interpositive), like the old Technicolor printing masters were made. The only limitation of that system was the lifespan and relative stability of the base, which could be a century or more under ideal conditions. The emulsion was silver halide, which is the most stable continous tone record that I'm aware of.

Of course, you end up with 4X the amount of film to store (1 print for each of the colors plus an optical print master for the sound) but for the right project its worth it. (IDEA: Why not go 4-up on 70mm stock?)

Many rereleases and DVD masters of films shot in the 70's were made from this type of archived material in the early 90's because the original camera negatives were so shoddily stored.

Don't have a projector or a telecine in 50 years? No problem, just scan it or re-photograph it and its a moving image again. The available Image processing software we'll use 20 years from now will certainly be able to correct any aberrations in exposure, dust, registration, etc.

OMG, Does film still have relevance in our workflow!!?? :)

[mezmo]

Hi Jeff,
Yes there is no better form of archival than film, end of story.
LTO for R3d Digital Original material, film out for final masters.
Tri Seperations if you have a masterpiece on your hands.
As you say shelf life approaching 100 years.
I suspect even with Digital as the main form of exhibition in years
to come people will still use film out for archival. Mad if they dont.
May even derive Digital Deliverables from the film out.
Mezmo.

[Chris Parker]

To store the ENTIRE group of digital negatives (.r3d files) is by no means a big hassle for a feature, if using LTO3 (or LTO4) data tape. really. would take many tapes at all. store a film-out...sure...of the final cut, director's cut, whatever.

but storing the .r3d files is still do-able, and should be done. in 10 years, when there is way better debayering processing available, it will be good to have the master .r3d files. and why not just keep everything on data tape. who knows when it would become useful? it is just so cheap to keep data tape, i'd keep it all if i were a production company who could afford it.

[Brent J. Craig]

LTO-3 / 4 / 5 tapes cost less each than a roll of film. It is ludicrous to suggest that you shouldn't keep every scrap of data a project generates. Who knows how you may want to use them in the future?

You could end up being the biggest filmmaker in history, but you'll look like a dork when the Academy wants to do a retrospective on your life's work and you tell them you threw away all your 'negs'.

LTO Ultrium is a good choice for long-term storage, because the official spec actually addresses future-proofness: "An Ultrium drive is expected to read data from a cartridge in its own generation and at least the two prior generations." (wikipedia)

I'm a bit of a digital pack-rat myself and just last week I needed, found, and used a document I word-processed back in 1994. Did I have any idea that I might have a use for it 14 years in the future?

[BASSAM MSSALATIE]

LTO 4 with 30 years of archival storage 20,000 loads and unloads
is good archival option for red footges as 120 MB/s transfer speed .
is it iwill be good for realtime playback :sarcasm: :help:

[Dane Brehm]

LTO-3 is the current archival standard in the financial world and since we still need $$$$$ to make films. We want to make the most sense to those putting the money down or insuring productions.

LTO tape now, SSD later.

[Dylan Reeve]

I read a really good article a few years ago about digital photography... It was examining how many famous photos would probably never have existed in a digital world.

Situations where the relevance of a photo doesn't become clear until months or years after it was taken. In a digital photography world, the article speculated, these photos may have been lost. The photographer may have deleted them on the camera during a review. Or they may have been culled when offloaded to a computer. Or they may have been left out of the backup process. Or never backed up at all.

With traditional film photography, it was common to keep all the negatives, for ages. Because it was easy, and they couldn't be reused anyway.

The same is true, I think, of file-based acquisition in film and video. It's important to consider the future - which means saving the camera original material with forward consideration. Maybe in the future you'll want to rerelease, or your indie film, finished in HD gets picked up by a big distributor and you have the chance for a directors cut, and finishing in 4K for a film-out.

An LTO4 tape will hold 800GB, around 500min of footage and costs under $100. That's great value - there's really no reason not to save all the footage.

[Brent J. Craig]

Situations where the relevance of a photo doesn't become clear until months or years after it was taken. In a digital photography world, the article speculated, these photos may have been lost.

TRUE! With hard drive space being almost free, I never consider deleting anything I have shot. I have found uses years later even for accidental shutter-releases that turned out to have captured a cool texture or an interesting color pallette I could use.

[Dylan Reeve]

TRUE! With hard drive space being almost free, I never consider deleting anything I have shot. I have found uses years later even for accidental shutter-releases that turned out to have captured a cool texture or an interesting color pallette I could use.

Very true in editing, where you find that movements, looks and reactions from before and after the intended action are often great.

The archivist in me worries about the transient nature of all the digital technology we use today... There's very little chance my grandkids (or even my kids) will ever see poems or love letters between me and my wife (if they exist) because there's practically no chance my old emails or whatever will be stored in a box in the attic 50 years from now.

I think archival backup should happen with RED (and other file-based formats) before post even begins - it should really be seen as part of the the production workflow, not the end of the post-production one. Just as we tend to keep our camera original tapes in video production, rather than just store a consolidated backup of the final edit or something.

[Bing Bailey]

Maybe paper punch cards would work better for long term storage :) LOL

[Dylan Reeve]

As long as there's no hanging chads in your punch cards, that seems fine :)