Best Archiving Solution

[Jeff Coatney]

IMHO this type of system represents the best type of archiving available to the RED community. I've been searching for something more reliable than magnetic media (I think there should be an alternative to LTO tape as a protective measure). This is solid state and can be preserved for a long time.

http://celco.com/cinesafe4k.asp

What are your thoughts?

[Dylan Reeve]

Seems like a good idea in theory I guess, but it's going to be really costly I think. Not to mention the process of recovering from archive in with that is going to be a lot more difficult. Also you lose the native film format. You no longer have REDCODE RAW, you have colour separations and no metadata.

Also, it's slow! 1.4sec per frame? Aiie.

A good option for archiving a finished cut perhaps, but really not practical for raw footage.

[Jeff Coatney]

Keep in mind that the system I reference has three advantages that no digital archiving solution can duplicate:

1. The content is Human-readable.

2. The content can be fully reconstituted by more than one technology.

3. The content cannot be corrupted after its written.

It also has the following advantages:

1. Will not corrode.

2. Has no moving parts.

3. Cannot be affected by magnetic fields.

4. No data dropouts or sector read-errors.

[Chris Swinbanks]

I think its a clever marketing attempt by Celco to remain viable and try to tap into what they might see as a "potential" new market.
The effort required to recombine images from a tri-sep master is really only worthwhile for extreme high-value finished films, think "classics" and certain "block-busters", and then really only for those that have further market potential or high historical value.
Its not to be taken for granted that the process is simplified somewhat through digital imaging (as opposed to optical re-combining back to film) with the much improved film scanners we have now, but it just ain't cost effective as you then also have to store all this film output under environmentally controlled conditions.
Your colour digital neg on intermediate stock that you produce from a final DI also has the potential to last a hundred years under the right storage conditions, although that's yet to be proven for certain with anything other than age-testing in controlled environment chambers, but many Archives are working on just that, as well as storage methods for tape, B&W film stocks, anything that has image and/or audio recorded on it.
Stick to the LTO 3/4 for now, and pay a much smaller price to keep it properly stored. If you think you really want to get at the r3d data again in 5 years time, then you can copy your tapes to the latest greatest format that will probably be solid state and 5 Tb on a postage stamp size chip.... and r3d workflow will be old hat, cut your feature on your iPhone mk10 and project from your penlight size 4k projector :biggrin:

[Brent J. Craig]

Hey Jeff, 1980 called - they want their machine back!

:-)

[Jeff Coatney]

I agree that the outlook for film recording may appear bleak, however, this is the strategy to pursue if they want to be relevant. As film restoration has proven, the Black and White film stock is inherently more stable and longer-lived than any color stock ever has been, including the ones they have now.

As for the cost, I expect the cost for this type of archiving to drop dramatically in the next three to five years and be available in the larger markets.

Obviously, I don't advocate replacing LTO with this, but this type of system is proven to be truly archival and should be part of any serious archiving strategy.

1. Footage and Final on LTO 4. (x2 or more)

2. Final on D5 tape (HD Master) (x2)

3. Final on B&W Film Seps (x1)

Stick one of each in a Salt mine and forget about it. Hopefully, it will never need to be retrieved.

[John DeBoer]

Hey Jeff, 1980 called - they want their machine back!

:-)

Were you even alive in 1980?

LOL

[Harva Raj]

archiving in blu ray disk seems to be the most reasonable solution for a desktop base system rite now. Its cheap,fast,compact,random access,easy to transfer and view. This solution is ONLY for indie filmmakers, short commercials, short movies and stuff. if your a studio, go buy this $700k film recorders and store your film in nuclear bunker.

[Jeff Kilgroe]

1. The content is Human-readable.

Yep. If you feel you need that. Because in the thermonuclear aftermath of WWIV, I know that everything will be right in the world if I can sit and silently enjoy my old film strips. Especially when I have three black monotone representations that I need to somehow combine to actually see a proper image.


2. The content can be fully reconstituted by more than one technology.

That's a stretch... It's film, you shine a light through it and find a way to capture the projected image. So I guess as long as people can make fire or hold the film up to the sun, we're good to go. One could argue the same about digital media. There's more than one way to skin a cat or read magnetically charged or optically reflective dots off a smooth surface.


3. The content cannot be corrupted after its written.

So says the guy who thought the film would be OK shipped FedEx, but it sits in a 130 degree delivery truck in Phoenix for 3 days over a holiday weekend.


It also has the following advantages:

1. Will not corrode.

Bull. All film degrades and corrodes over time. The question is how quick is the rate of decay, how can it be mitigated for maximum longevity.


2. Has no moving parts.

Me Grog. Magnifying glass lets Grog read film with no moving parts. Sorry, but there's forms of digital media out there with no moving parts, there's got to be a system in place with moving parts to facilitate reconstruction of the color layers into a color film print.


3. Cannot be affected by magnetic fields.

Yeah, but IR, UV and even prolonged exposure to visible light will F it up nicely. So will heat or significant temperature fluctuations.


4. No data dropouts or sector read-errors.

...Registration errors, jammed film feeders, chemical process, transition from digital to analog... pick your poison.


Yes, I'm kinda poking fun at this archival method. Some of it undeserved, there is some sound reasoning behind this film archival method. But I think there's a lot of industry-wide FUD out there that will push people into such solutions when they're not needed.

I can just as easily make fun of digital archival methods and there are plenty of problems on that side of the fence as well. But if handled properly, there are some distinct advantages to be had. IMO, archival has gone the way of acquisition - digital. From here on out, I only plan to touch film if it's needed for final distribution. Period.

As for sky1walker and his Blu-Ray opinion, I think he's nuts. The math just doesn't add up. The cost per gigabyte is way too high -- You're at close to $1300 by the time you have bought a Blu-Ray drive and made dual copy backups of 1TB of data. Yikes. That's more than double the price of doing duplicate backup of that same data to hard drive. By the time you hit the 4TB mark (think 8TB of physical media needed for duplicate backups), You have generated a lot more pieces of media to track and physically store than you would with LTO or HDD and you have already paid for that LTO-3 drive twice over plus the media.

Anyone who generates more than 50GB per week in data to be archived will be shooting themselves in the foot (through their wallet) by using Blu-Ray. May as well just back up to DVD-R, it's cheaper per GB and more commonly available and readable. DVD+R|DL media is a great fit for 8GB CF cards and they can be had in bulk for $0.75 pretty easily. And I cringe at the thought of the huge disc stacks that would be generated, even by an indie user with moderate workloads.

[Jeff Coatney]

Keep in mind that the system I reference has three advantages that no digital archiving solution can duplicate:

1. The content is Human-readable.

2. The content can be fully reconstituted by more than one technology.

3. The content cannot be corrupted after its written.

It also has the following advantages:

1. Will not corrode.

2. Has no moving parts.

3. Cannot be affected by magnetic fields.

4. No data dropouts or sector read-errors.

And it is now a required deliverable on Major Studio films.

[Sergei Franklin]

Delkin has released Archival BluRay disks that are they claim will last 200 years.
http://www.delkin.com/products/archivalgold/archival-blue-ray.html
The disks are about $23 per 25GB.
So an hour of footage would cost about $200 to archive plus $50 for storage on 2 harddrives ( 8 hours fit on a 1TB drive, 2 drives $400)
For the cost of a 400' roll and the volume of a 1000" roll of film you can archive an hour of red footage and Blu Ray and 2 Hard drives.
With a 6x Burner it should take about 20 minutes to copy 10 minutes of footage.

[Harva Raj]

Delkin has released Archival BluRay disks that are they claim will last 200 years.
http://www.delkin.com/products/archivalgold/archival-blue-ray.html
The disks are about $23 per 25GB.
So an hour of footage would cost about $200 to archive plus $50 for storage on 2 harddrives ( 8 hours fit on a 1TB drive, 2 drives $400)
For the cost of a 400' roll and the volume of a 1000" roll of film you can archive an hour of red footage and Blu Ray and 2 Hard drives.
With a 6x Burner it should take about 20 minutes to copy 10 minutes of footage.

good news!

[MichaelP]

Will you be able to mount a Blu Ray disc in 200 years? ;)
Great for near term storage though.

I have some Panasonic Phase Change Optical discs from 1994 that I can no longer access...

Michael

[MikeCurtis]

always a challenge to balance cost, longevity, ease of use, etc.

Keep in mind what kind of archives you want - do you want to archive source, in which case filming out blows away all your metadata, your RAW goodness, etc.?

Or archive the final - in which case I think the best strategy is multiple formats - film seps being the most future proofed. Yeah, I have magneto optical cartridges in both 5.25 and 3.5 inch form factors with old projects on them that can't be read from the 90s. Anybody else remember SyQuest, Iomega Zip disks, etc.? Good luck reading those readily.

I'd say data archival with all metadata intact, LTO-3 or 4 seems like a good bet at the moment, but frankly, for LONG term storage, even presuming the media lasted, stick the drive and the machine to read it in the vault with it...with printed out instructions on acid free paper on how it all works....

-mike

[C.J.Harvaraj]

Good to have you back mike ...miss your articles on HD for Indies

[Dan Hudgins]

One problem with B&W film for storge is that with color film you can use IR exposure to do automatic dust and spot removal, if you make Black and White seps you have more issues spotting the scans when you scan the film back in later to make a new DI. Black and White film stops IR, so that trick does you no good.

Also I think that you would need sub-pixel scans later because you cannot get film to register to 4K resolution, there would be some issues depending on if you make three exposures or one, if you make three exposures advancing you get weave between the colors, if you make one exposure you get size errors on the outside frames. If you scan a 4K image with a 4K scanner you end up with about 2K resolution because the film cannot hold 4K and the pixels do not line up when you scan them back at 4K, you shoot each pixel off its center and the film weaves around and tilts off level, what you end up with after you make new DI is about 1280x720 true RGB, which is good enough for film print projection, but not for 4K digital projection, the "restored" images scanned off the sep films will look softer than if stored 4K TIFF on tape or film base as digital.

In a CRT film recorder the raster is distorted by the sweep, and the lens adds a little distortion so when scanned out to a CMOS or CCD grid of pixels the CRT pixels recorded will not fall on the same grid points, cutting the resolution.

They should be using 70mm black and white film at a minimum, sideways like IMAX (tm), or at least VistaVision (tm). Then pixel raster register marks should be encoded so that 500 years from now the film can be scanned at 64K and the exact pixels center extracted.

The problem of spotting is not a simple one for Black and White film, but maybe later AI can be used?

It is possable to record digital on film base with a laser burning spots into the polyester film base, that would make the best archive since it would last more than 500 years, and would have all of the digital information with no loss at all, and be "human readable" little dots. It takes maybe about 64 times the footage to get a good dot spacing and 12 bit RGB. There is no coating to deteriorate, and uncoated polyester should last longer than most other things that bend.

[David Nardini]

I've decided to go with these :

http://www.novatech.co.uk/novatech/specpage.html?NOV-HDOCK2

Going to a tape solution just seems like going back in time :huh:
Film has never even crossed my mind !

[Jeremy Newmark]

Going to a tape solution just seems like going back in time :huh:

That may be what you want to do in 5 years when you pull that drive off the shelf and it doesn't spin up. :tongue:

If you go this route, just make sure you mount all those drives on a regular basis. Drives were designed to be used all the time and not sit on a shelf. Tape on the other hand was designed for archiving. LTO will last 20-30 years on a shelf, drives won't. Just my 1/2 cent.

[David Nardini]

That may be what you want to do in 5 years when you pull that drive off the shelf and it doesn't spin up. :tongue:

If you go this route, just make sure you mount all those drives on a regular basis. Drives were designed to be used all the time and not sit on a shelf. Tape on the other hand was designed for archiving. LTO will last 20-30 years on a shelf, drives won't. Just my 1/2 cent.

I just do not get a warm fuzzy feeling with tape (thin stuff that can rust (open to the elements)), the readers cost a bomb, etc.

I'd rather archive to a good quality HD, min 7.2k spindle speed (better quality bearings) in duplicates. These drives would only be used for archive, so usage would be extremely low (relative to expected life span). If needed, they can be on-line in minutes ...

Cheers ...

[Dave Blackham]

For those interested this is how we do it. It works really well and I know of nothing better.

We use 3 products that work together really well, catDV, DAX, Bru Server. Infact the Dax component was designed purposely to work with both catDv and Bru for media archive and retrieval.

We use catDV for the media indexing and operational front end, its much better than Apple's Final Cut Server for media management. We have an LTO4 library attached to the system, either by FC or SCSI. the Dax plug in to catDV pushes media with automatically or manually to Bru server which transfers the media to LTO4 tape. Bru is used really to drive the back up device.

To retrieve media its just a right click with in catDV instruct the media to be retrieved to its original (or new) directory. or an FCP sequence can be loaded in to catDV to conform the sequence from the origional media files.

The company that provided the solution is

www.quadlogic.co.uk

Links to the software are here

www.squarebox.co.uk
http://www.daxarchiving.com/en/solutions/broadcast-archive/challenge.html
http://www.tolisgroup.com/products/bruserver/

(I have no comercial connection to any of them)

Dave
Uk