Protocol for Footage on Set

[I Bloom]

This is a human engineering question:

What system will you use to prevent footage loss on set? In other words how do you insure that footage has been properly transferred from one medium say Compact Flash or a RED Drive to another such as a larger RAID.

I have my own system now for working with P2 cards, but I have to admit it can be flawed.

How will you ensure that all footage you shoot is transferred properly before formatting media, who will do what when you have multiple crew members handling footage? How will you prevent mix ups?

Will there be any additional tools and software that you will use in this process?

IBloom

[Dominique Grenier]

The proper way is to run an md5 checksum (http://en.wikipedia.org/wiki/Checksum) on the files before deleting anything. This way, you ensure that the data has been transfered and hasn't been corrupted in the process.

Also, copying to two drives, either in RAID1 configuration or on two seperate drives, is the way to go on the set. Two physically seperated drives is also safer than two drives within the same enclosure. So if you drop one, or loose it, or whatever, you still have another physical copy of the media.

[Mike Prevette]

simple. hire good people. for over a hundred years the toughest, hardest, most stressfull, and single most important person on a film set has been the loader. that still rings true on amodern film set. a lot of lay people skip over the importance of this person because they see the financial hiarchy on a film set, they don't realize the trial by fire system that has been set in place so everyone working their way up the camera crew knows the importance of a good loader.

[Jim Arthurs]

And before anyone thinks that the film workflow was loss-proof, I learned a couple of things by free-lancing in a motion picture lab in the mid '80's....

... the lowest paid guy in the building is the first one to touch your undeveloped film.

...And the "smack" sound as somewhere in the process a splice between reels fails and a couple of clients reels goes flying off the rollers into the soup.

... And the fun of fishing out a dead rat from the ultrasonic cleaning solution.

Film is bullet-proof. :)

[sbaechler]

It would be a good thing to have a custom software that takes care of the copying and validating of the files.

I've worked on a P2 feature and we lost a few media files (video or audio) without noticing. There was no error message and we only noticed when the card had already been formatted.

I'm working on a system for downloading and storing footage on the set. It also lets you watch clips and batch convert them. It will also feature a removable drive that can go to the editor every night. Of course it will be transportable and everything is stored redundant.

[Mike Prevette]

By no means was the film system perfect, neither was the p2, nor will the RED be perfect. My point was the best way you can hedge your bets is by hiring responsible, smart, educated people. There will always be room for error, but you can control that margin down to a minuscule percentage point. Software nor hardware will never fix that completely.
_mike

[I Bloom]

By no means was the film system perfect, neither was the p2, nor will the RED be perfect. My point was the best way you can hedge your bets is by hiring responsible, smart, educated people. There will always be room for error, but you can control that margin down to a minuscule percentage point. Software nor hardware will never fix that completely.
_mike

I agree with you. But smart educated people also get tired and make mistakes. A good system helps them prevent those mistakes. That's what I'm interested in a system or protocol. Software is only part of it, people are only part of it.

Ian

[donatello b]

never reuse the CF = no filed lost !!

[Chris Kenny]

It's been posted here before, but the David Fincher quote on this subject is priceless:



“The studios, for instance, often had what, for me, was a surreal response early on. They were trying to understand who, exactly, would take the digital media from the set and get it cloned and archived safely for them. I said, ‘The same, totally underpaid PAs who normally take your film from the set, in the middle of the night, to the lab. Now, instead, they will be taking an anvil case with a D.Mag in it back to the editing room.’” (Link (http://digitalcontentproducer.com/videoedsys/revfeat/video_going_tapeless/))


Thanks to REDCODE, we'll have it a little easier than Fincher, since footage will be small enough to fit on pocket-sized hard drives.

We're planing to download footage onto two LaCie Rugged (http://www.lacie.com/us/products/product.htm?pid=10722) hard drives, set up in RAID 1 mode. We already have several of these drives kicking around, and we like them a lot. When transporting footage back to the office, we'll keep the two drives physically separate, probably with one in the personal care of the director and one in the personal care of the DP.

As far as making sure nobody erases flash cards before they're downloaded, if you write up a list of instructions that goes something like...

1. Checksum original data.
2. Copy data.
3. Checksum copy.
4. Make sure checksums match.
5. Erase flash card.

...it's inevitable that as people get tired, or multiple people switch off downloading footage, steps will get skipped.

The way to fix this is to reduce the number of manual steps. I'm probably going to write a script that does all of this with one click, and provides nice prominent messages announcing success or failure. Unless Red has something like this in RedCine already, of course, which wouldn't be a bad idea.

[Michael Schrengohst]

LaCie's are diasters waiting to happen. Look at G-tech instead.

[Chris Kenny]

LaCie's are diasters waiting to happen. Look at G-tech instead.

Yeah, we've had issues with their desktop drives, I wouldn't recommend those. But we've had several Rugged drives kicking around for a while now, and they seem solid, despite taking a fair bit of abuse.

[I Bloom]

1. Checksum original data.
2. Copy data.
3. Checksum copy.
4. Make sure checksums match.
5. Erase flash card.


I'd add to that a sanity check:
4B. Make sure all the footage on one drive matches the footage from the other drive, visually.
4C. Make sure no two batches of copied footage are the same.

I think the biggest problems arise with multiple flashcards or drives. Here is something that actually happened to me:

I had worked out a system to transfer P2 cards using a Macbook with a DualAdapter connected to a drive. Usually the DIT would get handed one card, and they would go dump it and hand it back. Since we format in camera there is a hand off, at which point the AC asks, is this card ready to be formatted, the DIT says yes. Then they immediately pop out the other card so as not to get it confused. They insert the card ready to be formatted, they format it. Reinsert the other card and reselect that first card so that we continue rolling on it. (RED only has one CF slot (currently), so if we format in camera, the process will be different.)

Here is where a problem can arise. Sometimes, you aren't in a spot where you can continually dump and you pass two or more cards to the DIT that all need to be dumped. Disaster can strike if the DIT, copies the same card twice and forgets another card. The checksum can't overcome that problem, we need several layers of sanity checks and organization to prevent it. It would be nice if software could warn you when the same footage is copied twice as well.

I got totally screwed by this one time...by the way.

Rereading this I can see how formatting at the computer will eliminate this problem.

IBloom

[Chris Kenny]

Rereading this I can see how formatting at the computer will eliminate this problem.

It eliminates both problems. First, you never download footage twice, and second, the card gets formatted by someone who has just personally verified that the footage has been copied. (Or, if you've automated the whole copy/verify/format process into a one-click operation, the card gets formatted by the same process which just verified the footage was copied.)

From reading in the other threads, the camera will need to format the blank card again when you insert it. But this is not a problem, because it automatically prompts for this. IOW, you never have the camera operator manually choosing to format cards. If the camera op inserts a card and doesn't get a formatting prompt, he knows something has gone wrong somewhere and he should immediately eject the card and hand it over to someone who can figure out what's on it.

[I Bloom]

It eliminates both problems. First, you never download footage twice, and second, the card gets formatted by someone who has just personally verified that the footage has been copied. (Or, if you've automated the whole copy/verify/format process into a one-click operation, the card gets formatted by the same process which just verified the footage was copied.)

From reading in the other threads, the camera will need to format the blank card again when you insert it. But this is not a problem, because it automatically prompts for this. IOW, you never have the camera operator manually choosing to format cards. If the camera op inserts a card and doesn't get a formatting prompt, he knows something has gone wrong somewhere and he should immediately eject the card and hand it over to someone who can figure out what's on it.

This seems slightly convoluted no? Maybe REDCINE needs a format option. Is it a necessary part of your human protocol that both need to format the card. Seems like the operator should just roll on anything that they are holding, and only worry about passing off cards that are full.

Ian

[Chris Kenny]

This seems slightly convoluted no? Maybe REDCINE needs a format option. Is it a necessary part of your human protocol that both need to format the card. Seems like the operator should just roll on anything that they are holding, and only worry about passing off cards that are full.

The format prompt on the camera serves a useful purpose here, actually. The operator will know that they should always get that prompt when they insert a card. If the card were formatted on the computer in a way that made the camera fully happy (so it didn't prompt when inserted), it would leave open the possibility that a card with data on it could be inserted, and the operator might not notice. They wouldn't format that card, but they might lose a shot when they realized too late there wasn't as much space on the card as they thought. (I don't know how obviously the camera makes this situation clear.)

In an ideal world, RedCine would have a feature where, with one click, it could copy footage to two drives at once (not RAID mirrored -- it's safer to have two fully independent volumes in case of software-based file system corruption), verify the data, format the card in a way that made the camera happy, and pop up a nice reassuring message that everything had worked (along with a list of the files copied, including digital slating data). On top of this, the camera would have an extremely prominent warning if a card with footage on it was inserted.

[tj williams]

I have a friend who is a dit in the uncompressed world. They have very complete systems approaches already worked out. I will try to get a complete run down here. I sure hope the software folk at RED are reading this thread as the idea of the workflow being built into RED cine as a script which simply gives us a big COPY SUCCESSFUL/CHECKSUM SUCCESSFUL then asks if the dit would like to reformat the card or Mag. and only asks this When the copy and checksum are successful will do wonders to prevent embarrassment to our camera system.

I would also like to be able to dual record on 2 CF cards at a time or onto two hard drive mags at a time. This would be good for really important stuff when it just cannot be lost and enable the production to have 2 recordings at all times right from the camera. I agree that it will be rare for the mag drives or cf cards to fail, but the more redundancy you show the insurance folk the better accepted the gear will be! Also this clearly makes the reliability of the storage more capable than film.

[Ramesh Jai]

By no means was the film system perfect, neither was the p2, nor will the RED be perfect. My point was the best way you can hedge your bets is by hiring responsible, smart, educated people. There will always be room for error, but you can control that margin down to a minuscule percentage point. Software nor hardware will never fix that completely.
_mike
That has alway been my motto; hire the best people for the job so that you can do your job best.

[Jonathan L. Bowen]

Yeah I also find it ironic that the person who handles the footage, which is the raw components of the final product itself, is often underpaid and considered low on the totem pole even though if he fails, you are really in trouble.

I saw someone post an ad on Craig's List that really shocked me. The guy wanted a P2 card downloader, someone to transfer the footage to a laptop I mean, for $100 for a full day of shooting and expected the applicant to provide a laptop for them and the means by which to connect the P2 card to the laptop. I mean, $100 a day is terrible pay, but could be acceptable if you're just expecting labor and figuring the person will get experience, but to pay someone $100 AND expect them to bring expensive equipment to the set is ridiculous to me. I'm sure the guy still had 15-20 people interested even so. Sad.

[david farland]

I would also like to be able to dual record on 2 CF cards at a time or onto two hard drive mags at a time.

You can always keep on adding redundancy until you nuke it off the face of the earth but I'm not sure double disks or flash drives are the option.

If I was going to that length I'd probably prefer recording to 1 cf card and 1 Reddrive (raid 1) at the same time, swapping cf cards out as they were full and then backed up to laptop etc.
But I can see where it'd be cool to have dual hot swappable cf drives for continous recording...or to be still recording to cf as you hot swapped reddrives.
Cheers,

[M Most]

I have a friend who is a dit in the uncompressed world. They have very complete systems approaches already worked out.

"The uncompressed world" these days usually means S.two recorders. In this case, the backup situation is handled by a dock system made by S.two. The backups are not on mechanical disk storage, but on LTO3 data tapes. In most cases, the system loads the D.mag from the S.two (about 30 minutes worth of material) and backs it up to two LTO3 tapes simultaneously. The D.mag then gets recycled. Verification of the data is done by the backup software. This is a good way of doing it, and illustrates a couple of things about the high end market - where failure is not acceptable - and the more "DIY" market, where cost is the primary consideration. First, proper backup and verification takes some time. Second, mechanical storage - like hard drives - and fragile storage - like CF cards - is just not considered as reliable in the long run as a more solid, designed-for-the-purpose solution, such as LTO3. Third, proper backup at the source is rather critical when you're recording on a medium that is not permanent (i.e., temporary data storage as opposed to, say, videotape or film).

[Jochen Schmidt-Hambrock]

Maybe REDCINE needs a format option. Ian

I personally like the way Aperture handels this - although I always feel a little uneasy clicking "yes" to format and eject when I haven´t checked all the material yet.

Jochen

[tj williams]

mmost: Exactly I talked to my friend who has been working with the S2 equipment. you already knew!
I'm totally in agreement that built in work flow safety is critical to wide acceptance.

Seems to me that some of the folks here will also want to use LTO tapes as on set backup. Again if RED automates this in a way similiar and perhaps even easier to use than S2 it seems to me a good thing!

So: is this somewhat complete?

1. record to either single media, or if allowed to dual card or card/disk on camera.
2. download card to either dual lto or two seperate drives or redundant raid
3. RED cine automatically does check sum (or perhaps faster or better data check) and automatically gives a simple answer that data is copied correctly and checked and then and only then gives the option to reformat the card or disk mag.
4. RED cine reformats the disk with the same project settings.
5. media is installed onto camera and menu item flashes asking for reel number.
5. The cheap wage kid then delivers one to post and one to safe storage at a seperate location. (yea that is amazing when you think of it. (My son has been doing some 2nd AC on features and got a good laugh about this!)

Hi David:
The S2 system is pretty bulletproof as opposed to the RED light weight drives on-board camera or CF cards that can easily be damaged while being carried around. My point is not that, for every project this is right just that it will be good on high end projects, to be as reliable as S2. In fact I imagine S2 will introduce a some kind of S2 Light to work with the RED compressed and when the optional uncompressed output comes out they will also support that!

[I Bloom]

I hope this thread eventually leads to a consensus on how to deal with footage. I think people are getting lost in the technical specifics. The technology is helpful, but it still requires a human to make sure its working. And humans works better at these things when they have a good set of steps to check themselves.


4. RED cine reformats the disk and increments the reel number.


Seems like reel number should be incremented by the camera. Since otherwise you would need to keep all your cards in the order that you formatted them.

I actually think that reel numbers are less important in the digital world since each clip likely has a date and time of recording in metadata.

IBloom

[tj williams]

ibloom: other thread says longer to format in cam so good to format on set comp? Good amend to put reel number in camera, I wasn't thinking clearly about so many CF cards.
Poverty can do that to one....

The many posts about hiring good people carry across all set jobs. The plain truth is the budget for a data manager, will be similiar to the amount to hire a film loader..

I agree I'd like to see a consensual workflow come out of this thread. Here's maybe a beginning we can add to take away from or change to get a workflow which we can take to the bank.

1. record to either single media, or if allowed to dual card or card/disk on camera.
1A If using single recording, pull partially filled card or drive mag immediately after crucial shots for transfer.
2. download card to either dual LTO's, two separate drives, redundant raid. (S2 docking system.)
3. RED cine automatically does check sum (or perhaps faster or better data check) and automatically gives a simple answer that data is copied correctly and checked and then and only then gives the option to reformat the card or disk mag.
4. RED cine reformats the disk with the same project settings.
5. media is installed onto camera and menu item flashes asking for reel number.
6. Deliver one to post house.
7. Using a seperate delivery method deliver one to safe storage at a separate location from the post house.

[david farland]

In the risk adverse digital cinema world there are unique mobile, environments and just plain human error elements that don’t exist in the risk adverse financial markets that trade in billions of dollars each day and can’t afford to lose any of it.

However as data storage technology changes so quickly I’m sure digital cinema trails a little in this regard for a bunch of reasons, some valid…some just inertia and some just quirky 100 year old habits.
This area gets better, safer & cheaper every 12 months…..I hate seeing it shackled like the introduction of the automobile where people with flags worked out in front because others didn’t understand it.

I guess you need to redesign the model with these risk, time, environmental, human & cost factors taken into accounts.

I’m all for a list TJ, good work….but obviously each step will have its particular variances and qualifications.


Cheers,

[E.J. Sadler]

There are a few good reads for anyone who's concerned about stability of CF media -

http://news.bbc.co.uk/2/hi/technology/3939333.stm

http://news.digitaltrends.com/news/story/5140/blast_destroys_camera_flash_card_survives

http://digitaljournalist.org/issue0111/biggart_intro.htm


The vast majority of CF problems happen during handling - insertion/removal, static, etc. So if you're paranoid, having a static mat handy and powering off the camera is the way to go.

We use ChronoSync (http://www.econtechnologies.com/site/Pages/ChronoSync/chrono_overview.html) (http://www.econtechnologies.com/site/Pages/ChronoSync/chrono_overview.html) to copy data off of our CF cards/firestores, it will do a byte level file verification of the copy so you can skip the pre/post checksum. The copy is done to a mirrored raid, and then 'ChronoSynced' again to a secondary removable drive. The files are then burned to duplicate blu-ray discs,one of which goes to off-site storage.

We also don't re-use a CF or Firestore until all the stills/footage has been verified.

Once you have a workflow and stick to it, you'll wind up with a system that's far safer than film ever provided.


p.s. - Our still cameras write to CF and SD cards simultaneously, so we have a backup right out of the camera. So I agree it would be great to be able to send the data stream to two recording media for immediate back-ups

[Dan McCain]

I've been a first A.C. on many, many P2 shoots. The system I use is basic. When I get a "hot card" I pull the reprotect tab I put it in my left pocket (which is the same side as a powerbooks PCMIA slot). I then copy the card to 2 passport drives simultaneously. I open the folders to verify the size an physically make sure the files match. Then I pull the write protect tab and put in my right pocket. I only reach into the right pocket for new media, and never touch the left unless I am downloading. I have never screwed up once (and I've been on a lot of all night shoots where I am brain dead).

The beauty of the pocket system, is you can quickly shove the card into your pocket when you get busy. For example if you are in process of inserting the card into the powerbook and someone approaches you with a question you quickly put the card back into your left pocket so as not to get confused about what you were doing.

I have heard of people using ziplock bags and bins for organization but I like the pocket system because it is always right there.

I never hold a card in my hand, it either goes into one of the pockets or into the camera or computer. If you hold a card you can quickly forget if the card is hot or not.

[Chris Parker]

ha. mccainds. i love your system.

lots of good ideas here. but remember, there is only one negative on set. i think one CF Card, two hard drive copies, and MAYBE an LTO-3 tape would be as far as it ever needs to go.

[tj williams]

Hi David: Technology will change, maybe disk drives will be outmoded by years end? Maybe RED will allow simultaneous recording maybe it won't? flash cards will surely get larger, and thus more widely used. I'm trying here to incorporate everyones Ideas which we can all agree to to make a check list for the RED Dit. If you know about specific new tech. or gritty old habits which are no longer useful, I hope you will contribute your knowledge here! Perhaps this list will be to paranoid for some
It's easy to leave out steps you don't feel are necessary harder to invent your own additional steps. EjSadlers links above sure serve to point out that most of the time if we lose data it will be human error not technology failure. I think even nailing to a tree would come under human error!

1. record to either single media, or if allowed by RED Technology, to dual card or card/disk or dual disk mag. on camera.
2. Dit wears a grounding wrist strap and grounds camera during media removal. Possibly Dit asks for the camera to be powered
down during offloading? ChronoSync (http://www.econtechnologies.com/site..._overview.html) may be prefereable to check
sum. I strongly feel RED needs to provide the data check software as part of RED Cine to adapt a technology which they
stand behind.
2. If using single recording, pull partially filled card or drive mag immediately after crucial shots for transfer.
3. immediately pull the re-protect tab on CF media.
4. Place the recorded media into some particular bag or left pocket etc. where you always store the media until you get it out to
put into the onset backup storage.
(This is subject to the info. that handling is a majhor source of CF failure.)
4A Set computer and Esata or Firewire 800 raid are set up on an anit static matt.
5. download card to either dual LTO's, two separate drives, redundant raid. (S2 docking system.)
6. RED cine automatically does check sum (or perhaps faster or better data check) and automatically gives a simple answer that data is copied correctly and checked and then and only then gives the option to reformat the card or disk mag.
7. RED cine reformats the disk with the same project settings.
8. Assistant places the blank formatted media into some repeated storage bag or box and or perhaps keeps it in their right
pocket until it is reloaded into the camera (see () above)
9. media is installed onto camera and menu item flashes asking for reel number.
10. Deliver one to post house.
11. Using a seperate delivery method deliver one to safe storage at a separate location from the post house.

[I Bloom]

Hi David: Technology will change, maybe disk drives will be outmoded by years end? Maybe RED will allow simultaneous recording maybe it won't? flash cards will surely get larger, and thus more widely used. I'm trying here to incorporate everyones Ideas which we can all agree to to make a check list for the RED Dit. If you know about specific new tech. or gritty old habits which are no longer useful, I hope you will contribute your knowledge here! Perhaps this list will be to paranoid for some
It's easy to leave out steps you don't feel are necessary harder to invent your own additional steps.

1. record to either single media, or if allowed by RED Technology, to dual card or card/disk or dual disk mag. on camera.
2. Dit wears a grounding wrist strap and grounds camera during media removal. Possibly Dit asks for the camera to be powered
down during offloading? ChronoSync (http://www.econtechnologies.com/site..._overview.html) may be prefereable to check
sum. I strongly feel RED needs to provide the data check software as part of RED Cine to adapt a technology which they
stand behind.
2. If using single recording, pull partially filled card or drive mag immediately after crucial shots for transfer.
3. immediately pull the re-protect tab on CF media.
4. Place the recorded media into some particular bag or left pocket etc. where you always store the media until you get it out to
put into the onset backup storage.
(This is subject to the info. that handling is a majhor source of CF failure.)
4A Set computer and Esata or Firewire 800 raid are set up on an anit static matt.
5. download card to either dual LTO's, two separate drives, redundant raid. (S2 docking system.)
6. RED cine automatically does check sum (or perhaps faster or better data check) and automatically gives a simple answer that data is copied correctly and checked and then and only then gives the option to reformat the card or disk mag.
7. RED cine reformats the disk with the same project settings.
8. Assistant places the blank formatted media into some repeated storage bag or box and or perhaps keeps it in their right
pocket until it is reloaded into the camera (see () above)
9. media is installed onto camera and menu item flashes asking for reel number.
10. Deliver one to post house.
11. Using a seperate delivery method deliver one to safe storage at a separate location from the post house.

I'm going to add an idea, that will seem wierd but I think should be taken seriously: In addition to being numbered all your CF cards or drives should have kid stickers on them of various random things. Human memory is aided by association. If you see the cards with the horsey in the reader, you are less likely to confuse it with the other card you have with Mr. Yuck on it.

I'm at least going to do this. Maybe I'll start collecting stickers at concerts instead.

BTW I'm not convinced about grounding the camera, I think that's overkill.

Ian

[monolith]

I'm gonna add something funky to the mix here - timecode. Potentially as antiquated as reel numbers, except that in my P2 experience (which is also lowly, unpaid, abusive and extensive) I have found that TC is useful because it tells you that one clip goes before another clip, and that nothing was recorded in between.

Lining up the top and tails of your timecode for ingested clips in your NLE takes about 5 minutes - it can be difficult to concentrate on at the end of a shooting day, but always seemed to me the best way to ensure that ALL the footage is there at the end of the day.

When coupled with the P2 Store (which has a non-human verification that the data on the card and the Store are identical) I've only missed one card in the year and a half/60+ shoots that I've been using the P2 workflow. And that was in Jamaica. You get the idea..... (it was also before I'd discovered TC as my friend and ally)

Do with it what you will.


Ed

[david farland]

thanks steve.....nice work.
just a quick note (on break) for now.

As you've mentioned.....on critical shots you backup accordingly.
We do this every day when word processing. Say you were writing a novel in Word on a 80GB disk....
You wouldn't wait till you filled the disk before you backed it up!
When writing your piece you perform a risk evaluation on the content you just produced and backup accordingly.
You also evaluate the risks associated with your backup facilities and backup/distribute as necessary.
Single points of failure such a single disk/flash, single camera that houses all data, single location, single event, time of day (sunset) as you're alluded to, all present additional risks than require special mitigation.
If you don't have any redunancy (single flash card or housing) your backup regime would be in in the order of minutes whether that card/device can hold 4GB or 4TB.
So the choices associated with the data in my hand are about cost of recovery vs cost of backup.

Also ground strap is interesting. Normally i earth myself to the chassis before pulling cards and make sure i don't generate lots of static thereafter. i guess this is similiar to what your doing with the media. This may not be so effective if the system isn't grounded.

Cheers,

[Dave Weber]

LaCie's are diasters waiting to happen. Look at G-tech instead.
Funny you say this. I have a G-Drive down right now but all of my LaCie drives are running like champs (Knock on wood)

[sbaechler]

I think this thread is very important. A lot of people who ordered the camera haven't thought this problem out yet.

Reel or in this case card numbers are important throughout the workflow. Because it helps find media if the director wants to look at a particular shot. The camera assistant gives every card a number as he would do with a film mag. The script supervisor notes this number as well. And when the loader downloads he copies the media in a folder with this number. That way it will be quick to find a certain shot.
Maybe there should also be a track keeping of card-numbers like there is with film mags. In case there is a problem with the footage you know on which card this was recorded to. On the other hand if there was a problem then it's the card that was in your card reader when the error message appeared.

I like the idea of using LTOs. Maybe in combination with a RAID 5 that stays on the set and the LTO goes to the editor. It would be safer to ship a tape than to ship a harddrive.
Can you do multiple sessions on an LTO? You would not want to dump all of that days 100GB to the tape at the end of the day. Even if the loader doesn't get paid overtime ;)

[Cail Young]

The camera assistant gives every card a number as he would do with a film mag. The script supervisor notes this number as well. And when the loader downloads he copies the media in a folder with this number. That way it will be quick to find a certain shot.
Maybe there should also be a track keeping of card-numbers like there is with film mags. In case there is a problem with the footage you know on which card this was recorded to.

It's important to keep the same distinction the film production workflow has between "mag" and "roll" - the "mag" here is the CF card itself, and is worth tracking in the event of faulty cards. The "roll" is a specific set of data/footage on a given mag until it's copied and made safe. This is why the camera sets an incremental roll number on format rather than a mag number. The mag number (ID - I've seen letters used) should never change.

[Drew Mylrea]

Should this be in the Workflow section?

[Jay A. Kelley]

For myself, I am looking at copying to two drives.

One of them will be a RAID 5 and the other a faster larger RAID.

Off-Set these will be backed up to a couple LTO-4 drives. One Master, and one work tape.

We'll see.. This will change a lot I have no doubt.
Jay

[Paul Leeming]

I went looking on the weekend for a simple two drive backup solution and found this in one of the shops here in Akihabara:

http://www.visceralpsyche.com/misc/web_images/century_kd25_35ma.jpg

http://www.jpcentury.com/pro_con.aspx?id=P_00000005

Basically it looks like you use a USB connection from the host computer to write to two PATA or SATA HDDs simultaneously (in Mirror mode). It looks like it incorporates a verification function as well, plus it powers both drives from the included power cables.

For our on set workflow, having two 3.5" external HDDs wrapped in those silicone protection sleeves you can get for $5 should provide a great backup and handoff solution to the client without carrying around anything more than a CF capable laptop and this device.

It's priced at JPY 24,800 so I may pick one up to play with it and verify its abilities before committing to it for four cameras.

Any other solutions out there?

Paul

[tj williams]

trying to include what is new in the summary. any help appreciated!
TJ

1. mark each media mag (CF or Drop in disk/memory based mag.) with its own mag. number or symbol. This number or symbol is used to trace any recording failure to the specific piece of media used. This is usually called the "mag number" This along with the "reel number", or sometimes called "roll number" which increments each time media is replaced, should be marked at the head of the recording. They can either be put in as data or by shooting a slate at the head of the recording.

2. record to either single media, or if allowed by RED Technology, to dual card or card/disk or dual disk mag. on camera.

3. Dit wears a grounding wrist strap and grounds camera during media removal. Possibly Dit asks for the camera to be powered
down during off loading? ChronoSync (http://www.econtechnologies.com/site..._overview.html) may be preferable to check sum. I strongly feel RED needs to provide the data check software as part of RED Cine to adapt a technology which they
stand behind.

4. If using single recording, pull partially filled card or drive mag immediately after crucial shots for transfer.

5. immediately pull the re-protect tab on CF media.

6. Place the recorded media into some particular bag or left pocket etc. where you always store the media until you get it out to
put into the onset backup storage. (This is subject to the info. that handling is a major source of CF failure.also to avoid the possibility of confusion between the media just removed and the media which is ready, having been dubbed to be reused.)

7. Set computer and Esata or Firewire 800 raid are set up on an anit static matt.

8. download card to either dual LTO's, two separate drives, redundant raid. or perhaps some combination of slow raid/ fast raids/LTO's or seperate drives as seems good for the particular production. (S2 docking system. may be the best option at this time for uncompressed 4K as there is not yet a better system from RED)

9. RED cine automatically does check sum (or perhaps faster or better data check) and automatically gives a simple answer that data is copied correctly and checked and then and only then gives the option to reformat the card or disk mag.

10. RED cine reformats the disk with the same project settings.

11. Assistant places the blank formatted media into some repeated storage bag or box and or perhaps keeps it in their right
pocket until it is reloaded into the camera (see () above)

12. media is installed onto camera and (if RED will support this) A menu item flashes on the camera asking for the reel number.
the time code continues from the last media recording so there is not time code gap between reels. This will assure us that there is no reel missing on the backup, or that reels have not been copied to backup twice.

14. Deliver one to post house. *(when one of these must be shipped consider shipping LTO tape as it will probably survive better than disks in rough handling.)

15. Using a seperate delivery method deliver one to safe storage at a separate location from the post house.*

[Paul Leeming]

5. immediately pull the re-protect tab on CF media.My CF cards don't have a protect tab or slider, only my SD cards do. Do Red CF cards have a slider?

[Rocco Schult]

RED alert was supposed to be a temporary app, is that correct ?
I think theres still use for RED alert, even if REDcine is out.
A quick and dirty grading thing (exactly like it is now) and incorporating features needed On-Set, among them the stuff TJ collected and amended.

Just a little note on options and reformatting:

1. checksum source media/footage
2. copy files to one location, option of copying to 2 locations if available, option to reformat and unmount CF after copy
3. checksum destination, note if OK, or question what to do with options-dialogue
4. format CF or drive to record-ready status (incl. reel-number etc. if necessary)
5. write event into logilfe
6. unmount and hint for user if ready

Not meaning to leave everything out others mentioned.

If formatting is done by a RED app, RED could completely control the way and format how its formatted.

For me, REDalert could and should be the "need-2-be-done-on-set" app.


I'm going to add an idea, that will seem wierd but... I think should be taken seriously: In addition to being numbered all your CF cards or drives should have kid stickers on them of various random things. Human memory is aided by association. If you see the cards with the horsey in the reader, you are less likely to confuse it with the other card you have with Mr. Yuck on it.

BTW I'm not convinced about grounding the camera, I think that's overkill.

Ian, this is way funny, but exceptionally right what you're suggesting! I think I might go for a similar way (different icons;-)

And yeah, I think grounding the thing is too much.
Since I am assembling computers, I've never smoked a board.
But if you're shooting on plastic floor it might be worth considering.

[Rocco Schult]

Can you do multiple sessions on an LTO?

Yes, you can. Depending on the software, but generally possible.

But sending LTO to the editor is nonsense. You need an LTO drive at/on/in access for the editor instead of using the beenfit of ddirectly importing the files. We send HDDs since years, anf if one should break which of course is possible, you've got the backup.

[Chris Kenny]

But sending LTO to the editor is nonsense. You need an LTO drive at/on/in access for the editor instead of using the beenfit of ddirectly importing the files. We send HDDs since years, anf if one should break which of course is possible, you've got the backup.

Sending LTO might not be so crazy, if the expense of buying the editor a tape drive is not a big deal. The editor is almost certainly going to be copying the footage to his own RAID anyway, and LTO-3 and 4 are faster than most hard drives, so using LTO would probably actually make it faster for the editor to ingest footage.

[tj williams]

So I made some () and noted the more paranoid parts of the complete listing thanks for the new additions. Please double check and be sure that I am including your ideas clearly and correctly.
TJ

1. mark each media mag (CF or Drop in disk/memory based mag.) with its own mag. number color or symbol. This marking on the media itself, is used to trace any recording failure to the specific piece of media used. This is usually called the "mag number" This along with the "reel number", or sometimes called "roll number" which increments each time media is replaced, should be marked at the head of the recording. They can either be put in as data or by shooting a slate at the head of the recording.

2. record to either single media, or if allowed by RED Technology, to dual card or card/disk or dual disk mag. on camera.

3. (Dit wears a grounding wrist strap) (depending on static, plastic floor, wet etc.) Dit grounds camera to their body during media removal.(this is paranoid but is it paranoid enough?) Possibly Dit asks for the camera to be powered down during off loading? (ditto.) ChronoSync: http://www.econtechnologies.com/site..._overview.html[/url]) may be preferable to check sum. I strongly feel RED needs to provide the data check software as part of RED Cine to adapt a technology which they stand behind. If formatting is done by a RED app, (ie RED Cine) RED could completely control the way and format how its formatted. This also allows them to script the check sum of both the source and destination files and give a simple OK type output, so the Dit doesn't have to do the other step of matching the two.

4. If using single recording, pull partially filled card or drive mag immediately after crucial shots for transfer.

5. immediately pull the re-protect tab on CF media. (if this exists on RED CF media)

6. Place the recorded media into some particular bag or left pocket etc. where you always store the media until you get it out to put into the onset backup storage. (This is subject to the info. that handling is a major source of CF failure.also to avoid the possibility of confusion between the media just removed and the media which is ready, having been dubbed to be reused.)

7. Set computer and Esata or Firewire 800 raid are set up on an anit static matt.

8. download card to either dual LTO's, two separate drives, redundant raid. or perhaps some combination of slow raid/ fast raids/LTO's or seperate drives as seems good for the particular production. (S2 docking system. may be the best option at this time for uncompressed 4K as there is not yet a better system from RED)

9. RED cine automatically does check sum (or perhaps faster or better data check) and automatically gives a simple answer that data is copied correctly and checked and then and only then gives the option to reformat the card or disk mag.

10. RED cine reformats the disk with the same project settings. and automatically, writes event into log file, and unmounts media from computer. (or the Dit can do each of these tasks manually if RED does not support this.)

11. Assistant places the blank formatted media into some repeated storage bag or box and or perhaps keeps it in their right
pocket until it is reloaded into the camera (see () above)

12. media is installed onto camera and (if RED will support this) A menu item flashes on the camera asking for the reel number. The time code continues from the last media recording so there is not time code gap between reels. This will assure us that there is no reel missing on the backup, or that reels have not been copied to backup twice.

14. Deliver one to post house. *(when one of these must be shipped consider shipping LTO tape as it will probably survive better than disks in rough handling.) (It's necessary to consider that whoever receives the LTO esp. if a post house also has access to a compatable LTO playback machine.) (Editors preference to receive HDD or LTO probably good prep to work with Editor on short segment to see which works best in their workflow.)

15. Using a seperate delivery method deliver one to safe storage at a separate location from the post house.*
__________________
TJ

[I Bloom]

1. mark each media mag (CF or Drop in disk/memory based mag.) with its own mag. number color or symbol. This marking on the media itself, is used to trace any recording failure to the specific piece of media used. This is usually called the "mag number" This along with the "reel number", or sometimes called "roll number" which increments each time media is replaced, should be marked at the head of the recording. They can either be put in as data or by shooting a slate at the head of the recording.

It's possible that just like P2 cards CF cards each have a unique serial number or ID that is detectable by the software. In this case, tracing Mag Numbers can be automated. The same might be true for Red-drives.



The time code continues from the last media recording so there is not time code gap between reels. This will assure us that there is no reel missing on the backup, or that reels have not been copied to backup twice.


This is really important right, and it begs an interesting question that I think may have already been answered somewhere in the forum. It seems like every clip should be recording three channels of timecode:
1. A Freerun (jam synced to sound or other cameras)
2. A Record Run (useful for the purpose you just outlined)
3. A Time of Day (just generally useful see 1)

It's possible that the camera could always record all three all the time (tell me this is already the case.)

It might be nice then to visualize the order of recorded clips in REDCine, so that you can quickly solve problems or find missing footage in your left pocket.

Ian

[I Bloom]

By the way TJ and all, great work!

I put this up on the wiki:
http://www.redhax.net/wiki/Footage_Protocol_On_Set

I need to continue to rewrite it but I think we've done some really constructive work!

Ian

[Greg M]

So I made some () and noted the more paranoid parts of the complete listing thanks for the new additions. Please double check and be sure that I am including your ideas clearly and correctly.
TJ


__________________
TJ

You forgot to add:

1. Cover Post House windows with aluminum foil to prevent et transmissions from changing data on disks.
2. If crop circles appear in vicinity of post house, do not ship drives on third Thursday.

[tj williams]

DF
Thats silly crop circles only affect analog electronics.
TJ

[I Bloom]

I wish I was still shooting film. The problem with digital is the need to wear this stupid tinfoil hat

IBloom

[Rocco Schult]

Sending LTO might not be so crazy, if the expense of buying the editor a tape drive is not a big deal. The editor is almost certainly going to be copying the footage to his own RAID anyway, and LTO-3 and 4 are faster than most hard drives, so using LTO would probably actually make it faster for the editor to ingest footage.

You can plugin the drive and work from it, you can't with a LTO. Putting the data to tape is again another step as you can't write directly to the tape. You need to have the data laid down first and then archive them.
LTO is a backup medium and more than approproate at that in my eyes. But it takes double time compared to disc if worked from the disc.
All my current projects request the fastest way from acquisition to edit.
And if there should be a failure, yes, THEN theres a backup. ANd you can still dump the data onto the raid at night, after editing, but still gain the effect of profiting from the quick start.

Its another thing with uncomressed, there I agree with LTO being better, as you can't work directly from any drive-container (if its not a codex...)


By the way TJ and all, great work!

I put this up on the wiki:
http://www.redhax.net/wiki/Footage_Protocol_On_Set

I need to continue to rewrite it but I think we've done some really constructive work!

TJ and Ian: You did! Thanks for that!

[Cail Young]

I wish I was still shooting film. The problem with digital is the need to wear this stupid tinfoil hat

IBloom

Tape cans shut
Keep stock in fridge
Label EVERYTHING - including exposed film as 'EXPOSED' with the stock code even though it's encoded in the roll anyway.

I could go on. There are rites and rituals associated with shooting film, it's just after decades of them, they don't seem so odd any more.

[Colin Sheldon]

FYI The RED CF cards do not have a write protect tab on them (Which would be really nice!)

The other thing that I have found to be a good idea for RED and for P2 is to at least scan through the footage, or at least circle takes.

To me this is one of the real advantages of digital. I was able to catch soft focus, exposure issues, etc while there was still time to fix it.

I've also had P2 crap out in the transfer - the file was the correct size, but the footage was unusable - And the RED camera has that same potential. Watching at least some of the footage ensures that what you're backing up so carefully is actually usable.

Another thing to note about the RED is that when the camera formats a card it puts two small files on them. All you have to do is delete these files after you have copied the data and the camera will prompt you to format the card. That is how we ensured that the cards were empty. If the camera did not prompt, then the card was rejected and sent back to the DIT for rechecking.

Also, every time the camera rolls a new folder is created which includes the roll number. So on the first CF card all of the folders start with A001, the second A002, etc. This made it easy to determine if a card had been missed without resorting to time code.

[Clint Johnson]

FYI The RED CF cards do not have a write protect tab on them (Which would be really nice!)


That surprises me very much, I figured that switch would be a given. The 1st AC who removes the CF card is the only one who can write protect the card and the DIT is the only one who can write enable the CF card one they verify the transfer and formatting was good.

Without that little switch, the 1st AC would be relegated to doing something like throwing a piece of gaffers tape over the contact end of the card to make sure that it isn't accidentally over written before it is transfered.

But maybe I'm just paranoid.

[Colin Sheldon]

I agree, a way to mark the cards would be nice. The system we used, however, did work. If the camera did not request the card be formatted, it was sent back to the DIT. Also, the camera will not overwrite files, it will simply show up as "Media 57%" or whatever the case may be. If you choose to record on the card it will simply use the remaining space.

[tj williams]

Nice that the camera won't overwrite files. Clint your paranoia is a beeeeutiful thing. There is never enough paranoia!

[I Bloom]

FYI The RED CF cards do not have a write protect tab on them (Which would be really nice!)


I agree but I think it might not be neccessary. I'm not sure but I think RED is just rebranding CF cards that they know will work. Adding a write protect switch means more custom stuff which means added cost I'm assuming.

However if you look carefully at the workflow that TJ's outlined you can see it's not neccessary for the card to have that, since the only time a card is formatted is after it's been verified as completely copied. This creates kind of a one way street for footage.

On the other hand IMHO the write protect tab actually adds complexity and doesn't totally eliminate the possibility of human error. You can for example, forget to flip the switch, and a card will appear ready to format when it still has hot footage on it.

IBloom

[david farland]

TJ,
I'm still a little worried about the static procedures.
Say you're running round on synthetic surfaces (carpet etc) wearing synthetic clothing (non cotton/wool) with rubber soled shoes building up hundreds of kilovolts. If your camera is isolated ( not grounded to mother earth) and you swap CF cards in the camera you should be fine strap or no strap. Then you go over to your computer etc sitting on a static mat. Hopefully the next thing you do is clip your static strap onto the static mat which is electrically connected to the frame ground of the computer OR touch the computer frame. If you don't do all this there is a good chance you'll discharge those hundreds of kilovolts thru the CF card and thru the first piece of frame ground it touches (remember if you can feel a static discharge it's over 60K volts!). The best advice I can give is if you're doing DIT duties, don't wear synthetic clothing/shoes. Make sure after you move over static generating surfaces i.e. carpet or wriggle your arse on a synthetic (preferably leather) chair, you should touch the metal frame of your equipment before anything else! And realise if it's not earthed your won't discharge anything. Also note that long walks particularily with synthetic clothes/shoes over synthetic surface build up charge. It's as you learned in science rubbing different materials together is what builds up charge.

One last rant point not about static but keeping you alive! If you're gotta touch a piece of live (120/240V) equipment/metal....and you're not sure the frame is live or not, always touch it with the BACK of your RIGHT hand as the muscles in your hand will contract away from the live conductor and most of the charge 'hopefully' will pass down the right side of your body and not as much thru your heart.

Pardon the hijack!
Dave

[Brent J. Craig]

I don't believe a write protect switch is part of the CF standard as it is with SD cards. I have never seen CF with a switch, although that doesn't mean some enterprising manufacturer couldn't start adding them if enough of us requested it.

As for all this talk of grounding the camera, wearing wrist straps, etc - If the camera has a video-out line attached, and the video assist system isn't using any ground cheaters (which should be illegal on a properly-run set) then the camera is already grounded.

One thing we have done with P2 cards during the dry/staticy Canadian winters was to never pass them directly from person to person because of the potential for exchanging some free electrons (sparks). We either sealed them in their anti-static cases, or laid them on the table in front of the person doing the downloading. Probably a good practice for any flash media.

[I Bloom]

TJ,
I'm still a little worried about the static procedures.
Say you're running round on synthetic surfaces (carpet etc) wearing synthetic clothing (non cotton/wool) with rubber soled shoes building up hundreds of kilovolts. If your camera is isolated ( not grounded to mother earth) and you swap CF cards in the camera you should be fine strap or no strap. Then you go over to your computer etc sitting on a static mat. Hopefully the next thing you do is clip your static strap onto the static mat which is electrically connected to the frame ground of the computer OR touch the computer frame. If you don't do all this there is a good chance you'll discharge those hundreds of kilovolts thru the CF card and thru the first piece of frame ground it touches (remember if you can feel a static discharge it's over 60K volts!). The best advice I can give is if you're doing DIT duties, don't wear synthetic clothing/shoes. Make sure after you move over static generating surfaces i.e. carpet or wriggle your arse on a synthetic (preferably leather) chair, you should touch the metal frame of your equipment before anything else! And realise if it's not earthed your won't discharge anything. Also note that long walks particularily with synthetic clothes/shoes over synthetic surface build up charge. It's as you learned in science rubbing different materials together is what builds up charge.

One last rant point not about static but keeping you alive! If you're gotta touch a piece of live (120/240V) equipment/metal....and you're not sure the frame is live or not, always touch it with the BACK of your RIGHT hand as the muscles in your hand will contract away from the live conductor and most of the charge 'hopefully' will pass down the right side of your body and not as much thru your heart.

Pardon the hijack!
Dave

I don't think this is a hijack David, I think it's part of what we've been discussing. I think a lot of what you are suggesting can be solved by just touching the computer with your hand before you touch it with the CF card. Likewise on the camera. You could also touch them first with your tinfoil hat... that's my method ;)

If the shell of the CF card was a conductor, rather than plastic, the problem could also be solved since the actual electronics would be inside a Faraday cage.

Has anyone actually fried a CF card with static though? I'm just looking for some real world examples.

IBloom

[Jack James]

This is all very interesting, there are some very good points made here. However, on the checksum front, there are a couple of very important things to remember:

1. Unless the checksum is created by the camera itself during the shoot it is almost a moot point. Generating a checksum on a laptop requires reading the data in the same way as making a copy, and will therefore not be an indication of a robust file. In other words, if the file is already damaged by the time it gets to the laptop, you won't know about it. The checksum only ever "guarantees" (if that can be said about it) data from the moment it's generated.

2. A checksum doesn't ever provide any way to actually recover the data if it's later found to be bad. An infinitely better approach is therefore to generate parity: http://www.surrealroad.com/research/archives/tag/par/

[Brie]

Coming to this with a film background, I'm surprised that the Mag Labelling system used by many clapper loaders hasn't followed through to the RED workflow.
Basically it goes like this:
1. Have a label with Date or Time, Mag Number, other production details on it.
When you have a blank Flash card, you write the current Mag Number and Reel number on it, and as this card is inserted in the camera, the label is stuck to the side of the camera.
2. The Focus puller can tick off boxes/info on the label such as 4K/2K, frame rate etc. Possibly even the number of clips on the card. This acts as a check for the DIT when looking at the data.
3. When the card is ejected and given to the DIT, the label goes with it.
Stick the label on the card reader, so you know which reel is being copied - no chance of mix ups if there are multiple cards waiting to be copied.
4. When the card is erased, draw a big red line through the label and stick it on a daily "worksheet".

[Blair S. Paulsen]

Nice idea Brie. I have been doing a kinda half assed version of this but your program is much better. Fortunately I have been careful enough and focused enough that I have never lost a frame of footage, but, as my wife will tell you, my memory is not infallible - of course maybe if I wasn't using all my concentration on set I might have a few working brain cells when I got home :clown2:

[Etienne Caron]

This is a good idea.

As a camera assisant, i will bring to my DIT routine the classic camera report as well.

- All the info as the mag sticker.
- Detail of shoots and good takes.
- Info on the pre-programmed look use during the shoot and ASA (in the film stock section)
- Comments and extra info from DP to DIT
- etc...

But i'm sure that 2nd camera assistant on Red shoot do it similar...

No more "Prep. for video transfer" at the bottom of the camera report, but "Prep. for film transfer". :biggrin:

[Joe Carney]

For those using Windows...the built in command line utility XCOPY is a robust and reliable way to move files from media to permanent storage...including Network storage.

Here is a link that describes all the options..

http://technet.microsoft.com/en-us/library/bb491035.aspx

It's still available in Vista, but superceded by RoboCopy.

[Hans von Sonntag]

I like this thread because it shows how perception to a problem changes when theory becomes reality.

I shoot CF Cards only. For safety reason. I don't trust HDs they died too many times in my laptops.

We never erase a card on set. In a hectic environment this can end in a desaster.

All cards get numbered when full. Is a number missing - you know you are in trouble.

All cards get copied to a Nexto when taken out of camera.

Cards get reformatted only when all footage is on LTO3 tape and double checked.


This worklflow does need some cards. But as jobs come along there is the opportunity to "collect" cards.

Hans

[Blair S. Paulsen]

All this hard disk paranoia. If you are shooting in a rugged environment then CF is an easy call but for a lot of shoots the camera, and the Red Drive attached to it, is handled with great care and the risk to your data is minimal. That said, I swap out the Red Drives and back up the data several times per day because its the percentage move.

I do wish it was possible to write to RedDrives and CF simultaneously as that would provide an excellent data security level combined with great operational flexibility and convenience. Perhaps someday.

[Joel Kaye]

That said, I swap out the Red Drives and back up the data several times per day because its the percentage move.

So what's the best process for doing this?

I was thinking of using a portable mirrored eSata raid that could accept bare drives. You could use XCopy as indicated above and copy the RED DRIVE to one mirrored raid so you have 2 copies. You just bring a case of bare drives and store them in pairs with details taped to the drive - or written on white gaff tape.

Better ideas?

[mezmo]

The most elegent solution to all of this is larger faster cheaper RED CF cards.
Data recorded stays on a CF card or (camera data reel) for the duration of
the project and maybe forever if needed. Then you have a system similar to film and tape. The data is duplicated to a system like LTO3 and the cards and tape
get stored in a fire resistant safe back at the production office.
In 1080p land it will be possible to do this with the new Convergent Design
CF recorder. A feature film approx 10000 minutes can be stored at high
data rates on about 40-50 16gig CF cards with this new recorder. It's not
that far removed from 40-50 30 minute HD tapes in terms of cost and storage capacity if the cards can be purchased bulk at a discount rate and reused on the next project after film out or HD delivery.
Productions tend not to reuse HD tape on future projects and as you have a
back up LTO3 data tape all bases are covered for storage.
Look at bulk CF as an investment that can be used on many projects, the more thay are used the cheaper they become. You just have to fork out money to buy them in the first place, Mark them up 1-50, treat them like
film camera reels or source tapes and go for it.
Bigger faster cheaper CF may be needed for R3d @ 4K but this needs to be
the way RED Digital Acquisition works in the long run.
Cheers Mezmo

[Etienne Caron]

The most elegent solution to all of this is larger faster cheaper RED CF cards.
Data recorded stays on a CF card or (camera data reel) for the duration of
the project and maybe forever if needed.


Mezmo, your idea make so much sense... hear it before, but now i think its a very good idea.

The "positive" problem that i found about the Red Drive is its possibility of 320g. The best, from my pov, is if it was only recording 30 min. of 4K. When its full load, you do a fast check for quality control and deal with it like a film mag or HD tape. Ship it at the end of the day to the post-prod. office. No time consuming for data backup, etc...

Can do the same with 16 CF, will be a massive investment at start, but i guess that RED CF wont loss 20% of their value per year as any normal computer hardware.

[Fredrik Harreschou]

Can do the same with 16 CF, will be a massive investment at start, but i guess that RED CF wont loss 20% of their value per year as any normal computer hardware.

Why not? The capacity of flash cards follows Moore's law because they are produced with the same processes used to manufacture other integrated circuits. And RED will probably follow the pricing of the international market, with a premium of course.

[Blair S. Paulsen]

Joelnet, my general philosophy when backing up, whether in the field or back at the shop, is to create physically separate copies on individual HDDs. While I understand that some RAID topologies have excellent data security and redundancy protocols I like the ability to store the HDDs in different physical locations after the copies are made.

My theory is that physical damage to the enclosures/platters, fire, electrical spikes, floods, etc could take out several copies at once if they were all in the same RAID.

Just my two cents, cribbed from the estimable Evin Grant

[Joel Kaye]

I like the ability to store the HDDs in different physical locations after the copies are made.

I think my idea allows for that.

The idea is to have a bunch of bare drives in a case ready to get loaded with data. Take 2 out, pop them into a portable mirrored raid that accepts bare drives, copy the data, take the drives out (labled), seperate them and store each copy wherever you want. I'm not a raid guy, which is why I'm asking - but my theory is you should be able to grab either drive and mount it by itself later, right?

Anyway - I'm trying to come up with a redundant system that uses bare drives but doesn't require copying the data 2 times. I want to copy it once to two drives simulaneously. Just thought RAID 1 might be the fastest route.

[Joel Kaye]

Why not? The capacity of flash cards follows Moore's law because they are produced with the same processes used to manufacture other integrated circuits. And RED will probably follow the pricing of the international market, with a premium of course.

The other possibilty for those worried about hard drives is simply flash based storage. It can be much slower than the CF card you shoot to.

But I think if you dump data from a RED DRIVE every half hour and before you strike a set you're never going to lose a day because a hard drive fails.

RED should implement simultaneous flash and red drive recording though. Then you could cycle flash cards for a 1/2 or hour - when they are full you dump the RED DRIVE, verify the data and repeat. To me that's a pretty bullet proof short term solution.

I'm not worried about 2 hard drives sitting in secure storage BOTH failing. Even if they did the platters could be removed and the data recovered.

I'm more worried about the inevitable RED drive failure on set just after my actress lets the perfect tear slip down her nose and then a real UFO lands on the set.

[Jim Logan]

Fletcher's solution for backup -

http://www.fletch.com/rentalonsetdms.html (http://www.fletch.com/rentalonsetdms.html)

[Tom Lowe]

I lost an epic shot on P2 on my HVX due to lax worflow, so I will be paying attention to this thread.

[mezmo]

Mezmo, your idea make so much sense... hear it before, but now i think its a very good idea.

The "positive" problem that i found about the Red Drive is its possibility of 320g. The best, from my pov, is if it was only recording 30 min. of 4K. When its full load, you do a fast check for quality control and deal with it like a film mag or HD tape. Ship it at the end of the day to the post-prod. office. No time consuming for data backup, etc...

Can do the same with 16 CF, will be a massive investment at start, but i guess that RED CF wont loss 20% of their value per year as any normal computer hardware.
Hi Etienne,
Yes saftey in numbers I think is a good thing, the more your data is spread over multiple storage units the safer it becomes. A 35mm Feature Film is
usually stored on about 100 X 1000' rolls, tape on about 50-60 30 min reels.
Insurance is also about risk management, you lower the risk with more storage units and data loss is kept to 1/100th or 1/50th if a unit fails.
So keeping all data on a few hard drives or data tapes is very risky and uninsurable I would imagine with many neg risk companies.
Bulk CF data storage is the way to go IMHO, with data tape as backup.
Cheers Mezmo.