REDCINE to SCRATCH to Tape....

[Mark L. Pederson]

What no one talks about is that in scratch to do that final DPX render or what you would need to create those HDCAM-SR's its crrenttly 10:1 ie a 2 hour project will take 20. Imagine doing your dailies this way.

yes but you can play 1K in RT to tape with grade and it looks a hell of lot better than the QT wrappers - you doing 444 SR dailies?? that workflow does not appliy with logic to RED + Scratch

[M Most]

There are much easier ways involving much less render time.

QT wrapper transcode to Avid DNxHD 36 or higher
MetaCheater Quicktime meta data to Avid ALE
Relink and fast reimport Avid Quicktimes

There's an even easier way than that.
Install Avid Xpress Pro on the Mac you're using for this. Import just the ALE created by MetaCheater into a bin. Then do a batch import (pointing it to the QT wrapper files) directly into Avid MXF files using whatever resolution you want. The transcode will take no longer than any other method - including Compressor - and the files will already be in Avid format, ready to cut, with all metadata intact. The only reason to go your route would be to use multiple machines and split up the daily work load, in which case you could conceivably save some money by installing only Compressor on the additional machines. The efficacy of that would really be determined by the daily workload and its delivery requirements (i.e., how many of your shows are cut on Avid vs. how many are cut on Final Cut).

What no one talks about is that in scratch to do that final DPX render or what you would need to create those HDCAM-SR's its crrenttly 10:1 ie a 2 hour project will take 20. Imagine doing your dailies this way.

I don't know why anyone would do a full debayer, highest quality render for dailies. Dailies have two (sometimes 3) delivery requirements: Editorial, video viewing, and on occasion, a screening version. The first two can be produced in basically real time on any decent Final Cut system by using either 1/2K or 1K wrapper proxies, and playing them out via a video card (Kona, Blackmagic) to a DVD recorder for viewing copies. An even cleaner version can be done using Scratch, in the manner described by Mark. Screenings can be done directly from the r3d files in a screening room equipped with Scratch, also at real time, at least for 1K and hopefully at some point in the future for 2K. The only limitation at the moment is sync sound, but that will likely also be remedied.

The need for tape dailies exists primarily in television, which is one reason (among a number of others) why you haven't seen a network show shot using Red yet. It's my belief that the methods of television post will not likely change - which means that a workflow for television needs to basically mimic the film model, replacing processing and telecine with decent quality debayering and reformatting to 1920x1080. By what I've seen, Scratch can already do this pretty well by using 2K, rendering in "standard" mode and cropping on the output. Contrary to what others here have claimed, though, it is not going to take a lot less time that the reasonably common turnaround for film dailies, which is typically about 4:1 for transfer plus a 4-6 hour trip through the lab. However, overnight delivery is the goal, and if it can be met, it should be acceptable, particularly with the equipment costs being considerably less (a telecine room alone costs about $2 million - you can buy a lot of Scratch seats for that amount...). All of this is also the reason a number of us are hoping for either Red or a partner (AJA has been mentioned) to come up with a dedicated device to perform this specific processing (r3d->HD video at high quality with basic color correction) in real time.

And yes, full renders take a very long time. That's a reality that very few people here seem to notice or talk about.

[Jeff Brue]

fun stuff,
In regards to the HDCAM-SR layoffs. Its all fine and good that we're talking about this here and now but the above argument was for processing all of the material so that the HDCAM-SR's are the final masters hence the insane render time. I was told by Ted Schilowitz the other week to go through with the Full quality debayer at 4k and then downsample to 2k from that (which is what scratch does in its high quality 2k). This is what would be required for any HDCAM-SR layoff.

In regards to the quicktimes... they're fine for dailies. I'd put out there that 2k red quicktimes are still superior to HDCAM.

Lastly to the suggested Avid DV workflow, from the testing I've done there can be issues mainly due to the fact that other than DNxHD avid does not truly support 23.98 in any kind of data based editorial mode. I hope you'll hear me out on this. Avids SD 23.98 material must have 3:2 inserted for timecode to be correct. This was learned by us on Zodiac, avid doesn't play nice with real data i.e. system interchangeable data. Its a tape based system and is beautiful for that, anyting else can be hairy. So the answer is get the media composer and if you can deal with long import times great. For my lients I'm trying to come up with something more automated so its like a film LAB when they drop off their materials here.

[M Most]

fun stuff,
In regards to the HDCAM-SR layoffs. Its all fine and good that we're talking about this here and now but the above argument was for processing all of the material so that the HDCAM-SR's are the final masters hence the insane render time. I was told by Ted Schilowitz the other week to go through with the Full quality debayer at 4k and then downsample to 2k from that (which is what scratch does in its high quality 2k). This is what would be required for any HDCAM-SR layoff.

Thus adding to my statement about no network television programs using Red. Honestly, using Ted's method might produce the best quality, but nobody can afford to have 12-16 hour turnarounds for dailies. As you've already mentioned, one way of dealing with this is by quasi render farming the task - basically, splitting up the workload, either manually or by some scripting. Theoretically, this cuts down the turnaround by a factor determined by the number of machines you have, so in theory, using 6 machines would cut those 12 hours down to about 2, a good tradeoff (albeit about a $30K investment if you include the SAN connectivity you'd likely need), and likely the way facilities like yours (and probably mine) will go unless and until some kind of co-processing real time solution becomes available. Here's hoping that Redcine gets more reliable and trustworthy. And scriptable.

Lastly to the suggested Avid DV workflow, from the testing I've done there can be issues mainly due to the fact that other than DNxHD avid does not truly support 23.98 in any kind of data based editorial mode. I hope you'll hear me out on this. Avids SD 23.98 material must have 3:2 inserted for timecode to be correct. This was learned by us on Zodiac, avid doesn't play nice with real data i.e. system interchangeable data. Its a tape based system and is beautiful for that, anyting else can be hairy. So the answer is get the media composer and if you can deal with long import times great. For my lients I'm trying to come up with something more automated so its like a film LAB when they drop off their materials here.

Hmmm. I've done quite a bit of testing, and I haven't found that to be the case when the batch import method is used. I did use only MXF (not OMF), but I tested with DNxHD of various flavors as well as the more common SD 14:1 , although now that I'm thinking about it, I should probably check the SD a bit closer. It might be more significant if the metadata wasn't already embedded in the original source files, but in my case, the only information I really need from editorial is the reel and time code - i.e., what a standard EDL provides - because using Scratch for the conform allows direct recovery of that metadata from the original material. So unlike a Viper workflow, where other identifiers might be involved (shoot date, camera, LTO #, original Dmag ID) and other columns might be referenced via FilmScribe, the workflow through Scratch is simple and doesn't require that deep a database.

I completely understand your desire to have a common dailies workflow regardless of the editing hardware. The company I work for IS a film lab (and digital post facility), and we have the same need, which is one reason I found your remarks so interesting.

[Deanan]

Ted is referring to highest quality output as required by some network workflows.

[MichaelP]

When working in SD in Avid, it adheres to the NTSC format which is 30fps, NDF or DF - 23.976 and 24 are "created" as part of the capture process - this is done to maintain a bridge to the tape based world whether that be for capture or output. So the START timecode is reserved for that and a pulldown cadence needs to be defined for the relationship. One can always assume an "A" frame to 00:00:00:00. True 23.976 only exists in the HD world -

MetaCheater settings allow you to set the project type - so you can create an NTSC ALE in which to merge the QT refs from the camera or RedAlert!. The result would be the same as having captured from tape.

I'm not arguing the fact that there can be improvements to file only based workflows, but we are in a hybrid world where SD formats are still involved on some way on a daily basis in the post production workflow. Keeping the metadata distinct for all sources and record outputs guarantees guarantees lists, EDL. and soon XML wrapped metadata, etc.

As productions move to HD for offline, some of the issues will get easier, but not disappear completely. As far as a common dailies workflow; I think it would be great - And that will be achieved once all manufacturers get access to the native .r3d files.

Michael

[MichaelP]

I just did a bit more testing and although MetaCheater creates a valid NTSC ALE file, the START value needs to be adjusted to the 30 frame world and the native 23.976 TC value needs to be added to a TC24 column. I will let Jabez know this for an updated version when he gets the time. I have the 24/40 LUT done that he needs to use. The LUT will assume "A" frame for generated values of :00, :05, :10, :15, :20, and :25.

A similar adjustment will hold true for START in PAL based offline workflows with a 24->25 recalculation from 00:00:00:00.

Michael

[M Most]

When working in SD in Avid, it adheres to the NTSC format which is 30fps, NDF or DF - 23.976 and 24 are "created" as part of the capture process - this is done to maintain a bridge to the tape based world whether that be for capture or output. So the START timecode is reserved for that and a pulldown cadence needs to be defined for the relationship. One can always assume an "A" frame to 00:00:00:00. True 23.976 only exists in the HD world -

Yes, I thought about that after I posted and realized that it's really a matter of setting the pullin based on the first frame of the clip. Thanks for both pointing it out and working to adjust Metacheater to handle it properly.

As far as a common dailies workflow; I think it would be great - And that will be achieved once all manufacturers get access to the native .r3d files.

Only if hardware and software can get fast enough to do better extractions on the fly, and only if everyone has terabytes of storage. If that happens, there is no dailies workflow - you just point to the files. However, I don't see that happening any time soon, so until then, there is a need for creating files for both editorial and viewing that are high quality (if going to another mastering format, like SR tape) and low overhead - in other words, "pre-deBayered." And since different systems want to see different file types, it is necessary for the foreseeable future to do debayering and file format conversion at the front end of the process, as well as at the back end in some cases. The key to a common dailies workflow is the same with files as it is with film transfers: only the specific format specifications change, but the basic processing is the same. Each file goes through a deBayer and a resizing step, and the only thing that changes between projects are the specifics of the delivery format - file type, resolution, compression codec, etc. - which are all essentially output settings. This is very similar to current film workflow, where the lab, telecine, and color corrector used are all the same, with the only variables being 16 vs. 35, 3 perf or 4 perf (if 35), sizing specs, window placement if used, and what formats you're recording to.

[MichaelP]

Agreed -

m

[Jeff Brue]

The question though in regards to the conversion process of a quicktime 23.98 to a 30 fps NTSC project is where does the pull down get inserted for proper removal ? You're not capturing this, you're importing, importing doesn't insert 3:2 which would allow you to set a cadence to for proper timecode. This is even aside from the fact that red can do variable frame rates, and how those get imported with what variable time code?

Ah well guys I hope you have an answer this one has been bugging me for over a year, anyway Happy New year look forward to talking to you all more in 08.