Pull List / Conform / Batch Scale/Crop

[Mark L. Pederson]

Stuart -

I believe you mentioned that REDCINE is going to generate a "pull list" from an EDL and XML/AAF project files.

Are you going to keep it simple and only support CMX EDL format, or the whole nine yards (SONY, etc.)??

** FYI - a great resource on EDL management is:
http://www.edlmax.com/maxguide.html#format%

Are you thinking Apple FCP XML (which is not exactly the same as other NLE XML's)?

The reason I ask (or rather start the thread) is that EDL/XML compatability in terms of conform/batch re-process between NLE's and other applications is rarely done without issues.

Final Cut Pro Users have suffered through the de-bugging of Media Manager - with respect to speed effected clips, etc.

One thing that I face often is that very few filmmakers, and surprisingly few editors know how to properly "collapse" a timeline to generate a "true" cut/conform list.

I'm guess maybe you can "side-step" some of these issues (which CRIPPLED SC FINAL TOUCH and gave me many, many headaches) because (as far as I know) REDCINE isn't going to "conform" - theoretically it is just going to IDENTIFY which media is in the project via EDL/XLM - and then create a PULL LIST (I assume with a handle option) and then allow you to batch process/scale/crop/add LUT, etc. and then let the FINAL POST APPLICATION conform.

So, maybe the best approach is to make sure that ALL MEDIA in the "timeline" is in the PULL LIST - this way, if the editor has not "collapsed" the timeline correctly, it doesn't matter - as all media (used or not) that is in the timeline is in the PULL LIST -

Just wanted to get a PULL LIST thread going ...

[Don Woods]

This is a very important subject. It is the only way most of us will be able to post our 4K footage. I think allot of time and thought has been put into and should be. I can't tell you how excited I am to see RED CINE at NAB. I love the camera, I love all the other stuff. But what I want to spend hours talking about is RED CINE.

We are going to need a book and nice big book.. Or maybe not it will be that easy.

[GlennChan]

Curious: What do you mean by collapse?

[Thomas Mathai]

The Zodiac team used Final Cut for their offline and used the XML and seemed to be fine with it.

Moving between apps has never been easy, that's why there are always going to be skilled people doing conforms.

[Mark L. Pederson]

Curious: What do you mean by collapse?

Typically "collapse" all of the media to a single track in the timeline, with the exception of slit screens,wipes, etc.

[Mark L. Pederson]

The Zodiac team used Final Cut for their offline and used the XML and seemed to be fine with it.

Moving between apps has never been easy, that's why there are always going to be skilled people doing conforms.

Totally agree with you Thomas, I just wanted to address how REDCINE is going to identify what media it needs for the "pull list" feature - and that has less "issues" that need to be delt with than conforming.

However, there are still a few things to think about here - I do think REDCINE will need to identify and adjust duration for the pull list for speed effected clips, for example.

[Stuart English]

e-mail me off-list Off Hollywood, but you pretty much have it below -

"I'm guess maybe you can "side-step" some of these issues (which CRIPPLED SC FINAL TOUCH and gave me many, many headaches) because (as far as I know) REDCINE isn't going to "conform" - theoretically it is just going to IDENTIFY which media is in the project via EDL/XLM - and then create a PULL LIST (I assume with a handle option) and then allow you to batch process/scale/crop/add LUT, etc. and then let the FINAL POST APPLICATION conform.

So, maybe the best approach is to make sure that ALL MEDIA in the "timeline" is in the PULL LIST - this way, if the editor has not "collapsed" the timeline correctly, it doesn't matter - as all media (used or not) that is in the timeline is in the PULL LIST - "

[Nick Shaw]

It seems to me that a 'conform' from REDCODE media should hopefully be less complex than a traditional tape conform, as cleaning up the timeline before conform will not be so vital. Since presumably each 'take' will be a separate file on the RED-DRIVE, a lot of the time it might not be necessary to consolidate the time-line before conforming. One media file off tape may be very long, and contain multiple takes, or even multiple setups, when you only need one for the online. This would not be the case with RED.

If you worked this way, the files for your online would have a one to one relationship with those from the offline, giving a lot less scope for errors.

Nick

[GlennChan]

It seems to me that a 'conform' from REDCODE media should hopefully be less complex than a traditional tape conform, as cleaning up the timeline before conform will not be so vital. Since presumably each 'take' will be a separate file on the RED-DRIVE, a lot of the time it might not be necessary to consolidate the time-line before conforming. One media file off tape may be very long, and contain multiple takes, or even multiple setups, when you only need one for the online.

I don't think that's a problem with EDLs + tape workflow. Or did I miss something??

With a tape workflow, you only have to capture the clip you use, with handles + allowance for preroll. You also have to wait for shuttle speed, although that is not a problem for data-based formats with SDI out (i.e. XDCAM).

2- Collapsing: Got it. To me, it just seems... obvious... to collapse the timeline. From what I've seen, more common errors are not naming the reels correctly (5-8 characters max, depending on EDL), BREELs (these should be disabled unless doing a linear online), putting B on the end of a reel name(when there's no breel or timecode break), etc. This is in the context of SDI-based workflow, where an offline editor sends an EDL + media to the online post facility where they digitize through SDI. Which is different from FinalTouch.

3- For Redcine to pull all media from source clips would be extremely annoying if you have interview footage. It would have to convert the ENTIRE interview, and you have to wait for all the files to get transferred onto the online system's storage. If there is a need for this (I don't see why), it should be an option.

[MikeCurtis]

Having dealt with Media Mangler, I mean Manager, all too many times, if Redcine simply helps us identify which clips to use, and cooks them out to matching (or clean) color in the desired final size/colorspace/etc., then that is progress, and certainly Step One.

Step Two would be to do things like minimizing media with handles, script assistance to help automate the conform/build, etc.

I'm most interested in Final Cut Pro related workflows, but Avid obviously as well.

Ideally Redcine could be fed an XML or EDL and kick out a new one with links to the new media, but that tends to be more complicated than it sounds (witness The Horror, The Horror if Final Touch's XML non-work non-flow).

-mike

[MikeCurtis]

Glenn -

collapsing - for feature conforms, is SOP. But wouldn't it be nice if you could get away from that onerous task for onlining in FCP? Not all projects need to be fed to Scratch or FTHD or somesuch.

Item 3, the Long Shot Problem - YES I entirely agree, and my suggestion to the Red team has been to allow you to identify/mark/pre-cook subclips from source clips WITHIN REDCINE before kicking out the offline version if they can't cook subsections with handles.

It is a bit of a punt solution, but if you're stuck cooking the whole clip, at least go through and subclip it in Redcine BEFORE you kick out your offline versions.

And do it by order or operations, not where it falls in timeline - so first defined shot is shot_001 even if at end of the shot. Second shot defined is shot_002, etc. This avoids complications if you go back in and cut willy nilly, esp. if you do overlapping cuts. Naming conventions using timecode get cumbersome and problematic as well - what if I want two treatments of same footage, or overlapping subclips? Etc.

-mike

[Rob Lohman]

Yes, you can render out only sections of your source footage

[Steve Freebairn]

Will the process work with Premiere Pro? I think that Final Cut is great and all, but as of right now, only Avid and Premiere can edit 4k.

[Rob Lohman]

Which process are you exactly referring to?

[tj williams]

STUART QUOTE: "REDCINE isn't going to "conform"
So the offline is completed. its time in a film to conform to the digital intermediate format and take everything into the DI (digital intermediate) process. Which means if you originated on Film an HD or 2K timeline is conformed then the final color correction is made. This assures the most available color correction at the critical juncture when you are matching shots within a scene etc.

So we will be no better off than other work flows as we will be correcting in a constricted color space (compared to the original film or wave files)?

Encouragement here, for making an update that allows conforming in RED RAW to allow the filmmaker the best possible DI!!!!

[Steve Freebairn]

Which process are you exactly referring to?

I'm referring to using an EDL from Premiere to Online after editing in a lower resolution format. Thanks

[Stuart English]

From a market and customer support point of view, clearly Apple FCP and Avid "EDL's" are the two systems you would focus your efforts on first....

[Mark L. Pederson]

From a market and customer support point of view, clearly Apple FCP and Avid "EDL's" are the two systems you would focus your efforts on first....

I agree - but the dark horse here - PREMIERE is coming up fast ... and I would not take it for granted.

I think Premiere coming to the MAC is great for FCP - because hopefully it will stop the Apple ProApps guys from getting lazy ... Adobe has the ability to do software right. Even the Apple guys know LIGHTROOM nailed it - APERTURE was only close ... I sincerely hope that FCP doesn't dissapoint at NAB!

[Don Woods]

I Expect big things from Apple at NAB this year. Since last year there was really nothing new. And I agree Premire coming to the MAC was the big step. And I think that Adobe knows how to build software very well. If you think as of late Apple has kind of been following Adobe's modle of buying up software that works well and intrating it into there applications. So Who knows mybe it will be the big three instead of the big two.

[Mark L. Pederson]

I Expect big things from Apple at NAB this year. Since last year there was really nothing new. And I agree Premire coming to the MAC was the big step. And I think that Adobe knows how to build software very well. If you think as of late Apple has kind of been following Adobe's modle of buying up software that works well and intrating it into there applications. So Who knows mybe it will be the big three instead of the big two.

Maybe you are right, but I am preparing for major dissapointment. I don't think APPLE INC. cares so much about the high-end of the "pro user" group - because we are "too small". Steve is thinking phones and living rooms.

I expect NO new compositing application and no movement in 3D.

Watch Adobe buy LUXOLOGY - an emerging standout in 3D software, PRIME for acquisition - while Apple says ... the 3D market is "too niche" ...

Maybe I am wrong ... sure hope so ....

[Jeff Kilgroe]

Maybe you are right, but I am preparing for major dissapointment. I don't think APPLE INC. cares so much about the high-end of the "pro user" group - because we are "too small". Steve is thinking phones and living rooms.

I'm starting to have similar thoughts... Their high-end has been sliding lately, IMO. Then again, cell phones and an iPod for your living room. Wow, gotta admit those are two pretty [potentially] huge markets for them to jump into.


I expect NO new compositing application and no movement in 3D.

I agree. I don't think Shake is done yet, we may see some upgrades to it at NAB, but I doubt they replace it anytime within the next year.


Watch Adobe buy LUXOLOGY - an emerging standout in 3D software, PRIME for acquisition - while Apple says ... the 3D market is "too niche" ...

I really don't see Brad and the guys at Luxology selling out to anyone. They really have a great thing going there and I hope they keep it up.


Maybe I am wrong ... sure hope so ....

It's possible. I'm figuring we see updated Final Cut Studio apps at NAB, 8-core Mac Pro systems (hopefully with the new Stoakley chipset) along with some price adjustments, maybe even cheaper FB-DIMM memory. Beyond that, I'm not expecting much from Apple at NAB this year, maybe a new display model or two would be nice.

[Gavin Greenwalt]

Insert obligatory request that I've been asking for since before REDCine was announced:


Red Workflow:

Images are captured on set in 4k RAW and recorded to a portable hard drive.
The data is then ingested to an onset laptop connected to external hard drives for archival and editorial.

Once back 'home' the footage is generically processed out to an SD proxy using REDCine. Bulk naming is a possibility using simple scripting such as: TestProject_%Camera_%Tape_%Date_%ClipNum_SD.avi

Obviously some of the variables would be from the RedCode metadata and others would be defined within the REDCine project settings.

Within each of these processed RGB files there would also be some absolute clip and/or frame id meta data.

Within REDCine the software would also be logging a Reference Table for the footage:
REDCode4k_ShotA == Test_Project_CameraA….SD.avi
etc.

I would then hand off the SD proxies to an editor, and they can do whatever it is they normally do in their post workflow to edit. They will do no grading on this footage.

The editor will deliver an EDL to the post house (or using later mentioned features submit an EDL for processing to the post house.)

REDCine loads the EDL. Sorts through its Reference Table and starts exporting clips in one of several configureable manners:

Let's say for example that you have shot two takes that you have now edited.

You have a 401 frame long sequence named "DV_A_FOOTAGE"
and a 200 frame long sequence named "DV_B_FOOTAGE"

You edit this in say AVID and export an EDL which consists of

AVID EDL:
CLIP_0001: DV_A_FOOTAGE from frames 100 - 150
CLIP_0002: DV_B_FOOTAGE from frames 0 - 10
CLIP_0003: DV_A_FOOTAGE from frames 250-400

REDCINE looks at this EDL, sees that DV_A_FOOTAGE is only needed between the frame range of 100-150 & frames 250-400. And DV_B_FOOTAGE is only needed from frames 0 -10 The user has configured REDCine to use of a 10 frame padding in addition to the necessary frames the EDL requires for existing transitions in case they want to adjust transitions in the online, and a 5 frame slate.

So REDCINE now exports a new 4k online cut of two files consisting of:

4K_A_FOOTAGE: [5 frame slate][DV_A_FOOTAGE: 90-160][5 frame slate][DV_A_FOOTAGE: 240-400]
4K_B_FOOTAGE: [5 frame slate][DV_B_FOOTAGE: 0-20]

REDCINE now performs a simple math operation on all of the EDL frames (assuming frames start at 0):

REDCINE EDL:
CLIP_001: 4K_A_FOOTAGE from frames 15-65
CLIP_002: 4K_B_FOOTAGE from frames 5-15
CLIP_001: 4K_A_FOOTAGE from frames 90-240

The user exports this new EDL and loads it into Fusion, Combustion, AE, Smoke, Avid.

A second option would be instead of grouping 4K_A_FOOTAGE, split it up into two files:
4K_A_1_FOOTAGE: [5 frame slate][DV_A_FOOTAGE: 90-160]
4K_A_2_FOOTAGE: [5 frame slate][DV_A_FOOTAGE: 240-400]
4K_B_FOOTAGE: [5 frame slate][DV_B_FOOTAGE: 0-20]

EDL then is updated accordingly.

Settings to the user would include:
[]Reduce Footage (Instead of exporting the full clip frame for frame from the original)
[]Pad cuts by ________ frames (Contextual if reducing)
[]Create new Clip for Each Cut (Each cut based on the EDL)
[]Generate Slate [design]

Slate would be a ridiculously simple text creation tool maybe even following HTML basic formatting and XML parsing:
<ORIGINAL FRAME RANGE><BR>
<LENGTH IN FRAMES><BR>
etc...

This would greatly facilitate working with VFX and Color Grading where you don't necessarily want to send someone more frames than they really need. You don't want to grade frames you won't see and quite honestly people just might not have enough hard drive space for *another* copy of a significant portion of their film in 4k.

This way, if you only use 2 frames of a 1000 frame clip, REDCINE only exports the two frames (or maybe a few more if you have a defined Front and End clip padding). This would also greatly accelerate the REDCINE Process since it won't have to Encode, White Balance and LUT 1000 frames.

These clips are then graded in whatever grading application the user wants such as Combustion, Fusion, Lustre, etc… and rerendered into a new folder.

The EDL is loaded into either an online 4k editor such as Smoke or else loaded into a compositing application for render such as Combustion (w/ Automatic Duck) or else Fusion which supports EDLs natively. And rendered out as a master 4k.