REDCINE to DNxHD

[Jiri Bakala]

We are Avid-based. Can we expect a workflow transcoding from Redcode RAW to DNxHD? It's not listed at the moment on the workflow chart.

[Anders Holck]

Avid offers a free Quicktime codec pack at their site, which includes DNxHD.
Unfortunately the mac version is not Universal Binary yet, so it wont work inside Redcine on the Mac.

[Jiri Bakala]

The new version of Avid IS universal binary. Also, how is a free QT codec going to help with getting Red footage into the Avid NLE?

[Anders Holck]

Redcine should be able to render into any Quicktime codec installed, If you install the Avid Codec pack, you can render directly into DNxHD, making it possible to "fast import" into the Avid without transcoding a second time.

Well what do you know they just made an update this friday (March the 23'rd) and the pack is now Universal:
http://www.avid.com/onlineSupport/supportcontent.asp?browse=&productID=208&contentID=10554

[Jeff Brue]

For the record avid's not really playing nice with any outside data workflow right now. I'm involved in the S2/Viper workflow like was just used on Zodiac. If you're working in final cut its the dogs bullocks as we have automated batch capture, automated archiving, and auto conform. All based around open XML. If you're working with Avid though right now it means digibeta layoffs, hand written EDL's, and manual conform.

Avid is still very much stuck in a tape workflow, which is very tough when you're talking about switching to an entirely file based workflow.

[Trevor Meier]

If you're working in final cut its the dogs bullocks as we have automated batch capture, automated archiving, and auto conform. All based around open XML.

How does FCP do batch capture, automated archiving and auto conform work in a data workflow?

A couple guesses... Are you using the 600A for capture? Standard tape backup for automated archiving? Auto conform... with media manager?

[Jeff Brue]

The automated batch capture happens over HD-SDI from a device we have called the idock. It takes the properly formatted dmags with all the meta data attached and generates a final cut xml for ingest. It then takes the DPX's and plays them out realtime over HD-SDI. We can then do whatever res/compression we want to them via our Kona cards. After that we have an automatic device that takes the dmag and archives it to two LTO-3 tapes. All with attached meta data and whatnot after that point its just a matter of generating an edl grabbing just those frames from the tapes and conforming.

These are all tools I hope to see Red or third party developers create, because I really don't believe there has been a good post production workflow devised yet for archival/storage and generation of material for editorial. Right now it sounds like a 3 Hour Red-Raid is going to take 12 hours to debayer (based on the Dalsa's debayering algo, not graeme's).

Right now the lines between production/prepost/post are blurring more than ever before. This happens EVEN more when you step into a data workflow.

[GlennChan]

Hey Jeff, I think that slow debayer is only if you want high-quality. I believe that Cineform RAW demonstrates real-time (fast/low quality) debayering that's good enough for offline purposes. Presumably Red has options for debayer quality + speed, and that the fastest speed is pretty reasonable (and hopefully faster than real-time?). Regardless, I guess this speculation would be settled by NAB time. People from Red have been saying to hold off speculation until NAB.

2- Curious: How come if the dmag spits out HD-SDI, why would you need to layoff to dbeta? (Could you not get a HD Media Composer?)

And why would you need handwritten EDLs?

[Jiri Bakala]

Just read a note by Avid's Adam Kranitz on Avid AXP board regarding the use of DNxHD codecs in a file-based workflow:

"Ikegami Editcam system currently offers the only direct import of DNxHD from a tapeless camera. Other options would require a transcode to DNxHD or capture via HD-SDI on an Adrenaline with DNXcel. The source code for Avid DNxHD codec is freely licensable allowing any vendor to add it to their product line. We'd love more manufacturers to add it as an option."

The need to use Adrenaline with DNXcel is a pricey option. A software transcode using REDCINE from Redcode Raw to DNxHD would be amazing. Can anyone FROM RED chime in on this?

[Gavin Greenwalt]

This should be reallly easy to test. I'll give it a shot when I get home. (@Premiere shop :( )

[Jiri Bakala]

This should be reallly easy to test. I'll give it a shot when I get home. (@Premiere shop :( )

To be specific, I am talking about using Redcine software for the conversion from Redcode Raw to DNxHD. Is that something you have access to and can check?

[Gavin Greenwalt]

... how would it be any different than using combustion or fusion?

Any program that uses Avid's Codecs to write to DNxHD Quicktime will do it in exactly the same way.

[Cail Young]

Shall we say it one more time? Let's try a mathematical proof:

REDCINE will export to, amongst other things, any Quicktime codec.

DNxHD is available from Avid as a Quicktime codec (and now, a Universal Binary!)

Therefore, REDCINE will export to, amongst other things, the DNxHD Quicktime codec.

Make sense?

[Jiri Bakala]

Yes, it makes sense. My question was more to do with the process; so, if I understand correctly, Redcine will transcode to DNxHD, which in turn would be imported to Avid for editing.

Let me then recap (and correct me if I am wrong):

-shoot in (example) 1080p RGB
-use Redcine to transcode to (example) DNxHD 220
-import to Avid
-edit
-export to delivery format (example), say HDCAM or HD DVD/Blueray

Makes sense?

[Jeff Brue]

Hey Jeff, I think that slow debayer is only if you want high-quality. I believe that Cineform RAW demonstrates real-time (fast/low quality) debayering that's good enough for offline purposes. Presumably Red has options for debayer quality + speed, and that the fastest speed is pretty reasonable (and hopefully faster than real-time?). Regardless, I guess this speculation would be settled by NAB time. People from Red have been saying to hold off speculation until NAB.

2- Curious: How come if the dmag spits out HD-SDI, why would you need to layoff to dbeta? (Could you not get a HD Media Composer?)

And why would you need handwritten EDLs?

Because you always end up working with a post house that has a 4 year old avid investment :-( And wants to cut using it. The hardest thing is that there are so many businesses in production that WON'T ascribe to a data workflow to protect a current investment in tape. It can be as simple as I'm willing to offer them a 1 TB hard drive for free to cut with to use DNxHD (which I can get too) but they will not because avid doesn't ascribe to the workflow.

Quite frustrating and something I expect everyone here to be faced with soon the (you want to give me WHAT!!! variable).

[Sean]

What would a digital workflow look like using RedCine and Avid DNxHD?

The steps in general must be:

1) Use RedCine to export offline DNxHD resolution
2) Use Avid Media Composer to edit to final cut
3) Use Redcine to export higher "online resolution" to external hard-drive (according to EDL list).
4) Using that EDL list (?), re-import the higher res footage into Avid Media Composer and get a good night's sleep while Avid renders it all.

I'm not sure if RedCine will be able to deal with an EDL list (no one is sure until NAB I guess). Actually, I don't even know how the EDL is generated in an Avid all digital workflow.

[Dominic Jones]

Yes, it makes sense. My question was more to do with the process; so, if I understand correctly, Redcine will transcode to DNxHD, which in turn would be imported to Avid for editing.

Let me then recap (and correct me if I am wrong):

-shoot in (example) 1080p RGB
-use Redcine to transcode to (example) DNxHD 220
-import to Avid
-edit
-export to delivery format (example), say HDCAM or HD DVD/Blueray

Makes sense?

To my understanding, RedCine will work from RAW files only, so the above would not work. This, however, would:

-shoot in 4K RedCode RAW
-use Redcine to transcode to DNxHD 220
-import to Avid
-edit
-export to delivery format (example), say HDCAM or HD DVD/Blueray

[Jeff Brue]

DNxHD is not a mastering format even though they advertise it as such. The algorithm is really quite messy. Compare it to DVCproHD not cineform or redcode rgb.

[Nick Shaw]

To my understanding, RedCine will work from RAW files only, so the above would not work.

I'm pretty sure Rob or Graeme said at one point that REDCINE will work with all flavours of REDCODE, RAW and RGB. It's a while since the discussion happened, but it was also said (by Stuart I think) that it would be possible to record REDCODE RGB unprocessed, so that similar processing could be done in REDCINE as with REDCODE RAW.

Not with the initial release of course, and "all specs subject to change" etc…

[Jiri Bakala]

DNxHD is not a mastering format even though they advertise it as such. The algorithm is really quite messy. Compare it to DVCproHD not cineform or redcode rgb.

It all depends on your delivery requirements. I work mostly in broadcast and believe it or not, most broadcasters still require DigiBeta (SD) as their primary masters. HDCAM or DVCPRO HD will be very accepted delivery formats for years and so going from DNxHD220 to HDCAM should work just fine, shouldn't it?

[Bruce Allen]

DNxHD is not a mastering format even though they advertise it as such. The algorithm is really quite messy. Compare it to DVCproHD not cineform or redcode rgb.

I think DNxHD makes a perfectly fine finishing format in the real world if treated with a little bit of care!

A company I worked for printed over 10 trailers to film from DNxHD. Going DNxHD meant they could online in-house, which saved them a ton of money. They spent that money on more DaVinci time and mixing time and also passed a little bit on to the client. The client loved the images and said it was a massive improvement from when they were onlining out of house. The out of house onlines had been done uncompressed on Avid DS or discreet fire suites.

Bruce Allen
www.boacinema.com

[Sean]

Over at DVXUser, MichaelP, the Avid moderator wrote in a thread about DNxHD:

I did a split screen test of [DNxHD]220 versus uncompressed and output to film and no one could tell me where the split in the frame was done.

[Jeff Brue]

Its not the issue of audience perception its the real world work practice of multiple recompression steps that invariably happen in post. So far the only compression methodology I'm willing to give a shot is a wavlet based method (I rather like cineform). Not just for the reduced cost of hardware but for just the logic of it, including its scalability with multiple processors.

As far as mastering formats forgive me its not about what you can actually see in the image after a final compression its what you're able to push the image to do to allow for finer granularity in control of the image. My classic case in point is tracking points for 3D tracking where the contrast neededs to be bumped up a lot to get a decent track sometimes or keying where artifacting in the hair causes the need to blur. I saw a rather horrible thing the other day when a vfx facility was given dnxhd 8 bit plates, they couldn't pull a key to save their lives out of it.


In regards to the realtime debayering well all the ones I've seen have some form of artifacting usually in the form of moireing so for any kind of editorial workflow where they HAVE to output from editorial and not do a conform they're not going to be willing to accept that as a file.

Ultimatly the thing I have a problem with is that the editorial and gear companies are flexing their engineering muscles and saying, yes you have this great new camera but we're going to limit what you can do with it.

So many people have so many purposes for this camera that the lines between everything is blurring and its making life extremely hard for anyone outside of a proscribed editorial workflow.

Its insane that you're going to have to run a transcode through RedCine into an avid dnxhd wrapped quicktime add compression and still have to manually edit the timecode, scene, and take info. The turn around time for something like that is slower than a film workflow.

Ok end of rant ;-) As I said I've been working on these exact same problems with the stwo/viper workflow. They're not easy because essentially we're talking about dragging the entire film and post industry into the future. It took stwo years to get their product to the state its at now (something that actually works). These same problems will be presented to anyone that actually attempts to make a living off of DAQ (digital aquisition).

[GlennChan]

I saw a rather horrible thing the other day when a vfx facility was given dnxhd 8 bit plates, they couldn't pull a key to save their lives out of it.

In my experience, difference keying with DNxHD actually kind of works (albiet DNxHD was used for acquistion... recording straight into an Avid MC). The key is noisy but matte choking cleans it up for the most part. So it might be for other reasons that DNxHD didn't work for them?
*The Avid crashed on some takes. Not cool.

[Jeff Brue]

Er I just don't get that mentality....especially when you can capture into final cut uncompressed 10 bit for a tenth of the cost.

[GlennChan]

Jeff you're right in that a FCP system would've been a better choice for that application, but the circumstances worked out that I ended up using the Avid. It was for a no-budget short I messed around with.
A- The Avid MC was free.
B- I couldn't work off the RAID, so I had to transfer my footage onto a FW400 drive and work at home. DNxHD compression, ironically, since I did have drive throughput issues.

2- Anyways I guess the point is... DNxHD is perhaps not ideal, but workable.

[Jeff Brue]

Ah indie you do what you need to do. I'm more along the line that if you're paying a vfx facility huge amounts of money you don't want them cleaning up with roto a badly artifacting matte.

In any case though I'm a strong believer in the right tool for the right job. I'm just still trying to figure out what and how avid fits into this entire mess (without buying a 45 thousand dollar media composer). Its like come on guys I'd rather buy another seatt of scratch.

[Jiri Bakala]

Jeff,

don't forget that not everyone with a Red camera will be shooting features. I for one, am a documentary producer and for most of my broadcast-intended work DNxHD will be just fine.

[Jeff Brue]

Well its not even so much the feature thing. Its just there are better codecs out there, and honestly I want avid to step up and meet the competition. As far as documentaries are concerned a wavlet based compression makes so much more sense in terms of file size. Thats why I'm hoping the 2k redcode RGB becomes a standard for editorial. That would be the perfect run and gun editorial codec, any other type of software conversion for an editorial codec is going to get time consuming quickly.

[GlennChan]

In any case though I'm a strong believer in the right tool for the right job. I'm just still trying to figure out what and how avid fits into this entire mess (without buying a 45 thousand dollar media composer). Its like come on guys I'd rather buy another seatt of scratch.
I think perhaps the problem is two-fold:

A- Avid charges a lot for their systems. MC isn't really designed for the highest online quality; you're looking at Symphony and Nitris (no compression). And for that price, you can get something arguably as good for a lower price.

B- Data/file-centric workflows aren't standardized and therefore not all systems work with them. If you can get your data to appear as a tape VTR, then things will work exactly like the traditional workflow. This is what I like about XDCAM HD. You also get the benefits of data (i.e. scene detection, faster than real-time transfer/ingest) in the offline editing stage if your offline editor supports it.

With Red, perhaps support for traditional workflows could be solved if they were to integrate with a DDR vendor like raveHD (which could act like a virtual VTR?).
*I'm not sure why something like this didn't work for you for your Viper projects. / It may not work for you.

C- In the future, perhaps vendors could standardize on a data workflow.
Standard wrapper
Standardized essence (i.e. all SDI formats, plus metadata for matrix, black point, white point, and gamma/transfer function)
Standard project exchange (EDL, AAF, or something else)

So far this stuff is a mess. For example, MXF was supposed to standardize the wrapper but not every app supports all its flavours.

[Mike Devlin]

In any case though I'm a strong believer in the right tool for the right job. I'm just still trying to figure out what and how avid fits into this entire mess (without buying a 45 thousand dollar media composer). Its like come on guys I'd rather buy another seatt of scratch.

Perhaps I am confused, but we only pay about $4k for media composer, much less than $45K if I understand your post correctly. Media composer Adrenaline is about $21k last time I checked, but you only need that for tape ingest, not data workflow. Media Composer (software only) does everything MC Adrenaline does and is actually a bit faster without the Adrenaline hardware in all of our tests.

I agree you need Symphony or DS Nitris for an uncompressed Online. But you can do a pretty nice job with 10bit DNxHD include pulling keys and tracking and lots of other stuff these days in MC.

[Sean]

Perhaps I am confused, but we only pay about $4k for media composer, much less than $45K if I understand your post correctly. Media composer Adrenaline is about $21k last time I checked, but you only need that for tape ingest, not data workflow. Media Composer (software only) does everything MC Adrenaline does and is actually a bit faster without the Adrenaline hardware in all of our tests.

I agree you need Symphony or DS Nitris for an uncompressed Online. But you can do a pretty nice job with 10bit DNxHD include pulling keys and tracking and lots of other stuff these days in MC.

And then can't you walk over to a post-house with your hard-drive for a single day's rental on their Nitris? That is if you even need to (i.e., after a sale or to meet broadcaster requirements). For most indie projects, I would bet this stage is never needed. And one day's rental doesn't seem so bad if you already have a sale.

[Mike Devlin]

Good point Sean. And with a bit of care the confrom from MC to Symphony or DS is much more straight forward than other alternatives. No EDL, just bring your timeline&media over. Depending on how much final grading you are doing it might be a couple of days rather than one. However, Avid lets you do an effects-rich HD offline with a (relatively) easy conform to the uncompressed online, so you can do more of the work in the offline. Horses for courses as several people have said, but it is not a bad option.

[Jörgen Persson]

The only workflow I would consider for an Avid offline is a REDCINE export to DNxHD MXF with translated metadata.

Importing a Quicktime file will give the offline editor a clip name and a UNC.
This will force the editor or the assistants to manually add information/metadata to the clip.
It also adds an extra step when importing the clips into the MediaFiles folder since Avid doesn't use Quicktime as a native format.

I really hope RED and Avid has a solution for this when the camera ships.
In order of convenience:

1. REDCINE exports directly to Avid MXF with metadata without the extra step through Quicktime.

2. Avid MetaFuze (will it ever be released?) imports REDCODE with metadata and saves as Avid MXF.

3. REDCINE saves an ALE and then you import the clips directly from REDCODE codec.

This is one surprise that I hope Red delivers at NAB.
A new camera is great, but without the "standard" editing workflow it will take a while before the industry adopts this new technology.

[Jeff Brue]

The 45k mark was what I was quoted for an HD-SDI solution. Essentially the same thing you get from a 5k final cut box. Personally I hope avid starts playing nice, but I think they believe themselves to be the adobe of post...taking a year to release MC 2.7 is kind of silly, I hope they have something better to present at NAB.

[Bruce Allen]

The 45k mark was what I was quoted for an HD-SDI solution. Essentially the same thing you get from a 5k final cut box.

Your FCP box isn't a solution because you haven't counted the hard drive array. Look at Mike Curtis' DV.com article for realistic FCP solutions.
http://www.dv.com/features/features_item.php?articleId=196602702
His mid-range system is 12k.

I totally agree, Avid needs to improve. But DNxHD means that you need to spend less on a fancy drive array - since its data rates are similar to uncompressed SD you can even re-use an old Avid array.

If you want to do long-form, want to go RAID-5 want to have some kind of backup system for your HD media (useful if you don't want to do offline / online), the FCP solution balloons in price because of the massive cost of uncompressed storage.

Avid needs to improve but Final Cut Pro needs something like Cineform built in ASAP.

Bruce Allen
www.boacinema.com

[Sean]

Avid needs to do way better marketing of their DNxHD workflow. It sounds like it actually offers an economical HD solution for home users. I have yet to see side-by-side comparisons, but some knowledgeable people tell me you can't tell the difference between the higher bit-rate DNxHD and uncompressed.

[Steve Connor]

Actually Bruce my 6K Final Cut system IS a solution .Cineform is actually coming to FCP but I don't need for an offline workflow as I use DVCProHD as my offline codec which doesn't actually need a fast RAID array at all, I've actually run DVCPro HD clips from my i-pod!. Most of the time we actually finish with it as well, the quality is excellent and from what I've seen compares well with the DNxHD quality.

However If I need to I can online at 10 bit uncompressed HD on my 2.5TB RAID array that cost me just $2000. My FCP system cost me in total with the RAID array some $6000. OK it's not RAID 5 but I'll accept the risk as my projects are always backed up.

No chance at all of matching that with an Avid system, I'm not knocking Avid but it's not cheap if you want Uncompressed HD.

[Bruce Allen]

Ah, DVCPROHD - gotcha, makes more sense now. I used that and like it but personally I think DVCPROHD can look bad when printing to film, while DNxHD at 10 bit and highest quality is just good enough to be a theatrical solution. I do agree that DVCPROHD probably a good solution for many other things and also agree with you that FCP kills Avid at uncompressed output prices. I just think that with DNxHD you don't really need uncompressed.

Also, cheap Avids can do DVCPROHD. Why don't you compare a cheap Avid to a cheap FCP system? The only place the cheap Avid falls down is if you want to go with a HD-SDI input and transcode to DVCPROHD, but since you're probably renting the HDCAM or D5 deck for something in the range of $700 per day, it is perhaps a false economy in some cases? Depends on the nature of the project, long-form or short-form, turnaround time, etc, budget approval stages, etc, I guess. I could see how FCP could totally beat Avid in value in many situations.

I too am excited about Cineform on FCP. If FCP had that, it would really have the format bases covered with a nice selection of quality / compression levels for all kinds of projects. I just wish Cineform didn't cost extra.

Bruce Allen
www.boacinema.com

[Jeff Brue]

Well again I think it comes down to what you have to do with the material you are shooting, and what your end goal of it is. I'm working on a horrible green screen shot right now and I'm having my ass saved constantly by having the higher bit rate uncompressed frame off the viper. If I did not have the capability to push the under lit green screen into no mans land this shot would remain horrible and not possible.

DNxHD as a mastering format gives vfx backend guys like me the chills. I KNOW whats being discarded, and after working for so many years to generate every pixel in 3D , you find out what you can do with them in compositing. Its not something that matters for a documetary about the penal system, but I guarantee you if you're shooting the latest doc on skiing or graveyards, having imagery thats of a non artifacting nature with greater bit depth does make a difference.