Z-Depth Channel anyone?

[Benni Diez]

Don't know if anything related to z-depth has been disussed yet. Well, the search didn't show anything, so...

I've stumbled upon several infos about a "Z-Cam" over the last few years (google it, you'll see). They developed a technique that allows an additional Z-Channel. First this thing was a bulky (I guess 16 pounds) add-on for SD studio cameras. But they seem to have improved that technology, now using it at a smaller (webcam-ish) scale mainly for gesture detection and stuff.
But they also developed some kind of chipset for use in combination with other camera sensors.

So think about this: A RED cam with not only those great features it already has, but also adding a Z-Channel! I guess once the tech works it's fairly easy to implement another channel into the r3d data.

Think about the possibilities: Changing the overall atmosphere (fog, v-light), masking objects by depth, enhancing DoF or changing the focal range. Even derive 3D-Models from the footage, or use it as additional data for stereoscopic stuff. Endless!

So, if one of the wizards over at RED reads this: Have there been any thoughts on that matter yet? Cause I truly believe you guys could pull it off if you got in talks with the inventors of the system. Or maybe figure out an own way.

The stuff exists, and it works, but by now only in SD afaik.
4k with depth channel would kill any competition once and for all, I betcha! And I'd cream my pants once more.

[Martin Weiss]

Didn't a similar idea get Red's first prize in a competition a good while back.

Does indeed sound very intruiging.

Maybe something for Epic and/or Red One's SDK?

[Benni Diez]

Here's something about how it works. Apparently it measures the time of infrared light impulses travelling from the camera to the object and back. Per pixel and at a decent frame rate.
http://www.kf12.com/blogs/techno/category/3d/

[Brian Reisdorf]

Didn't a similar idea get Red's first prize in a competition a good while back.


Good Memory. That's what got me and my company #97. I'm pretty sure the thread is still floating around DVXuser.com.

[Benni Diez]

$($)$%=$&=!!!! ...Damn, too late. :)

So, did they say anything about if they'd invest in z-depth?

[directedition]

How well does it work? I would imagine it would only be able to approximate the Z depth, and it would have problems at object edges as the perspective of the IR depth device would have some parallax. Not to mention that objects that don't reflect IR won't get a depth reading.

It's a fascinating idea, but I can't imagine it could be implemented without these kinds of problems.

[Noah Kadner]

This comes up a lot in the forum, methinks an enterprising 3rd party company is a better candidate to implement.

Noah

[Benni Diez]

I just think that, since there obviously are ways to do that, a little more research on that matter, done by the right kinda folks, could come up with something usable.
It still has its flaws now. Reflecting and transparent surfaces can cause problems, as does noise in general. But they seem to have developed some post processing that still makes the data fairly usable. Still, only in SD so far.
But who would have thought that we get something like the Mysterium sensor only a few years ago...

edit: There are also some post processes that can get a z-channel out of regular material. Based on motian detection algorithms and stuff. Also pretty unreliable I guess. But both combined could really make a neat depth image.
A bit like what Nuke does to enhance the possibilities with 3D-stereo footage using additional processing.

[Priyesh P.]

I fear the whole technology, both soft- and hardware will be way too expensive since it`s of limited use. I mean, when can docu and featurefilmmakers use such technology? It doesn`t have enough mass appeal to put it into every cam...so it`s going to stay expensive.

[MiguelOrtega]

I fear the whole technology, both soft- and hardware will be way too expensive since it`s of limited use. I mean, when can docu and featurefilmmakers use such technology? It doesn`t have enough mass appeal to put it into every cam...so it`s going to stay expensive.

when can feature film use zdepth? u mean besides every single visual effects shot made since it was created?

[Peter Majtan]

There is a SW solution that might work great with RED footage, I will do some tests and post it here later in the week...:

http://www.thefoundry.co.uk/pkg_plugins.aspx?ui=44213E51-76C0-4926-9D5D-3A8059449900

Scroll down to "F_Depth"... :)

[Benni Diez]

Yep, that's what I meant. But like 3D-tracking and other automatic processing plugins it heavily depends on the footage.

Who needs this?? Well, since RED is even more interesting for VFX stuff than anything else (quite a few features used it for their effects work although the main part of the production was shot on film) I guess it's not far behind HDR capability on many wishlists.

@Peter: That would be great! I'm working with AFX and they haven't bothered releasing Furnace 4 for it yet. I'd love to see some first hand tests!

[Priyesh P.]

when can feature film use zdepth? u mean besides every single visual effects shot made since it was created?

I`m not talking about movies for children, like those from Michael Bay or Brett Ratner but normal ones - AND features that don`t have a 10-200 million $ budget - low-and-no-budget movies is what I`m talking about - flicks that 90% of red one owners plan to shoot.

[Brian Reisdorf]

I`m not talking about movies for children, like those from Michael Bay or Brett Ratner but normal ones - AND features that don`t have a 10-200 million $ budget - low-and-no-budget movies is what I`m talking about - flicks that 90% of red one owners plan to shoot.

This was one of the main arguments during the contest 2 years ago. The response as to why RED should bother with trying to design something like a sensor based Z-depth channel was simply "Why not?". It's RED. They do whacky things like this and seem to do it better and cheaper than anyone else around.
Besides, even a system that wasn't terribly long range/had issues with transparencies would be utterly awesome in the world of compositing. Imagine being able to pull a pixel perfect greenscreen key just because your subject was seperated from his background by a few feet....just that much functionality would revolutionize the industry....I'm pretty sure even documentaries could use that.

[Benni Diez]

Docfilmers and low-budgeteers also won't need 4k for a while. Or 120fps, or Scope, or HDR.
But hey, why not do it anyway, cause it's frakkin awesome! :)
It's not about what's the biggest consumer segment, dude. It's about changing the industry. That's the spirit.

[Joseph Ward]

Sounds like NHK Axi-Vision Camera for Virtual Studio System.:nerd:

http://www.nhk.or.jp/digital/en/technical/01_2_virtual.html

[MiguelOrtega]

I`m not talking about movies for children, like those from Michael Bay or Brett Ratner but normal ones - AND features that don`t have a 10-200 million $ budget - low-and-no-budget movies is what I`m talking about - flicks that 90% of red one owners plan to shoot.

movies for children are cool

[Benni Diez]

Sounds like NHK Axi-Vision Camera for Virtual Studio System.:nerd:

http://www.nhk.or.jp/digital/en/technical/01_2_virtual.html

Even more sophisticated than the ZCam. But you can also see that it might take another few years to get something like this into a reasonable size. Dammit. :ranting2:

[Martin Weiss]

Who needs this??

Well, one could make much better blurs in post-production. Would also make Graphics easier, for example getting letters to disappear behind people.

If the resolution is high enough, it would make it quite easy to paint out unwanted elements, something that would vastly improve the quality of low-budget projects.


I'm pretty sure the thread is still floating around DVXuser.com.

When I am not on my iPhone, I can actually cut and paste, and thus found the original thread:

http://dvxuser.com/V6/showthread.php?t=59238 :weight_lift:

[Benni Diez]

Jarred Land wrote there:
"i dont expect this to show up on any of the first 3 generations of the Red camera"
That was 2 years ago.
So maybe the one after Monstro?
I'd call it 'Zeus'. For the 'Z', ya know... :)

Lotsa cool links in that thread, btw. Thanks!