SONY 3D.

[Neil B Smith]

Just to add.

My main line of work is in 3D animation. I use flat 2D images projected on to a 3D object all the time to give the perspective of 3D for that object.

Example: I want to create a 3D model of the Earth. I want to use one of the famously used NASA images of the Earth taken by the Apollo mission. I would create a 3D sphere then planar map the flat 2D image of the Earth on to the 3D object. Voila!! A 3D planet Earth. (After adding atmosphere glow etc..) I can lock together 2 cameras in my 3d software or use an available plugin that will render out 3D image (for use with 3d glasses). I would most likely render out 2 separate camera views then edit in post.

The Depth map will create your 3D modeling, the 2D colour video image will be your paint to the 3D model. What then comes from this is then up to you, either 2D or 3D stereoscopic images. The single perspective is destroyed when outputting to 3D as the software will split itself out from the centerline with 1 camera for left and 1 for right.

I think a Youtube video may be necessary for people to visualise my thinking. You may think I'm way off on another planet! I'm not, I'm just forward thinking. 2 cameras, or 1 camera with 2 lenses just seem a simple way out to create 3D, its not my thinking. 2 camera filming is not something new. Its like the internal combustion engine, because it works doesn't been we have to stick to it. Lets think out of the box and invent!!

[Neil B Smith]

If you have 3ds max and post DOF then you know the horrendous artifacting that results from trying to use a depth channel to apply depth of field even with a *perfect* depth map on a single layer.

Most 3D depth sensors today are about 320x240 pixels and extremely artifacty. Even if we improve the resolution and reduce the artifacts they won't really be reliable until they can output a perfect 3D scene. If they can output a perfect 3D scene then you're effectively using performance capture anyway.

The future future is performance capture and scene scanning and then actually rendering two eyes. There are just too many artifacts and limitations to using something like even Lidar.

Good points. I actually use 3ds max and know the limitations of Depth maps. 3ds max doesn't do a great job in creating a Depth map but it gives you great control over focus when using either After Effects or Combustion. It saves you from re-rendering the images if the Depth-of-field isn't the way the client wants. I believe it all comes down to how software evaluates the Depth map. After Effects would just blur the image as a 2D image rather than translating in to a 3D image (relief image), then applying depth-of-field blur. Maybe rather than a greyscale image you may need to store per pixel info of Depth. i.e. 800,450,175,50,225..... in a scanline process. You can therefore increase the accuracy beyond the 0-255 limit of a greyscale image, say increasing it to 0-999.

Lidar is an excellent example, thanks for sharing. Its very new tech, just see the possibilities for video and film work.

Getting back to video...
I do believe the technology for Depth map censors are only in their infantcy. Look at TV in the 1920's and look how its today. Now leap a few decades in to the future and see where things will be then. Its just going to be time before it happens.

[Jeremy Torrie]

You compare the pictured Sony rig to Bill White's tiny, 67 lb. S1-2K rig (they provided the cams for Saw 3D) it's an easy choice to make. They record uncompressed 2K to a proprietary box, and he's got something like 23 cameras.

I've seen the images first-hand, and they're incredible...and he too believes is true 3D, not this comped 2D stuff.

[Dan Hudgins]

The image is captured in through a single lens then split using a prism so that one copy goes to a Depth Map censor and the second copy goes to a CMOS or CCD censor. Both images for each censor are then saved in a single file known as RGB-D.

As I understand your quote,

The high resolution image of the "second copy" would not have full detail of the occluded parts of the background.

The "Depth Map Sensor" is an array of low resolution cameras that shoot using various parts of a lens that has a front element about 130mm wide or wider. The low resolution depth images captured off center of the front element will more coma, CA and Asymmetric distortion.

You cannot capture the background parts that are occluded with equal resolution for both eyes and in all respects equal to the other parts of the image with such a system. Nor can the sides of the subject and occluded parts of the background be recorded at high resolution as they would be with two high resolution uncompressed cameras.

The depth accuracy is limited by the interpolation of the so called "depth map sensor" which is a small array of cameras using parts of a common front element.

Fine detail on the surface of the subject would not be recorded in 3D by such a system, as it would be using two high resolution cameras with uncompressed recording, such as two 5K Epic with there "lossless" REDCODE option, perhaps.

What you end up with is something that would record cardboard cut-outs at various depths within the accuracy of the depth recording, but shoot a pole with writing going down both sides pointed at the camera and then try to read the printing on both sides, which you can if you shoot with two cameras.

If you have the single image using a large opening of the single lens, the printing will be out of focus on the sides of the pole pointed at the camera, so you will not be able to map an in-focus version of the writing over the 3D model of the pole in the computer.

In all cases the type of camera you decribe will produce lower resolution on the sides of the subject that using two high resolution cameras where the DOF can be adjusted better.

You can use an array of high-resolution cameras without such a large single lens, and maybe interpolate the 3D depth from those, but you will have high resolution surface depth data, rather than a smoothed and interpolated 3D surface.

The viewers eye-brain can model the 3D depth better than the current type of "depth map sensor" just by looking at two high resolution 3D images and see fine texture on the surface of the subject.

What you are talking about seems to be the Equivalent of the the NTSC system for 3D,

1) You have higher bandwidth and resolution for a "central" single image to map over a lower depth resolution 3D model.

2) The 3D depth data is lower resolution than the image, so motion blur at edges needs to be resolved so that the 3D model does not form odd depth extensions between the background and the foreground.

3) Occluded parts of the background and sides are reporduced with depth that may or may not be right and at lower resolution.

4) The edges of the 3D model may not align with the edges of the high resolution central image that gets mapped over the 3D model.

5) The 3D depth noise needs to be filtered, so the motion and texture of the 3D model could be distorted by low pass filters.

In NTSC the chroma is lower resolution than the Luma, in such a 3D camera, the surface texture and depth are at a lower resolution than the Luma, something that true Stereo 3D using two high resolution cameras would not be afflicted with, and in a two camera setup the DOF of the occluded parts of the sides and bakground that show in the other-eye's-view would look natural, not distorted.

Adjusting a 3D rig is not hard if you take the 3D glasses OFF and look at the stereo pair superimposed. You cannot monitor on a small 3D monitor for projection on a large screen because that tends to have people set the divergence to wide, as divergence on the screen no matter what size cannot be larger than the spacing of the eyes, so its best to have NO divergence so that the images can be viewed on "any" size screen from "any" distance.

Do you have any 4K resolution 3D models captured by such a camera and processed in software that can be rendered without the image mapped over them, of subjects with motion blur? That is like a plaster model on a plaster landscape?

[Neil B Smith]

http://www.sony.net/SonyInfo/News/Press/200910/09-117E/

Different to what I'm thinking on, but demonstrates that 3D images can be captured using a single lens system.

[Dan Hudgins]

http://www.sony.net/SonyInfo/News/Press/200910/09-117E/

Different to what I'm thinking on, but demonstrates that 3D images can be captured using a single lens system.

I think sony is using that optical way of making sure that there is no disparity in the plane of focus so that the 3D LCD TV sets look sharper, if you have disparity on Lenticular TV sets the parts out of the screen plane do not look as sharp.

Also for the 3D Blu-Ray I think their encoding system works better when there is no disparity in the plane of the focus where the MPEG encoding needs the most bandwidth, having both eyes images the same at those sharp points reduces the difference signal you need to record.

Having that large lens front element makes the camera much larger and heaver than it would be with something like two SI-2K mini.

The same for your single sharp image plus several smaller images over the area of the large camera front element.

With two cameras you can adjust the spacing to be wider for subjects in the distance, but with a single lens camera you are limited to the width of the single heavy large front element and its heavy optics that back it up, so you cannot make a small light weight 3D camera that sees the sides of the subject in the distance with a moderate size single large front element.

You also have the problems of Asymmetric lens aberations from using the parts of the optics that are not paraxal.

With two cameras you can stop the lenses down to get better DOF on the sides of the subject, and both lenses are centered on their sensors so the aberations are symmetric.

==

Your idea of a 3D capture device is useful for some kinds of work, yes, but it is not the best way to capture stereo-3D images to reproduce all the Subtle surface texture that makes viewing 3D images all it can be.

If people only see rounded off and cardboarded 3D rather than lifelike 3D textures, it will not give them the maximum benefit.

When a film producer makes the decision of what way to photograph the subject matter, he should understand the tradeoffs.

[Gavin Greenwalt]

. 3ds max doesn't do a great job in creating a Depth map but it gives you great control over focus when using either After Effects or Combustion.

Great control and almost always pretty lousy results from a single layer rendered (which is all you would get from a camera)

Spitting out a single render and applying DOF only works when nothing is occluding anything else to any significant degree.

Similarly if you took a single camera and projected it back onto 3D geometry you then have to *create* everything that the second eye couldn't see that was occluded. The most extreme example would be where one eye sees nothing but fence post and the other eye sees everything else. If your camera only sees fence post your depth map will just see fence post.

You would get horrendous artifacting:


(Super quick and dirty mocked up crossed-eye stereo example rendering.) Aka... cross your eyes to view it.

And that's a "best case scenario" of managing to somehow magically recreate the 3D scene perfectly. With just a depth map it would be 'displaced' and you would have 'smearing' of the 3D geometry.