2d to 3D conversions in Movies...good enough?

[Clyde DeSouza]

Hi all,
I went to see two recent 2d to 3D converted films (Harry Potter and G-Force), simply because I was "teed off" with 2D to 3D conversion houses saying that 2D to 3D conversions are as good as and possibly "superior" to shooting in 3D

So I put in a document of my observations after the visit to the cinema.
I seem to be getting a few emails and comments that I know not what I speak for defending the "notion" that conversions are inferior to capturing in 3D in the first place.

For the record, I like conversions but for older movies as a novely concept. and I regard conversions like any other tool or "special effect" at the disposal of film-makers.

Here's the document:
http://www.slideshare.net/clydd/2dto3dconvertedmovies

if you have the time, please do browse it and leave your thoughts on it.

Kind Regards
Clyde

[Jonathan Payne]

Having personally worked at a 3D conversion house I understand everything you are saying and I have experienced what you are seeing. I can't speak for every conversion house but I have not heard either my former co-workers or the reps for other houses my current employer was getting bids from ever explicitly say that a conversion is a truly superior process in terms of ultimate 3d realism. They are constantly trying to sell their services instead on suggestions that conversion can be a better solution for certain types of films, shots or as you pointed out in your document, for legacy material/films. Several years ago I think their was a tendency to speak more unrealistically about the pro/cons of the process but this has been changing as both the houses and clients become more experienced. Ultimately what you are seeing when these problems arise in the conversion of these films/film clips is not a lack of absolute ability. There are instead compromises being made due to lack of time or money. Personally I am a quality purist because I feel like you only get one chance to do each shot right but sadly, as I'm sure you well know, time and money cause prioritization or outright compromise every day in the visual effects and animation industry. This conversion process falls right between those two disciplines and just as with either animation or effects, if someone who is experience and talented takes the proper amount of time and care they can make completely lifelike and/or totally satisfying final shots.

[Jonathan Payne]

also I think this thread belongs in the 3D forum, not the cinematography

[Clyde DeSouza]

Hi Jonathan,
Thanks for replying!
...."Ultimately what you are seeing when these problems arise in the conversion of these films/film clips is not a lack of absolute ability. There are instead compromises being made due to lack of time or money....."

Actually the problem is not lack of ability and/or corners being cut..
the main problem that needs to be addressed is

1) The claims that 2D to 3D conversions are "superior in many ways" to shooting in 3D - this claim has been made by very well known conversion houses (I use plural so as not to point fingers)

2) At least one of them has also said that they would recommed all 3D films being shot in 2D! and converted!

3) as in the Quote I copied of yours above... It's nothing to do with in-ability.. You CANNOT re-create a 3D dimensional world from information that IS Not there in the first place! It's as clear cut as that. All conversions are a 2.5D film at best.

Now what conversion houses will tell you is...sure you can, you just need to film a blank plate of the scene so that they can use it to "fill holes" in the scene etc etc.
None of this works as well as shooting in 3D . (either with physical cameras or CG stereoscopic cameras) and yes its easier to capture and edit in full stereoscopic 3D. for anyone who knows and wants to take the initiative to either learn or Hire someone who does! (no im not in the market for a job)

Conversions are just FAKE 3D. there is no information to create the "sides" of objects and or humans.. they are Filled in by a person using all sorts of "techniques". Additionally for those who do not know.. here's another little thing... One of the reasons why converted 3D looks "flattened" and conversion houses then call it "eye strain free 3D".. is because IF they actually extended the depth budget more for a scene where there is just a foreground and background , and not much inbetween.. then the "hole" to fill with missing information is a lot more.

Conversions are not a true representation of the real world as taken into context of 3Dimensional capture of the world since the days of old Stereoscopic photography and cinema all the way thru...to Today.
The sad truth is, Directors and producers either do not have the knowledge that they are creating 2.5D movies because conversion houses are not telling them these caveats.

And so far we have only seen rapid cut converted films... wait till other genres of films come in.. in so called "converted 3d".. Then see how much "immersion" is possible with this wrongly promoted methodology of creating 3 Dimensional films.

kind Regards
Clyde

(p.s yes this could be moved to the 3D section of the forum.. I initially posted this in Cinematography beacuse 2D to 3D is more of a cinematograpic style of filming rather than deserving a label of "3D" )

[Clyde DeSouza]

If you are interested in following the ongoing debate here's a link to the entire debate. It started out on LinkedIn, But links to downloading my document and a reply PDF from In-Three (a good conversion provider) is also there:

http://realvision.ae/forum/viewtopic.php?f=3&t=29

After reading the two documents my answer to their pdf is continued it the same thread there.

Kind Regards

[Jonathan Payne]

Clyde,

Due to the fact that there is no debate on the forum link you sent me (I'm not sure if others removed their comments or if they didn't follow the conversation from Linked In to the forum but either way I will reply here).

Point by point:

1&2: You're absolutely right that the variations on these claims range from sometimes right/sometimes wrong to completely untrue. Are you really that surprised? They're trying to sell their services. Say you want a vehicle to commute to work quickly and cut through traffic. You are thinking about a motor cycle and the conversion house is a car only dealership: They'll say: "You can get to work faster in a sports car because it has a has a higher top speed (lets say that's true for this example). It's up to you: they both get you there just as effectively, one is safer in the event of a crash and one lets you weave between cars during rush hour.

3: Which brings me to your statement: "You CANNOT re-create a 3D dimensional world from information that IS Not there in the first place! It's as clear cut as that. All conversions are a 2.5D film at best." and others like it.

Just as conversion houses are over simplifying the logic to suit their needs, you seem to be doing the same thing to either suit your interests, your opinions or your (possible) lack of experience with conversions. If you are very experienced with conversions then disregard the last portion of that sentence.

As you very well know, our stereoscopic vision naturally allows us to sense color, light intensity, depth and shape of objects depending on how close we are and how far apart our eyes are. All fine surface texture detail is sensed as depth cues by how we see light/dark and color but unless we have small surface details right in our face (within your personal bubble you could say) we can't actually sense their depth or true 3D shape. Again you likely know all about this but it's worth bringing up.

When you have good 2D-3D conversion tools, time and a good artist/operator you can completely reproduce the depth and realistic shape. you can put in as many details as you want to recreate everything. If you get good reference (clean pass from other parts of the shot or clean footage) or if it's possible to accurately re-create the missing information then you can create a truly accurate, completely life like 3D scene/shot without anything that is artificial or detectable in the clean-up/in-painting/missing info reconstruction. As an object gets closer and you begin to sense truly new 3D shape of folds in clothing, small strands of hair etc. etc. the operator can keep adding more 3D details to "person object" so that it perfectly mimics what you would see had it been shot stereoscopically.

Think of it like sculpting. To tell a master figure sculptor that they aren't faithfully recreating the real shape of a human is completely un-true. A truly great sculptor can create a completely life like figure that appears completely real except for the fact that it does not move and is not colored correctly.

The problem as you have pointed out is that if you don't have real talent and experience behind the wheel and you don't have a enough time and reference to help them nail it perfectly, you're going to have a subpar conversion that fails the potential of what 3D can do. Shooting good stereoscopic 3D is the most direct, fastest way to insure proper 3D because it has very little interpretation by an artist. That said, you CAN create TRUE 3D converted content with information on the sides and the like. To call it 2.5D is unfair, untrue and disrespectful for the people that create wonderful nuanced conversions. The only reason you don't have texture information from unviewable areas is because you have to match the original frame where you started with only 1 or 2 viewpoints. There is no other info around the edge that should be present.

Back from my first comment, it's just like animation/visual effects. It's EXACTLY like that. Great animation works perfectly. Bad animation destroys a sense of reality.

3 (cont'd): "One of the reasons why converted 3D looks "flattened" and conversion houses then call it "eye strain free 3D".. is because IF they actually extended the depth budget more for a scene where there is just a foreground and background , and not much inbetween.. then the "hole" to fill with missing information is a lot more. The reasoning behind this is unrepresentative to the point of offense. You can pull any object out as far as you want. Farther than is realistic in nature and farther than is comfortable to view. That said we did what was right for the shot only. Yes it's more work to fill in the missing background info but we always positioned our characters/points of interest based one where they should be in the scene not based on the amount of missing info. We weren't going to neuter the 3D, we'd have to do clean up even if it was artificially flattened so why not do it deep and at least have something more interesting to show for our work?

3 (cont'd): "And so far we have only seen rapid cut converted films... wait till other genres of films come in.. in so called "converted 3d".. Then see how much "immersion" is possible" Again, not true. An action style quick-pace short-cut film is the hardest to convert, they just happen to be the kind of film that is likely to be converted right now when 3D is still viewed by most as a gimmick. Converting a slower paced, less active genre film would be easier and faster to convert in a way that appears truly authentic.

So yes, conversion houses are overselling their wares, all types of animation, effects, production etc. houses do this. They're just trying to stay competitive. Yes as people continue to grow more educated they will understand the caveats, but frankly I think the dishonesty of selling yourself is the problem and it is in no way specific to the 3D conversion field of film making sadly.

With all due respect it appears that as your post goes on less and less of your statements appear to be backed up by experience with conversion.

In my opinion the REAL problem with 3D conversion is that to do it perfectly it takes much more time and requires someone with a lot of experience/talent. It's easier to learn how to properly operate and shoot in 3D. Knowing what I know I would shoot a 3D feature with 3D cameras because of huge speed gains and better control over reflection/opacity effects, not because it couldn't be done in a way that feels real during conversion. I'm also a perfectionist so while I would go with in-camera stereo 99 percent of the time, I am sure glad that the ability to do completely life like conversions exist when it's needed to fix/change a shot without re-shooting.

The real value in conversion is legacy material. Classic films that can be given depth if it makes sense and doesn't water down the integrity of the original film. Sadly studios make these decisions on financial merits and not creative ones most of the time.

[Clyde DeSouza]

Hi Jonathan,
That was a long post by you, but to keep focus I will first answer some of your outstanding questions about my ability/experience and then take it from there. Also I've disabled registration on the realvision forum till i can find a foolproof way of stopping viagra etc spam.

so..
1) Regarding my experience, Have a look at my Linkedin profile http://www.linkedin.com/in/clydesouza. I have been doing conversions since 2003. and for Autostereo screens which have to generate a lot more "missing depth" info so I have day in day out experience in that area from that date forward.

2) I have contributed in parts to companies such as Newsight, Philips 3d, and they can vouch for my "conversion skills" and expertise, and more so for my all round knowledge on stereoscopy - not limited to "software techniques to convert objects" only.

3) I also do not have access to Hollywood budgets for R&D but believe me, I am well in tune with what's state-of-the-art in conversion tools / trade secrets.

4) I refrain from "market speak" to counter any challenge on know-how etc. I do not answer questions with a "seems you do not have experience in this" shortcut answer, unlike commercial marketing manager type answers. if I'm wrong on something AND SOMEONE proves it.. and NOT JUST SAYS I'm wrong... then I always learn from that and stand corrected and move on.

So...
>3: Which brings me to your statement: "You CANNOT re-create a 3D dimensional world from information that IS Not there in the first place! It's as clear cut as that. All conversions are a 2.5D film at best." and others like it.

When you say others like it.. I can say that others HAVE NOT been made aware of both sides ,.. so they HAVE to like it. I am showing what's missing which conversion houses for (obvious commercial reason) will not show the missing side for people to make a choice

When you have good 2D-3D conversion tools, time and a good artist/operator you can completely reproduce the depth and realistic shape. you can put in as many details as you want to recreate everything. If you get good reference (clean pass from other parts of the shot or clean footage) or if it's possible to accurately re-create the missing information then you can create a truly accurate, completely life like 3D scene/shot without anything that is artificial or detectable in the clean-up/in-painting/missing info reconstruction. As an object gets closer and you begin to sense truly new 3D shape of folds in clothing, small strands of hair etc. etc. the operator can keep adding more 3D details to "person object" so that it perfectly mimics what you would see had it been shot stereoscopically.

Ok don't take my word for it .. put this question to more 'experienced and famous' Stereographers out in hollywood I'll give you some.. RayZone, Lenny Lipton, Phil Mcnally ... and see what answer they give you to the above. I was going to asssume (like you did on my argument) ..that you were only familiar with software and what you have learnt from conversion 'specialists' and did not do work with stereoscopic 3D capture, or studied anything on Psychological impact of stereoscopic 3D on the mind, scripting for a stereoscopic movie etc... But then I read your last paragraph again, and you do say that you Shoot in 3D, so i will have to un-assume :-)

just one thing... shooting a clean plate does not guarantee you a "visual info bank". Think about occlusion not only as cardboard cutouts of foreground / background. Think of occlusion as more than that, people occluding people, Shadows, a closeup of a persons shoes occluding a footpath when the camera zoomed in as she was stepping out of a car (beacuse the director was told to go-to-town shooting for 2d) .. So how MUCH filler plates can you shoot?

Isn't it completelty counter intutitive, to tell someone to shoot a scene (movie) in 2D, then RECREATE a fake virtual version of that scene (and entire movie), object by Object in a 3D software package... then match the lighting of the original scene with CG lighting.. then render a new Left / Right view????
-Instead of planning from the start, scripting for 3D, shooting in 3D and editing / compositing in 3D? All these thoughts are put into that document that I hope some savvy Director out there chances to read, for what it's worth . We Have to remember that it was Hollywood itself that said.. this time around we want "Immersion of the senses" in a 3D movie not gimmicks!

So skipping ahead, if you have seen the document I uploaded on the FLAWS in both G-Force and Harry-Potter and can actually PROVE that they are NOT flaws..until that time, I will maintain that 2d to 3D conversions be labelled 2.5D Movies at best.

Remember for someone like me..(who you perhaps thinks does not have experience in conversions to make such strong comments).. to actually point out these flaws WITHOUT having a remote control to pause the 3D film in the Cinema... but to catch these flaws on-the-fly... it speaks something about the state-of-the-art that the converted movies are going through as of Today! :-)

Now if the excuse is ..no time...no budgets... yada yada... then my simple answer is, if it's a job half done and is complex to do (yes I appreciate how much work goes in it).. Then DONT promote it as a METHODOLOGY of creating 3D movies, instead promote it for what it is .. a SPECIAL EFX TOOLKIT.

...and so this brings me back full circle to the ORIGINAL POINTS in my debate on Linkedin and in the Document which are:

A) I do not think its right for Conversion Studios to tell movie Producers and DPs to Shoot in 2D we will make a better 3D film by conversion later - This is blatantly wrong!

B) The flaws pointed out in the two movies G-force and Harry Potter - Solidfy (pun intended) point A above. That was the reason to upload that document

C) I still maintain that 2D to 3D is the best (only!) option for ressurecting and immortalizing older classics and blockbusters - THIS IS WHERE THE CONVERSION ARTIST'S ABILITY will SHINE!

D) Yes you can/should use conversions for fast paced stunts etc in current movies

E) The False (again people will say 'easy for u to say') belief being promoted by conversions houses that conversions give you Head ache free viewing - NO .. scripting for a 3D movie, Shooting for one, LEARNING NEW CINEMATOGRHIC GRAMMAR for 3D movies, is what will give you EASY , CORRECT, spectacular and head ache free 3D movies

2D movies are scripted for flat presentations, and Conversion houses touting that there is no disparity errors, depth can be controlled etc.. is exactly what they HAVE to do .. to make the conversion work. No wonder converted movies look so "flattish", and have a possibly higher chance of leaving you dis-oriented because it has been shot-for-2d.

To sum up I urge you to please get an answer from a few experienced stereoscopy professionals on that first paragraph in orange... and NOT from 3d artists who are doing conversions.


*All the capitalized words are for emphasis only and not to be taken as me shouting them out :-)

Kind Regards
Clyde

[Jonathan Payne]

Clyde,


Please note that I said if you have conversion experience, please disregard any assumptions in that line of reasoning. I did not assume for certain one way or the other. I did not intend to come off as condescending.

After your "2.5 at best" quote, when I wrote, "others like it" I meant: "And other comments like that" Sorry that wasn't more clear. I wasn't suggesting other people like it.

Sorry if I sounded combative in that post, I fell into the trap of getting passionate which is not too productive when we're trying to have think clearly and not let emotions become involved.

So that said, I have a few remarks to make on the last post:

I am agreeing with you on several points yet it seems you consider me in disagreement. so I will list them again without all of the rambling I'm prone to do:

>I would always choose to shoot in stereoscopic 3D if given the choice. It's more original 3D information, less interpretation, way faster.

>I did not say shooting a clean plate will guarantee you all the info to fill in missing details, it will help but not necessarily be the end all be all.

>I said countless times that it's not the best method of making new 3D movies but it is an option. I agreed that it's dishonest to tell filmmakers that conversion is a better general method.

>For whatever it's worth I think we each have experience with two different methods of 3D software conversion. My former employer's particular in-house software did not require us to re-light any object. All image/texture date was drawn from the original plates. This might be meaningless in our conversation but it case it makes a difference, I thought I'd mention it.

> We might have to agree to disagree but when i was a conversion and keyframing artist i had the ability to make any object as round and detailed as I wanted. From what little I know about other company's conversion processes I don't understand why they would be limited in their ability to create any sense of object bulge or shape.

> I don't know any specialists outside of my old coworkers and I don't claim to. I was just working in the trenches day in day out back then so that is where the bulk of my 3D conversion experience comes from. At the time our biggest limitation was creating convincing transparencies in scenes with objects like cloudy windows beams of light in fog etc. This is another reason I would shoot a film in Stereo were it up to me.

> I certainly never suggested that G-Force didn't have conversion errors. (I didn't see the 3D Harry Potter footage). It had a lot of good work and tons of errors that drove me crazy when I saw the footage. What stood out most in my mind were several of the closeups of Bill Nye (his face in 3/4 view tapered unnaturally in several shots, his glasses were always jumping between being too opaque and too transparently depth graded. There are at least a few bizarre incomplete looking conversions of certain cars, there are oversimplified object shapes in one extreme close-up of Zach Galifianakis' beard that made the mustache portion taper at the bottom like a round donut, objects popping in and out during the low 'dolly shots' of the running guinea pigs.)

>Bottom line, in my opinion I was able to recreate any shape but sometimes that required creating a very detailed version and this didn't usually lend itself to easy animtion/in-betweening (a shortcoming in my opinion). Thankfully as I became more and more disenfranchised by the fact that I was constantly required to dumb down more detailed models, the frustration pushed me to start experimenting with sculpture. For me that is the silver lining on the whole story. Back to our topic at hand....

It seems to me we agree on most points with the exception of our personal opinions on how extensively one can recreate a correct shape in 3D in conversion software. This might be a moot point since we both have not used each-others conversion process. As I'm sure we'd both agree your mileage with missing info reconstruction will vary dramatically on a shot to shot basis.

[Clyde DeSouza]

Hi Jonathan,
I am also at fault for getting too passionate and letting emotions creep into a debate.

I guess it's inevitable when something that you do is not only for making money, but is an artform in itself. (same hold true for some other mundane things, like my passion for single malts :-) )

I did read some of your other posts on other subjects and can clearly find that you never mean to sound condescending, but encourage a fair debate.

I guess we will keep that last point moot, on object detailing. one such company using basic geometry it seems can be seen in the few pictures in that document on the very basic looking 'blob' like head model of the actor. yes it could be that they refine it further, but then overlaying the video as a texture is akin to "bump mapping" as done in plain old CG softare to fake realism.

Rebuilding a 2D scene, object by object and re-rendering is so counter intuitive than actually shooting in 3D and doing stereo compositing in the first place!

In Stereoscopic 3D, these shortcuts show up as glaring errors. You can't fool the brain that much when it comes to stereoscopic 3d. That is also one of the reasons I lable it as 2.5D, not to belittle the artists who are working on the conversion itself.

Like one conversion house stated in another forum to the effect of: "We have invested time, effort and money and have fought for patents for our process". Fair enough.. that does not mean that because of those exercises, the output is justified as "superb" and/or "better" or even equal to the plain simple process of planning, shooting and editing in stereoscopic 3D.

many people in Cinematography who are delving into sterescopic 3D are learning that there is a new visual Grammar being defined even as we debate.. and learning that stereo movie making, things like "mise en scene" is more appropriate for immersion than cuts / Dof, and other conventional 2D film making techiques. Will this change the way movies are made, maybe, will 3D then be used for All genres of movies, we yet have to see...

....and im rambling again. :-)

Cheers!

[Jonathan Payne]

Clyde,

Once again I think we are in agreement.

For what it's worth, in my past experience we were creating, or at least could create, very detailed shapes that represent the hero characters throughout the shot very faithfully. Examples in the Harry Potter section of your document are indeed rudimentary and I was completely turned off at the similar conversions created for the sections of superman returns as well (I believe it was the same company). So we are in agreement at rudimentary shapes do not come close to creating a desirable outcome. Furthermore in my experience froms of camera projection and not bumpmapping etc, was vital to the success of the effect. If it helps make a valid argument for my opinion I have seen very little converted material released publically that I would consider truly successful in mimicing a realistic 3D scene. The Nightmare Before Christmas conversion was another example of one that felt hasty and was riddled with glaring errors throughout.

I agree conversion is much more complicated and laborious when compared to native stereo shooting which is why it has an important place in 3D cinema but in my mind it will never become the center of 3D cinema.

Jon