Thread: Adventures in Virtual Production

Reply to Thread
Results 1 to 10 of 10
  1. #1 Adventures in Virtual Production 
    Member
    Join Date
    Aug 2019
    Location
    Los Angeles, CA
    Posts
    33
    Since the industry shutdown all of my live action work, I decided to learn a new skill set and dived deep into the world of virtual production. I said it in college and still believe it today that narrative film will increasingly move towards virtual production. Big budget films already utilize a ton of CG environments and the Mandalorian showed us how incredible real time rendering on an LED wall can be.

    As a primer, I found this doc from Epic games gives a fantastic intro to virtual production: https://cdn2.unrealengine.com/Unreal...d2bf83138d.pdf

    Matt Workman has been documenting his journey with indie virtual production on YouTube and it's becoming clear how easy it is for even the smallest of budgets to utilize real time rendering. I wouldn't be surprised to see sound stages or studios opening up that offer LED volumes and pre-made virtual sets at an affordable price for even micro budget features.

    Here's what I did so far:


    Attached are renders from three engines: Blender's Eevee, Unreal Engine 4, and AMD ProRender. (please excuse the heavy JPG compression, shoot me a DM for an uncompressed version)

    I made the building meshes from scratch using textures from OpenGameArt.org and following Ian Hubert's lazy building tutorials. The man is a 3D photoscan from Render People. All of these assets were free.

    Lighting wise, I was more focused on getting a realistic image over a stylized one. Personally, the scene is too bright and the emission materials on the buildings need more nuance.
    • Blender's Eevee:

      Honestly, I'm beyond impressed with Eevee. It's incredibly fast. For those who don't know, Eevee is Blender's real time render engine. It's what Blender uses for rendering the viewport, but it's also designed to be a rendering engine in its own right. Most of Eevee's shaders and lights seamlessly transfer over to Blender's Cycles (their PBR engine).
    • UE4:

      After building and texturing the meshes in Blender, I imported them into UE4 via .fbx. There was a bit of a learning curve and manual adjustments for the materials, but ultimately I was able to rebuild the scene. The only hitch were the street lamps.

      In Blender, I replicated the street lamps using an array modifier which duplicated the meshes and lights. The array modifier doesn't carry over the lights into UE4 via the .fbx, so I had to import a single street lamp and build a blueprint in UE4 that combined the mesh and light.

      In the attached image, the street lamps aren't in the same place in UE4 because I was approximating. My next step is to find a streamlined way to import/export meshes, materials, camera, and lights between Blender and UE4. I believe some python is in order!

      As expected, UE4 looks great!
    • AMD ProRender:

      Blender's PBR engine, Cycles, is great. However, it only works with CUDA and OpenCL. I currently only have a late 2019 MacBook Pro 16". It's GPU is the AMD Radeon Pro 5500M 8GB. Newer Apple computers only use the Metal API, which Blender currently has no support for.

      Luckily, AMD has their own render engine called ProRender. Needless today, the results are great and incredibly accurate. This engine isn't a real time. Render time for the AMD shot was 9 minutes. This render is definitely too bright and needs minutiae everywhere.

    My final thoughts:

    Even though this is a group for Unreal, I'm astonished by Eevee, particularly how incredibly fast it is for running only on slow display GPU. (Again, Blender has no support for Apple Metal, so it defaults rendering to my Intel 630 GPU which is for displays)

    The next iteration of Blender will be utilizing OpenXR. According to the Blender Foundation, they'll slowly be integrating VR functionality into Blender and version 2.83 will allow for viewing scenes in VR. With that in mind, I'm definitely going to experiment with virtual production inside Blender.

    As for Metal support, I believe Blender will be moving from OpenCL/GL to Vulkan in the near future. From what I've found, it's easy to translate from Vulkan to Metal. (This part is a bit above my head as a DP, so I'll just have to wait or use a Windows machine)

    Does anyone have any useful guides that have a streamlined process for moving things between Blender/UE4? I'm from live action and love how easy it is to bring footage from Premiere/FCPX to DaVinci and back via XML. Is there something similar or is .fbx the only way?

    Correction:

    After further research, I discovered that Eevee can only run on the GPU (according to Blender's documentation). Eevee is in fact running on my GPU on my laptop, but it is running on the display GPU (Intel UHD Graphics 630 1.5GB). My MacBook Pro has 2 GPU's, one for processing graphics and one processing displays.

    Regardless, impressive.
    Last edited by AJ Young; 06-04-2020 at 09:38 PM. Reason: Corrections
    Reply With Quote  
     

  2. #2  
    Senior Member Joel Arvidsson's Avatar
    Join Date
    Nov 2008
    Location
    Sweden
    Posts
    2,974
    From watching the Ue5+playstation 5 demo I have not seen such progress in a long time in games. Cant wait to se how the UE5 engine will shake the industry.
    Epic #06696
    Epic-W #004069
    Reply With Quote  
     

  3. #3  
    Member
    Join Date
    Aug 2019
    Location
    Los Angeles, CA
    Posts
    33
    Quote Originally Posted by Joel Arvidsson View Post
    From watching the Ue5+playstation 5 demo I have not seen such progress in a long time in games. Cant wait to se how the UE5 engine will shake the industry.
    I agree! The dynamic indirect lighting in UE5 looks amazing and will disrupt the industry.
    Reply With Quote  
     

  4. #4  
    Senior Member
    Join Date
    Nov 2013
    Posts
    471
    I don't know much about UE, but Unity works with native Blender files. It's actually very amazing to me, I have .blend files in my Unity project folders and can edit them in Blender and the changes turn up immediately in Unity. This is really useful for things like UV editing, scale, pivot points etc. I recently learned how to use level-of-detail structures directly in Blender as well. These turn up in Unity with LOD support already in place. For a free piece of software, Blender is so good that it makes one wonder what Adobe does with all the money they make.

    I have a massive Unity project about to release. It's a virtual version of the famous Kalalau Trail on Kauai. I released a teaser version a year ago but this is the full eleven miles of trail. I did it mostly as a learning experience but it became something else. I am very fond of both Blender and Unity.
    Last edited by Med D; 06-03-2020 at 01:22 PM. Reason: words
    Reply With Quote  
     

  5. #5  
    Member
    Join Date
    Aug 2019
    Location
    Los Angeles, CA
    Posts
    33
    Awesome! I'd love to check out your project when it's released.
    Reply With Quote  
     

  6. #6  
    Member
    Join Date
    Jan 2013
    Location
    Toronto, Canada
    Posts
    62
    For anyone interested in joining in, I'll be hosting a panel discussion for the Canadian Society of Cinematographers on Virtual Production tomorrow at 2pm EST/11am PST. Check it out here: https://www.youtube.com/c/CanadianSo...nematographers.

    Karl Janisse, Associate CSC | Toronto, ON | P: 905.320.3319 | E: krj0013@gmail.com
    Reply With Quote  
     

  7. #7  
    Senior Member Tom Gleeson's Avatar
    Join Date
    Nov 2007
    Location
    Sydney
    Posts
    1,371
    AJ,

    Congratulations on immersing yourself into Virtual Production. Skills learnt working with the render engines building environments will greatly improve your communication depth and understanding when working on a Virtual Production.

    At the ASC we made this video looking at Virtual Production specifically through the lens of cinematography

    https://youtu.be/3g_GhufsQ7E
    Tom Gleeson
    Reply With Quote  
     

  8. #8 Very Informative - Thanks for the insights 
    Junior Member
    Join Date
    Apr 2021
    Posts
    2
    Here in Portland, OR things hit a halt during the pandemic and three local stages have popped up with these setups since. Have had the opportunity to play around with it as they were being setup and first marketed to the very small soundstage presence in Portland. Thanks for the links and illumination here filling in some of the gaps. Has been a mixed bag so far, but see the great potential in it with enough computing power and knowledgable techs who specialize (also still developing here)
    Reply With Quote  
     

  9. #9  
    Senior Member PatrickWebb's Avatar
    Join Date
    Dec 2020
    Location
    Norway.
    Posts
    167
    In Norway's South East there have been some really exciting developments into VP stages (much like the Stagecraft for the Mandalorian). I have been talking with some people here about it and it is really a hot interest. There is still a long way to go, especially with the content creation and the work force needed to drive projects. Larger film budgets will probably take the most advantage of this, but I am excited to see the smaller projects, maybe music videos bring out the creativity. I have been getting into Blender, as of last week, so I can learn and understand more of how VP works.

    It's really exciting stuff ~ but for a lot of people, midblowing and baffling. :D
    Patrick Webb,
    Director of Photography
    Heapsgood Productions, Norway.
    (www.Heapsgood.no)
    EPIC-X #06833 "Monsta" & SCARLET-X #01773 "Scratch"
    Carl Zeiss, Voigtänder & Russian glass
    Reply With Quote  
     

  10. #10  
    Senior Member
    Join Date
    Nov 2012
    Location
    Hollywood, CA
    Posts
    133
    Great thread. I started with Blender 2.8 last year with an intent to apply it to pre vis, and of course that lead me to Unreal engine. It was an exciting experiment. You've gone much farther than I did, AJ. I found Maya to be a much more intuitive interface, but although it has nothing to compare with Eveee, it's included Arnold renderer is worth the price of the program, now just $300 per year for the indie license. For what I'm doing I have no need of UE4, but am certainly keeping a close eye on Blender. An astonishingly powerful program that keeps getting better.
    WGA West
    NABET Local 53, IATSE Local 695 (inactive)
    Sony F65, FS700 Odyssey 7Q+
    Reply With Quote  
     

Posting Permissions
  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts