philip lelyveld The world of entertainment technology

Spotlight: The Virtual Production of Europa

... The overall goal of Europa, as with ETC’s previous projects, is to experiment with and validate new cinematic techniques while exposing students to a professional environment in preparation for careers in film and television. For this project, the latest technologies to be tested included metadata, artificial intelligence in various forms and a new virtual production stage facility at Sony Pictures Studios, operated by Pixomondo. ...

The production employed cloud-first principles, allowing for global collaboration across different time zones and regions, including the US, Australia, New Zealand, India, Scandinavia, Iceland, and Canada. ...

Sony is one of Europa’s major supporters, providing Stage 7, its state-of-the-art virtual production volume at Sony Pictures Studios in Culver City, California. The stage features Sony Crystal LED B-series 1.5mm panels in a classic C-shape with a ceiling and Sony’s proprietary processors. Sony also loaned VENICE 2 cameras with a custom Virtual Production Tool Set to enhance their use in a virtual production environment. Virtual VENICE, Sony’s Camera and Display Plugin for Unreal Engine, is a unique tool that visualizes undesirable moiré artifacts before they can potentially ruin a shot, a key concern in virtual production along with the ability to preview wall content on specific optics and a suite of Color Calibrator tools geared to the VENICE’s sensor. ...

Artificial Intelligence for Virtual Production

Another important area of exploration for Europa is artificial intelligence and how it can enhance virtual production workflows. One of the critical challenges for VP is content creation. To successfully capture in-camera visual effects, the virtual environment displayed on the LED volume must be camera-ready before production commences. This is inverted to the more traditional visual effects workflow, where a significant portion of the frame is captured as a proxy via a greenscreen or other placeholder. Then, the visual effects are completed during a more extended post-production period. 

Creating this content can be a bottleneck because virtual production works best with photorealistic backgrounds. Artificial intelligence offers a potentially powerful shortcut by creating custom visuals in minutes rather than weeks or months. Europa used Cuebric, an AI image generator designed for virtual production backgrounds on a test shot, to explore its viability as a tool. Addy Ghani, vice president of virtual production at Disguise, whose show-running software is tightly integrated with Cuebric, says, “Cuebric enables creators to go from concept to the wall within a matter of minutes.” ...

It’s also helpful to distinguish between generative AI tools such as Cuebric and utility AI tools, which are much more commonplace in production workflows. Utility tools help to automatically apply metadata, identify elements within captured footage, maintain color balance and automate quality control. Europa aimed to leverage these existing utility AI while also exploring the more cutting-edge flavors of generative AI.  ...

See the full story here: https://virtualproducer.io