Cinematic Still Imagery Using VP
- Jan 30, 2024
- 2 min read
Updated: May 30
OVERVIEW
The prototype demonstrates an accessible workflow to build a virtual production volume for cinematic storytelling. The project was a collaborative effort with Photography and Image Lens technicians at the Edinburgh College of Art.
The project was a catalyst for introducing a dedicated training programme for students interested in learning film production using Virtual Production technology at the Edinburgh College of Art.
THE CHALLENGE
The project’s main challenge was to assemble a stable workflow using readily available equipment that could produce presentable visuals for students to adopt without necessarily relying on professional-grade equipment.
ROLE & FOCUS AREAS
The project began my personal exploration into Virtual Production technology. In the collaboration, I handled the research and setup of a workflow to composite green screen footage in real-time with a virtual environment.
Additionally, I designed and built the virtual environment, which would then be blended with real-world footage.
SOLUTION & PROCESS
IDEATION
For accessibility, the workflow assembled for students and staff interested in the technology would feature real-time green screen compositing technology using existing equipment and hardware available at the university.
As a subject, we decided to recreate interesting cinematography that posed a challenge for scene lighting. For the prototype, a still from Eyes Wide Shut was chosen.
DESIGN
The prototype was intended for still image production. As with all forms of virtual production for filmmaking, the setup would feature a physical foreground and a virtual background. Using built-in techniques within Unreal Engine, the foreground could be composited onto the virtual background.
The scene’s lighting and blocking was iterated using Blender, giving clarity to the stage setup. Once approved, the composition was developed in Unreal Engine.
DEVELOPMENT
With a general idea of the atmosphere established, the development stage involved assembling a live connection between the physical and virtual. Camera footage was streamed in real-time to Unreal Engine, superposed over the virtual environment.
For the prototype, an Elgato Cam Link was used to feed data into the game engine for compositing.
The above visual is a time-lapse depicting the various stages of recreating the foreground's stage lighting. Through rapid prototyping, with refinements made to the background reflecting decisions made to the foreground, and vice versa.
The camera’s footage was streamed into Unreal Engine as a 2D media plate. The media plate was then rendered on a 2D plane matching the aspect ratio of the footage following which, the footage was strategically positioned in the virtual environment to match the perspective and scale of the physical scene with the virtual backdrop.
OUTCOMES & NEXT STEPS
The prototype was presented to upper management of Information, Technical and Technology Services and members of the CoStar (Convergent Screen Technologies and Performance in Realtime) project as a proof of concept to introduce Virtual Production practices and workflows into the teaching curriculum.
While successfully demonstrating the production of a VP volume using affordable processes and equipment, this particular compositing process lacked the ability to track camera movement. Subsequent experiments used an HTC Vive VR headset and Vive Trackers to track camera movement using the Live Link feature.
