Within the Trees

Experience Type: Virtual Production Shoot

Role: Virtual Production Supervisor / Stage Operator

Platform: 2D Film

Game Engine: Unreal Engine 5.2.1

Additional Tools: Blender 3.6

Within the Trees was the culmination of the first year of a three-year Virtual Production fellowship training programme organized by Final Pixel Academy in collaboration with Nightsky Studios and Blazing Griffin. The first year of the programme lasted around 8 months, from December 2023 to July 2024.

The shoot was a collaborative effort between the trainees of the programme and a professional choreographer with the concept provided by the latter.

During the development stage of the project, I worked closely with the Virtual Art Department to ensure that the environments built in Unreal Engine are well-optimized to maintain the desired frame rate when displayed on the LED walls during the shoot.

On the day of the shoot, I served as Stage Operator, responsible for manipulating the view presented on the LED wall while ensuring that the scene was stable during production.

IDEATION

"Creating a choreographed sequence with Virtual Production from poetry."

The concept began as few verses of poetry presented by the choreographer which was then refined into a storyboard by the production team's director and director of photography. The final product was envisioned as a choreographed sequence of the talent dancing through a forest, eventually stumbling upon a mystical artifact.

DEVELOPMENT

"The challenge of ensuring stability without sacrificing aesthetics."

As the Virtual Production Supervisor for the duration of the project, one of the main challenges faced was to be able to ensure the development of a well-optimized virtual environment that retains the original aesthetic and atmosphere devised by the Virtual Art Department.

Above: A highly detailed environment of a forest created by the Virtual Art Department in Unreal Engine.

Above: Scene profiling tools in Unreal Engine providing a detailed breakdown of the cost of various components of the scene.

The process of optimizing the scene began with the reduction of density of 3D models and the resolution of textures to reduce the cost of rendering and time taken to process the assets during runtime.

Above: Unreal Engine's Statistics tab showcased the polygon count and texture resolution of each of the assets in the scene, providing information about the complexity of their geometry and textures.

Above: Use of image cards to replace high-poly meshes in the backdrop.

In addition to reducing geometry and texture resolution, most meshes at the peripheral regions of the environment were replaced by image cards of silhouettes of trees. These image cards were scattered all around the peripheral regions of the environment with fully three-dimensional trees and other assets being used closer to the position of the virtual camera.

Similarly, the complexity of materials was also reduced to improve general performance and frame rate of the scene.

Above: Specific parameters within the landscape material were adjusted to reduce the density of grass spawned across the terrain.

The steps taken to optimize the environment were done prior to the on-set day shoot to ensure that there were no complications with the scene's frame rate as it had to match that of the camera.

"Assuming the role of the Stage Operator on the day of the shoot."

During the day of the shoot, I served as stage operator under the supervision of the a trainer from Final Pixel Academy. Core responsibilities during the shoot included modifying specific aspects of the scene's aesthetic as per the guidance of the Director of Photography and Director.

Above: The above visual depicts a sequence created to animate camera movement for a double dolly shot.

With the above setup, contrary to the default setup in an ICVFX volume, the physical camera and the virtual camera are animated separately. The final shot can be seen below.

FINAL THOUGHTS

The project served as my introduction to how immersive technology can push the boundaries of 21st-century filmmaking. As the first year of the training program was intended to be an introductory phase, this project provided me with first-hand experience of the film development process. It also highlighted the various challenges involved in balancing creative concepts with the available technology and time constraints.