SUMMARY_22

By Sten-Kristian Saluveer

The one-day conference took place on October 13 in the cinema of the National Film School of Denmark and presented 18 speakers from Denmark, Finland, Germany, Ukraine, the United Kingdom, Estonia, Poland, and the United States. The professionals ranged from studio owners and visionaries, virtual production supervisors and experts, Directors of Photography, tech and infrastructure companies, media futurists, VFX supervisors, commercial and independent film producers, and service production professionals.

The shift towards Virtual Production technologies has undoubtedly been one of the greatest (r)evolution that the film and television industries have experienced in the past decade. Within a few years, techniques such as combining computer-generated sets with live performances in LED screen and camera tracking-powered studios have not only become a staple new toolkit for blockbusters but are rapidly becoming democratised globally.

The year 2022 marked a turning point for Virtual Production in Europe. Not only have Virtual Production technologies become more democratised, with production knowledge becoming increasingly accessible outside major studios and tentpoles such as "The Mandalorian", soundstages and streamers. But also, different Virtual Production facilities started to spark across Europe and Scandinavia - including Denmark, Finland, Poland, Germany and elsewhere.

Thus, the key focus of the 2022 edition of the PICTURE THIS_ Conference in Copenhagen was to explore if Virtual Production is finally transforming from novelty to reality for smaller country producers such as the Danish film industry.

Furthermore, how could independent creatives, producers, post-production supervisors, and decision-makers harness Virtual Production's power to benefit small film countries and industries?

What could the Danish film industry learn from established virtual production facilities, supervisors and practitioners across the spectrum from facility managers, virtual production supervisors, virtual production equipment and technology providers?

And what new knowledge and takeaways could be acquired from pioneering virtual production case studies?

Finally, the conference set to explore how new groundbreaking technologies such as Web3, Artificial Intelligence and Metaverse are influencing the film industry and what significant technological trends Danish producers and industry professionals should keep in mind.

The 2022 edition of the PICTURE THIS_ Conference was curated by audiovisual and media innovation strategist Sten-Kristian Saluveer (Storytek Innovation & Venture Studio, Marché du Film - Cannes NEXT) and produced in collaboration with the National Film School of Denmark with the support of Nordisk Film Fonden.

Session 1: Pioneering the Future of Virtual Production in the Nordics 

Focused on two pioneering virtual production facilities, which have just opened in Denmark and Finland.

Martin Madsen, a veteran Danish VFX Supervisor with experience in more than 75 productions, presented the case study of designing and developing the Nordisk Shortcut 360-degree VP studio in Copenhagen that was launched only a few days before the conference.

Martin titled his presentation ‘Recreating Reality’ by focusing on how virtual production, especially 360° lighting by virtual production, allows for achieving realism that was previously impossible using green or blue screen technology. Utilising virtual production for lighting thus not only helps to achieve more natural-looking tones but also significantly improves the results in post-production. 

Martin articulated that:

• Lighting, especially sunlight, is a million different light bulbs illuminating an object.

• Getting the lighting right on your actors before you put them on the set makes all the difference.

• In Virtual Production, light dynamically changes on the set, and you'll always have naturalistic lighting on set with the background because it's being lit from everywhere.

• The key objective of this is to show how much one can control the lighting with the screens and how instantaneous it is that you can go from not just one side of the scene to another in a second, but you can change your entire environment. 

• You can do everything in pre-production with VP and the Nordisk 360° stage, with LED screens around the set and the ceiling and the floor, you can do everything in pre-production. You can do even five sequences per day and simplify the work of everyone, from actors’ performances to VFX supervision. A complete spherical 360 volume is especially perfect for reflections.

• The key issues to keep in mind for Martin are understanding the black levels and how to combine game engine technologies with 360° live shot plates.

The second part of the session was presented by Mikko Kodisoja, the founder and CEO of Fireframe Studios, a virtual production company with an ICVFX stage in Helsinki. Mikko is a Finnish entrepreneur and investor who has founded several successful gaming companies, the last being Supercell, the maker of Clash of Clans and Europe's first-ever decacorn. After 25 years in games, he decided to move to the film industry to pursue his childhood dream.

Mikko presented that:

• When he founded Super Cell in 2010, he got a lot of learnings from the previous companies they had created, including what went wrong there, how to build a successful company, and how to build the culture.

• Epic Games invited him to demo the LED volume, and he was blown away by what you can now do with the game engines, and thus he decided to create the Fireframe Studios.

• Fireframe is a group of dreamers and doers who want to redefine how the liner content is made. That means using game engines even more, to create fully flexed stories on the silver screen. That includes engineers, VFX artists, line producers, XR and live shows.

• The Fireframe ecosystem consists of Fireframe studios, Fireframe films, and Fireframe Academy developing the whole VP ecosystem.

• Thus, Fireframe primarily seeks to co-produce or help the production companies to make the reality of the movies shot on the LED volume.

Session 2: The 360 Virtual Production Toolbox

Focused on technologies such as synthetic audio, virtual art department, and AI-generated sets and lighting for enhancing the quality of virtual productions.

Alex Serdiuk, the CEO and co-founder of Respeecher and in charge of the Business Development and Strategy of the company developing Emmy-winning voice cloning technology, joined the conference via a video link from Kyiv, Ukraine. 

Respeecher technology was the first synthetic speech adopted by big Hollywood productions, starting in 2019, including recently the “Obi-Wan Kenobi” series from Disney / Lucasfilm. 

Alex highlighted the potential of synthetic, i.e. AI-generated audio not only for big Hollywood productions but also for independent feature films and documentaries. Alex especially stressed Respeecher’s unique process of focusing on the ethics of synthetic audio but also presented Respeecher voice marketplace that makes generating voices for voiceovers, dubbing and other feature film use cases for low budgets. He also called for more support towards the Ukrainian film industry, entrepreneurs, and innovators.

Watch the session with Alex Serdiuk:

The second presenter was Michael McKenna, the CEO of Final Pixel. On productions, he leads as Director of Virtual Production, overseeing the VAD and OSVP for Film, TV and Advertising clients. With over 15 years of industry experience, including at BBC and EndemolShine, Michael has successfully grown companies producing content for Netflix, Discovery, Channel 4 and many more. 

Michael’s presentation was titled “The rise of the Virtual Art Department: achieving 'Final Pixel' on-set" where he presented the following takeaways:

• Final pixel means working from concept to a state when you come away with what you've shot on set, which becomes what goes into the edit and what comes out in the final edit.

• VP is a problem-solving tool - not just because you think you want to do something virtual production, which sounds fantastic. 

• The key in VP is to create parallax. Thus, we want to create as much parallax as possible to make VP feel like it's a real realistic-looking place.

• A virtual art department (VAD) is a team that builds the 3D environment, and A team that does all the work around the background works closely with a traditional production designer. There's a bit of a handoff, a crossover of the roles of a VAD supervisor and runs a VP supervisor who's there to deliver it to the live-action production team.

• The workflow for the VAD team starts with the script and treatment. It's doing mood boards, reference images, and then storyboarding. Then it usually proceeds environment sketch for every location to be 3D built. This gives a Concept Designer environment example that everyone can get around and agree on.

Watch the session with Michael McKenna:

The final presenter in the session was Kaur Kallas, the CEO of Digital Sputnik. Digital Sputnik is the manufacturer of premium colour tunable LEDs used by Hollywood Studios Productions. Digital Sputnik was the leading lighting technology provider for “The Mandalorian“ TV series and Oscar-winning “Dune” as well as having a long history in collaboration with both independent and major productions, including “Ghost in A Shell”, “True Detective”, “The Batman” and others.

Kaur presented the next generation of Virtual Production set solutions, including AI-generated sets and mobile app-guided lighting systems. Specifically, Kaur pointed out that: 

• Digital Sputnik has wanted to find a way to scale down and make micro-virtual production possible.  

• Everything can be custom-tailored on the big shows, but the custom-tailored approach makes it difficult to scale it down to smaller productions.  

• Digital Sputnik's approach is that instead of creating an image in a post, why not make light on the set and already have an image very close to the final before you finish shooting so everybody can work in a more visual environment.

• On the big sets, the creativity is compromised somewhat because the technicians are running the show because there are so many moving parts, and it takes quite a lot of effort to keep everything going. 

• Digital Sputnik has created a template-based solution; this can help with the environment creation because the main concern is the cost of draining the environment. We can figure out how to make environments reusable. In that case, the lower productions and students and small film crews will be able to access high-quality environments at a fraction of the cost of a day to make high-quality environments. 

• The future solutions from Digital Sputnik will include AI-based virtual set generators that can be synchronized with lighting to make virtual production accessible for a wide range of productions. 

Watch the session with Kaur Kallas:

Session 3: Demystifying LED Virtual Production

Focused on high-end LED studio productions such as the HBO TV series “House of the Dragon” and presenting what types of technical questions need to be answered when setting up a LED technologies-based volume.

Louisa Bremner is a Virtual Production Supervisor who has worked on films and TV series such as “House of the Dragon”, “Matrix 4” and “Death on the Nile“. From her time at DNEG and Lux Machina, she has worked across a full range of virtual production toolsets, from virtual scouting, facial performance, mixed reality (simulcam) and in-camera VFX LED workflows. Over the last year, Louisa has been a Virtual Production Supervisor for Lux Machina, working on some of the biggest Virtual Production stages in the whole of Europe and the world.

Louisa presented the case study from the first seasons of “House of the Dragon” highlighting that:

• VP is increasingly used not to imagine new environments but to replicate natural habitats in a studio environment.

• The considerations are location restraints (such as nature reserves), weather, mobility of the crew to difficult locations, such as on GOT, and the need sometimes to extend the set. 

• In these workflows, the original environment is usually scanned with Lidar technology, cleaned, and turned into a VP asset for production. 

• Even in big studio productions, VP requires careful coordination between creative and technical elements to achieve an excellent-looking outcome, and the involvement of a VP supervisor is an absolute must.

The second presenter in the session was Peter Kirkup from London-based Virtual Production technology manufacturer Disguise, one of the critical providers of VP playback servers and technologies from live venues to studios.

Peter’s talk focused on the theme of “What the Tutorials Do Not Tell You? Experiences From Enabling World Class Virtual Productions”. 

In his talk, Peter pointed out that:

• There are a lot of different crossover terms that mean the same thing but don't - virtual production, LED stages, LED production, ICVFX, and real-time production.

• So, one of the key things we're striving for with virtual production is talent immersion.

• Flexible studio spaces allow for quick reconfiguration to create various setups. This leads to the ability to make impossible environments for shooting. Location shoots are costly and often complex, so this option eliminates the need for travel.

• LED screens were born in stadiums to do scoreboards. LED technology comes together mainly from outside the industry.

• Camera tracking gives us an understanding of where that camera is in the 3D world and real-time data about that camera focus data, iris data, and so on that we can use to compose the film. 

• Disguise system provides servers running into a volume, but it doesn't have to be the only solution; many different technologies could do that.

• The critical questions that producers and studio managers need to remember are: What do we want to put in the space? What vendor, what product, and what pixel pitch? All of that. How much rendering power do we need to build a volume? How are we going to mesh it all together? How are we going to calibrate it? What's the latency going to be in the book? And how do we compensate for that?

• What about disks? How are we going to play out the plates in that volume? What format will they come in? What resolution, what Kodak? How do we train the operator to run this space? How are we going to synchronize it? How are we going to get a colour pipeline that's suitable for post-processing into that space? And what about the lighting?

• Also, What about redundancy? What about camera technology? What cameras work best for large volumes? What lenses work best for large volumes? And finally, What about the remote control? 

• There are a lot of different things that come together to make this work, and it probably is multiple people. So having one or two key people with some experience and knowledge in that house is valuable.

Watch the session with Peter Kirkup:

Session 4: How to Manage Virtual Production Successfully

Examined how established professionals build virtual production workflows between small and large crews, especially as the technology is still in its infancy. Approaches from Hollywood-sized productions to smaller, independent virtual production facilities in Europe were presented.

The first speaker was Kathryn Brillhart, a cinematographer, director and producer who uses volumetric shooting, visualization techniques and visual effects monitoring to enhance projects. Her real-time game engine and virtual production techniques to achieve in-camera visual effects include projects such as Warner Bros.’ “Black Adam” (2022), Netflix's “Rebel Moon” (2023) and Amazon Studios' “Fallout” (2023).

Kathryn Brillhart presented a talk titled "Leaning into Collective Genius" in which she shared a guide for filmmakers on how to maximize production value and increase artistic creativity by working closely with their Virtual Production Team. 

Kathryn's presentation included a series of case studies on virtual production experimentation and highlighted the following topics:

• In just 10 to 15 years, the ability for individuals to produce content using cell phones and real-time 3D engines has become a reality, further democratizing the art form.

• If we look at the history of Virtual Production workflows and tools, they have also evolved from innovative directors who were so confident in their process and the way they had to approach their craft that they pushed the evolution of the technology.

• One of the most important lessons we should have when planning VP projects is that content and directors drive virtual production.

• If VP requires extensive real-time workflows to be integrated into the project, this should be a separate department.

• VP is inherently complex and requires extensive cinematography, visual effects and engineering knowledge.

• The team's skills often need to overlap in ways that are essential to agile workflows and can contradict the teamwork style of the traditional union role.

• The virtual production department requires leadership that understands these nuances, anticipates how the differences can cause friction and can smooth the way for the artists involved, the department heads, and the director.

• A VP supervisor would report directly to the director, but that's an unusual position regarding the history of crew and workflow structures.

The second speaker was an established Polish producer and Virtual Production expert, Maciej Zemojcin, the CEO PixelRace – the virtual production software company, and CTO of ATM Virtual Studio – the first and most advanced VP Studio in Poland. He is currently acting as a script consultant for virtual production technology in storytelling, backed by 20 years of experience in international film production and film technology. 

Maciek first presented a breakdown of the Virtual Production planning process, articulating that it is not a linear process consisting of idea sketching, mood boarding, storyboarding, tests in a studio and refining in a 360° process to achieve a final and good-looking result.

Secondly, he pointed out that a successful project requires dragging your director, your dop, and your production designer into communication as early as possible. 

Additionally, he recommended that: 

• How you go from one scene to another, how it'll go into edit and paper and pencil are the cheapest and the best things.

• This gives you the dimensions, timing, and measurements, and you can plan it on your VP production successfully.

• Good Virtual Production assets are the key, especially locations that cannot be otherwise accessed. Pixelrace has been, for example, developing a digital twin of Auschwitz as, due to restrictions and historical value, the location is unavailable to film shoots. 

• This poses many challenges but develops a unique industry and licensing market for the VP industry in Europe.

Watch the session with Maciej Zemojcin:

Session 5: Learning by Doing

Focused on presenting three case studies of Virtual Production experiences across Europe to give producers and the audience a "hands-on feel" of what it would mean to execute a virtual production.

Mikael Windelin moderated the session. He started his career as a film and television producer over 20 years. He has worked for several production/postproduction companies and his Postproduction facility during this time. Since 2016 Mikael has worked internationally as an independent VFX producer on award-winning films and won a Swedish Academy Award for best visual effects.

Case Study 1 focused on How virtual production helped an independent artistic feature project, "Euphoria" to rapidly change its location from war-torn Ukraine to Germany, replicating Ukraine in Virtual Production.

The case study was presented by Adrian Weber, a Virtual Production Supervisor and TD at HALOSTAGE. He is teaching at the FH Salzburg in a new Virtual Production master’s degree and guiding multiple student productions from different film universities through the preparation process and a shoot at the LED stage in Babelsberg. At Halostage, he’s also a presenter at regular and free of charge “Hands-On” workshops, to give industry professionals the chance to experience ICVFX hands-on without the pressure of production.

Watch the session with Adrian Weber:

Case Study 2 explicitly focused on the potential of Virtual Production for the Production service sector and how it was possible to achieve a globally recognized commercial for the "Assasins' Creed" game and film franchise only in two days. It was highlighted that shooting raw tests and planning interactive storyboards for critical to achieving good results.

The case study was presented by Ilya Medovyy, a Producer/Partner at Nafta Films – the largest production company in the Baltic region, working across Estonia, Latvia and Lithuania, and Europe. He has been focusing on doing service for large international clients, and in recent years has taken a particular interest in LED Volume stage productions. Ilya has produced and co-produced several feature films, including a Giornate Degli Autori nominated “Conference”.

Watch the session with Ilya Medovyy:

Case Study 3 presented a graduate film from the National Film School of Denmark, which utilized VP as its central technology to bring a unique story between life and death onto a screen. VP was used throughout the production from inception to delivery and how to apply the tech on a shoestring budget was explored in the case study.

The case study was presented by Sofia Due Rosenzweig who is a director studying at The National Film School of Denmark. She has directed several short films and has worked as an assistant for director/DoP Ellen Kuras (ASC) and writer/director Steven Zaillian on his HBO miniseries “The Night of.”   

At the discussion Sofia was supported by her film crew including screenwriter and VP supervisor as well as producer Kristian Pugholm who is an emerging producer from The National Film School of Denmark, graduating in summer 2023 with his graduation short “I Ghosted You” (working title). At the film school, Kristian has continuously introduced VFX and animation into his productions, and more recently, VP production and LED screens.

Watch the session:

Session 6: Closing Fireside Chat: "What's Next? Real-Time Technologies, Metaverse and its Near Future Impact on Film and Entertainment Sector

At the concluding fireside chat, "What's Next? Real-Time Technologies, Metaverse and its Near Future Impact on Film and Entertainment Sector" Sofie Hvitved was a guest. She is a futurist and head of the media department at the Copenhagen Institute for Futures Studies, where she is currently working on the future of the metaverse and how it will affect our lives and businesses. She works extensively at the intersection of media, technology and entertainment and has a solid strategic background in the media industry, including at DR (Danish Broadcasting Corporation).

The discussion was moderated by Sten Saluveer and involved the audience interactively.

The main points Sofie articulated were the following:

• The CPH Institute of Futures Studies puts us five to 10 years into the future to analyze and predict what will happen then.

• No one yet knows what the Metaverse will look like. Still, developments toward the next generation of Internet (Web3) infrastructure will create a series of "metaverses" that will profoundly affect all aspects of human life.

• Artificial Intelligence (AI) will radically impact the creation, production, and distribution of content using synthetic media, making working with AI, i.e., promoting, a highly relevant skill.

• NFTs are becoming a viable business model used by major studios and brands like “Rick and Morty” writers to engage audiences.

• For the film industry, here are three recommendations: Explore a Web3 digital space, create or buy an NFT to understand the economics, and play with Artificial Intelligence tools to know how they will impact workflow and working methods.

Watch the session with Sofie Hvitved:

The fireside chat was followed by a short presentation by Henrik Schönau-Fog of the ViZARTZ program: “Where Filmmakers meet Techies" which takes place at Aalborg University. Henrik presented “behind the scenes” of the first edition of the program and invited speakers to join the second “interactive tech meets film lab”.

Watch the session:

The curator and moderator then closed the conference summarizing key takeaways and findings, and Sten-Kristian Saluveer welcomed guests to the 2023 edition the next year.

All rights reserved to the original copyright holders.