SUMMARY_23

By Sten-Kristian Saluveer

The 2023 edition of the Nordisk Film Fund signature project, PICTURE THIS_23: 'Change of Everything', which took place on the 4th of October at the National Film School of Denmark, brought together 18 world-leading experts working in film powered by novel technology. Those included extraordinary speakers from some of the world’s most known companies, including Disney Imagineering to Apple, the creators of the hottest AI Academy for Filmmakers – the Curious Refuge, as well as film directors, monetization, and virtual production experts. Through a range of sessions PICTURE THIS_ focused on how AI and next-generation technology are transforming the industry and fostering creativity throughout the independent film life cycle - financing, packaging, screenwriting, production, and post-production.

The conference was curated by Sten-Kristian Saluveer, founder and CEO of Storytek Innovation & Venture Studio, and Head of Cannes Next - Marche Du Film, Festival de Cannes over the period of March to October 2023.

The world in 2023 as we know it has experienced an unexpected tornado - the explosion AI has burst in from doors and windows and began to challenge the tradition of how we conceive, produce and distribute creative content. AI tools, unimaginable a few years back, are used to write and enhance scripts, creatives are prompting Midjourney for photorealistic characters and film sets, game engines are becoming studio locations, and top production quality films are made fully with using open-source tools.

“AI” has been selected as the word of the year by Time magazine, three AI-generated films launching at the Venice Film Festival, and the first large-scale AI film festivals taking place in San Francisco and New York.

This never-before-seen transformation has made it clear - creativity in the audiovisual and film industry has taken a 360-degree turn, and the future winners will be those who master the new AI tools and technologies, whereas the others will inevitably jog behind.

Nowhere is this transformation more apparent than in independent filmmaking, where innovation from storytelling to post-production is crucial to remain competitive and relevant.

The 2023 sessions included:

• Opening Keynote: Exploring the Intersection of History, Technology, and Multi-Platform Storytelling

• From Algorithms to Artistry: AI as the Next Frontier of Filmmaking. From Screenwriting to Directing, and Post-Production

• There is an AI Lurking Behind the Scenes: Showcase of Kaspar.Ai – AI Powered Editing Assistant for Feature Film Editing

• Making Generative AI Tools Work for Independent Production

• Mastering Virtual Production and Real-Time Filmmaking for Independent Producers

• Reinventing Film Financing: Journey from Tokenized Funds to Achieving Success through Transparency

• From Feature Films to Virtual Production and Immersive Storytelling

• The Journey of Creating an Interactive Documentary Murals, inspired by the works of Banksy in Ukraine

• Case Studies of (Independent) Productions: Virtual Production and Innovative Tools and Pipelines

Summary of the key topics and sessions

Opening Keynote: Exploring The Intersection Of History, Technology, And Multi-Platform Storytelling

Amy Jupiter, a globally acclaimed storytelling and audiovisual pioneer, a former Disney Imagineer, an Experience designer at Virgin Galatic, a VFX supervisor, and a Creative Producer, delivered the opening keynote at Picture This 2023. She explored the magic of storytelling, inviting the audience on a journey where history, technology, and multi-platform storytelling converge. Jupiter emphasized the power of human thought and imagination to create immersive experiences in various mediums, not just limited to technology. She drew parallels to her experience in theme parks with examples from major productions such as Avatar, where storytelling weaves a tapestry of immersive experiences.

Jupiter showcased to the audience several design thinking tools on how to design narratives and experiences, highlighted the significance of 'places of respite' within building stories, whether in film or theme parks, and recommended that these areas are crucial for visitors to process their experiences and create lasting memories.  She underlined the necessity of emotional spaces in productions, stating that such design considerations are vital in ensuring that they cater to the psychological needs of the guests as much as to their entertainment.

Jupiter specifically connected with the audience by referencing the widespread familiarity and experiences of Disney theme parks. She emphasized the importance of nurturing creativity and innovation by stepping 'outside the box' through Design Thinking and Quantum Ideation.

With three films generated by AI, launched at the recent Venice Film Festival to a muted welcome, a full-blown AI festival taking place almost simultaneously with PICTURE THIS_, and full-blown yet cost-effective generative AI tools sparking like mushrooms after the rain, it is clear that AI applications are here to stay whether one likes it or no.

The session brought  together the world’s leading thinkers and practitioners, presenting how they use AI in screen- and filmmaking, offering insights into the real-world applications and implications of this emerging technology.

From Algorithms to Artistry: AI As The Next Frontier Of Filmmaking From Screenwriting To Directing, And Post-Production

Peter Hirshberg, an industry veteran with over 30 years of experience in emerging media and technology companies, delivered a presentation titled "AI in the City" to discuss the impact of AI on writers and screenwriters. He emphasized the potential of AI hallucination (i.e., AI making outputs and mistakes not intended)  as a tool for writers, directors, and producers to generate new ideas and push creative boundaries.

Peter started his presentation by recounting his experience of running a writer's room during the Hollywood writer's strike in France with the curator Sten-Kristian Saluveer, where they used various AI tools to rewrite and explore French history for documentary film concepts. This experience led them to explore the creative potential of AI, which eventually led to the presentation at PICTURE THIS_.

Peter started with the explosive exponential growth of AI in the film industry in the past year.

Peter pointed out that AI's artistic implications are not limited to just using it but extend to its potential for creatives, especially for writers. He cited Sequoia's generative AI report, which predicted that AI would drive the creative industries in the future. He also discussed the impact of AI on the film and gaming industry, citing examples of how AI has been used to create mockups, final drafts, and even movies.

Peter then showcased various examples of how AI hallucination can be used as a brainstorming tool for writers, directors, and producers in the creative industry. He presented a fully AI-generated TV episode of South Park as well as clips that were created using only AI tools for Picture This, including a fully AI-delivered clip from Seinfeld. Peter emphasized that AI hallucination can help creators explore new storylines and characters that they may not have considered before. By leveraging large language models, the technology can push creative boundaries and help creators generate new ideas.

Finally, Peter emphasized the importance of AI in exploring the spiritual frontier of what we create and how it can be used to understand the human condition. He concluded by stating that AI has the potential to create a new industry and lead to a wondrous year of discovery.

Watch the session with Peter Hirshberg:

Curious Refuge is an online platform founded by Caleb and Shelby Ward that is pioneering the use of AI in film. The Wards are dedicated to democratizing the creative process and unlocking everyone's inner storyteller. They have created the world's first online course that teaches how to use AI to create films. The training covers various aspects of the production process, from prompt engineering to animation and movement.

Curious Refuge was founded after the Wards saw a Harry Potter Balenciaga video that was created with AI. They were inspired by the video and wondered what other uses they could find for these tools to create compelling films. Caleb put together the first version of one of these trailers, the West Anderson Star Wars trailer, and it went viral, gaining over 3 million views on Youtube.

The Wards received emails from people looking to learn these tools, and since they have an extensive background in online education, high-end VFX training, and growing education communities, they created the world's first online home for AI filmmakers.

At Curious Refuge, they create AI-driven film content every week where they talk about AI film news, specifically AI tools, as they relate to filmmakers. They also curate an art gallery  and newsletter where they find different AI pieces around the web, curate that, and have a page for people to come and find references.

According to Caleb, "If you go to the press, you're going to see a lot of conflicting information about what artificial intelligence means. But for us at Curious Refuge, we really feel like we want to push into optimism and view this as a creative opportunity for people to tell stories in a new way." They believe that within 12 months, we will see a point where AI will create narrative-length photorealistic content, and learning AI tools will help filmmakers tell stories in new and innovative ways, and they are excited to see where AI will take the industry in the future.

Watch the session with Caleb and Shelby Ward:

 Nick Laurant, Director, Producer, and Showrunner of the Corridor Crew YouTube Channel, delivered a keynote speech on the use of emerging technologies in creating cinematic narrative stories. Laurant highlighted the importance of eliminating barriers between storytellers and their audience, emphasizing the role of technology in achieving this goal.

For the past decade, Corridor Crew, a small Los Angeles-based production company, has been using a mix of technology, filmmaking, and emerging technologies such as AI to produce videos that aim to inspire and educate the next generation of filmmakers, VFX artists, and creators. According to Laurant, the company's ultimate goal is to tell a good story, regardless of the medium.

Laurant also emphasized the need for storytellers to embrace emerging tools, such as AI, to experiment with new and innovative ways of storytelling. As Nick noted, "Technology can help eliminate barriers between the stories we want to tell." He shared several examples from the short films and VFX projects but also an example of one of Corridor Crew's recent projects, Anime Rock, Paper, Scissors, which utilized AI to create a short film about two brothers battling over the throne in their kingdom. Nick did a full breakdown of the process of how Picture This participants can use AI tools for their productions, but also highlighted that the filmmakers need to be aware of ethics when putting the tools to the use, especially when using datasets from films created by other authors.

Watch the session with Nick Laurant:

There Is An AI Lurking Behind The Scenes: Showcase Of Kaspar.Ai - Ai Powered Editing Assistant For Feature Film Editing

Mads Damsbo, co-founder of Kaspar AI, presented a keynote on the intersection of AI and film editing. Kaspar is a deep tech company that aims to equip audiovisual storytellers with unprecedented creative powers by creating AI-powered tools for film editing. In his talk, Mads introduced K1, a Premiere Pro extension tool that uses custom-trained AI to revolutionize the process of film editing. The tool makes use of large language models that understand the semantic meaning of words, phrases, and even concepts, rather than just looking for exact matches. It helps filmmakers search for footage based on the meaning of words or phrases, rather than just the exact words themselves.

One of the key takeaways from Mads' talk was that AI has the potential to help filmmakers edit automatically, especially given the vast amounts of footage that are being shot on professional film sets. Kaspar’s solution is cost-effective and can be installed locally on a film editor’s computer, helping to maintain the copyright of assets, but significantly improving the feature film editing experience. Specifically, Kaspar's AI can help filmmakers find footage that matches their creative vision more efficiently and accurately.

Mads highlighted that “we really believe from the experience we've had that there is a true intersection between humans and machines where the co-creation, the creativity we've heard about all morning, can exist and that we can have these insane powers using AI. He also added that, "The next step in our mind is the theory of montage. Between any moving images in any order, and people sitting in the cinema will know this by film school hat, there is a subtext and a story to be told between the arrangement of moving images. This is something that we are very interested in. How does AI understand this language, the language of cinema?"

During an exclusive hands-on workshop,  Peter Hirschberg, and Shelby and Caleb Ward, demonstrated the power of AI in the creative process for filmmakers and storytellers. With a full house of 50 participants, attendees had the opportunity to learn how to use AI tools to bring their creative ideas to life, from the earliest stages of conception to post-production.

The workshop covered a range of tools and techniques, from screenwriting to post, as well as looking at different strategies for prompting. During the workshop, Peter, Caleb, and Shelby collaborated with participants to develop a short video concept, from screenwriting to visuals. They also answered questions and discussed professional quality features and opportunities of different AI tools in professional workflows. The key takeaway from the session was that current tools are already well developed for proofs of concepts, but it would take another 6-12 months for the workflows to achieve professional feature-grade fidelity.

Watch the session with Mads Damsbo:

Mastering Virtual Production And Real-Time Filmmaking For Independent Producers

Stacia Pfeiffer, Co-founder of Chaos Inc., hosted a breakout session on virtual production (VP) that covered the entire process from technical planning to budgeting. She shared insights into the practical applications of VP using case studies, including Apple's "The Hijack" with Idris Elba. Stacia also answered questions from participants, focusing on different workflow and planning processes for VP  based on her wide experience with top VFX companies such as Lux Machina and Dimension/DNEG.

Reinventing Film Financing: Journey From Tokennized Funds To Achieving Success Through Transparency

Miguel Faus, a screenwriter, director, and producer from Barcelona, has made history by funding his debut feature film, Calladita, through NFTsas despite of the success of his short film, which even sold to HBO, he was unable to raise financing from public funds which were often nontransparent about their decisions.

The film, whose trailer was shown as an exclusive preview to the participants, is the first European movie ever to be funded through NFTs and will be premiering at the 2023 edition of the Tallinn Black Nights Film Festival in the First Features competition in November.

Faus explained the financing process to the participants, showing how created still frames from the short film Calladita and sold them as NFTs on the Ethereum blockchain. The campaign was similar to Kickstarter, where people could contribute different amounts to fund the film and receive different rewards.

The campaign raised over 750k euros, with 1,358 NFTs sold. The average contribution was around 1,200 euros. Faus explained that integrating emerging technologies, such as NFTs, into the financing of film projects is doable, although issues of taxation and understanding the technological components are necessary, "This wasn't happening on anything like Kickstarter or any platform like that because nothing like that existed at the time," said Faus, pointing out that besides funds, the model allows a direct relationship between the filmmakers and the audience.

Watch the session with Miguel Faus:

Maria Tanjala, the co-founder of FilmChain, discussed the need to modernize royalty payments in the film industry. She spoke about the challenges faced by filmmakers and how FilmChain's platform is transforming the industry by bringing transparency and modernizing payments, arguing that “The way everyone in the value chain accesses their money and data is absolutely fractured."

Maria emphasized that The film industry's royalty payment system is slow, opaque, manual, and reactive, leading to frustrations and delays for producers, sales agents, and financiers. FilmChain's platform brings transparency and modernizes payments by providing a digital collection agent for independents and a royalties management solution for content distributors.

Up to 30% of revenue is lost due to human error, mismatched funds, and unreported revenue, as a result of overworked accountants using manual spreadsheets. The lack of infrastructure and technology is breaking the trust across the entire industry, and there is a need to embrace new solutions to make the industry more sustainable. Maria also showcased different opportunities for how Filmchain’s proprietary rights management system can deliver automated royalty collections while providing instant payout opportunities for everyone included in the film’s payment waterfall, from producers to sales and creative positions.

The Journey of Creating an Interactive Documentary Murals, Inspired by the works of Banksy in Ukraine

MURALS is an immersive documentary experience about war in Ukraine that seamlessly blends traditional filmmaking and state-of-the-art scanning and virtual production technology. This captivating and inspiring project is the result of a collaborative effort among global talents, including the Grammy-winning studio Aggressive, 3D artist Artem Ivanenko, virtual production experts PixelRace, and the Polish production house ATM Virtual. MURALS made its debut at Cannes Next 2023 and won after Picture This, a prestigious audience award at the BFI London Film Festival’s Expanded section,

The session pointed out that a Polish-Ukrainian-Spanish-US co-production, MURALS, is an expression of solidarity with the Ukrainian nation, and the team wanted to reach a wider audience with the project. Thus, they created a cinematic and VR version of the documentary to make it accessible to more people.

Alex Topaller and Dan Shapiro, co-founders and creative directors of Aggressive, talked about their background in music videos and commercials and how they combine various techniques such as live action, visual effects, animation, and CG. They use experiential technology, including virtual production, motion control, and 3D scanning, to create their work. In the case of MURALS the directors specifically stayed away in altering the original material captured in the war zone using volumetric techniques to highlight the real nature of conflict and suffering.

Radoslawa Bardes of ATM Virtual and Maciej Zemocjin of PixelRace presented the technical aspects of MURALS, including the integration of 3D scanning and virtual production. She pointed out that “MURALS is an immersive documentary experience about the war in Ukraine. It's a very unique example of how we can combine technology, art, and speak about a very difficult subject at the same time”. They emphasized how technology can be adeptly harnessed to enhance storytelling even when the producers come from narrative film and not necessarily from immersive technologies, also touching on their recent collaboration with Agniezka Hollands’ “The Green Border”, which premiered at the Venice Film Festival.

The key takeaways from MURALs include the importance of collaboration among global talents, the ability to seamlessly blend traditional filmmaking and state-of-the-art technology, and the power of immersive storytelling to reach wider audiences.

Watch the session with Alex Topaller, Dan Shapiro, Radoslawa Bardes and Maciej Zemocjin:

Case Studies Of (Independent) Productions: Virtual Production And Innovative Tools And Pipelines:

PICTURE THIS_  closed with a tradition, a case studies session moderated by veteran VFX supervisor Mikael Windelin. The 2023 edition panelists included Nick Laurant of Corridor Digital, Jennifer McSpadden of Nant Studios, Ian Bach of Hitch, and film director Morten BH. Each panelist presented case studies from various regions from the US to Denmark, offering the attendees a global perspective on the application of VP technologies in production in 2023.

Nick Laurant highlighted the opportunities that emerging technologies and AI present to independent filmmakers. He emphasized the role of these tools in bridging the gap between vision and execution, saying, "These AI tools can be a tool I can use to fill in gaps in my ability to tell a story." He also discussed how AI could enhance accessibility and creative freedom for filmmakers by showing how directors can use AI from storyboarding, concept art and concept design with available open source tools.

Jennifer McSpadden, from one of the world’s top VP Studios., Nant, shared insights into the latest workflows of LED wall productions and the application of AI and MidJourney in creating dynamic 2D and 3D content for live productions through 3 case studies.
In her technical presentation, she discussed the practical challenges and advantages associated with generative content and real-time workflows, saying that  "The power of AI and real-time workflows is that you can make changes on the fly and not have to go back and redo things."

Finally, Ian Bach and Morten BH discussed the Danish experiences in using VP for the new Advent calendar series Valdes Jul. They recommended the importance of taking a practical approach to using these technologies in film and television production and discussed how VP can also be utilized on smaller Scandinavian budgets. For the director, the key learning was not to fear the new environment and the availability to put adolescent actors into a physical environment through VP, which contributed greatly to the working speed and better acting performance.

The key takeaways of the session were:

• AI tools can help fill in gaps in budget and ability to tell a story for independent filmmakers. - AI and real-time workflows based on Midjourney can be used to make changes on the fly in LED wall productions.

• Using AI tools can enhance accessibility and creative freedom for filmmakers, especially in pre production, even using open source tools such as Stable Diffusion. Directors and set designers should learn basic skills of operating Stable Diffusion to train their models.

• Virtual production can be used to overcome creative and budgetary challenges for Scandinavian film, but producers and creatives need to take a practical and pragmatic approach to using these technologies in local film and television production.

The conference closed with Sten Saluveer and the audience thanking the participants for an extraordinary conference and welcoming them back to the 2024 edition.

All rights reserved to the original copyright holders.