A photorealistic image of a man mid-jump where the back half of his body is rendered in live-action footage style, capturing lifelike textures, shadow.

1: Why Live Action and AI for an animation workflow?

For my Master's in Computer Animation thesis, I have opted to do a short film on a story I wrote a few years back. While I am really excited to work on this project, I'm also very new to animation and my script is running over three minutes, so there is a daunting task ahead. 

With my background in live-action filmmaking, I wonder what could integrating live-action footage and generative AI video-to-video processing could do for my pipeline. This could make the animation process smoother, open up creative possibilities that traditional methods would not permit with my current time, budget and technical limits and make a long project like mine, a more attainable one. Maybe.

So, in this blog series, I won't dive into the short film itself just yet. Instead, I'll play around with live-action footage and see what current AI and machine learning tools can do.

Here's What I'm Aiming For:

  • Refine the Workflow: Figure out the best way to produce the animated content I need to tell my story.
  • Explore Creative Options: Experiment with different artistic styles to find a look that fits. Explore the possibilities to work with that material and integrate 3D assets.
  • Assess Quality: At the end of the day, the main objective is producing a top-quality short film; the pipeline should only integrate AI if it makes it better.
  • Assess Feasibility: See if this approach is practical, especially since I'm working solo on this.
  • Contribute to the Field: Share my journey and findings to help other filmmakers interested in mixing AI and 3D animation into their projects.

What's Next

Over the next few weeks, I'll be testing out different techniques and tweaking the workflow. This means playing with various footage and AI models, seeing what works (and what doesn't), and adjusting things to get the best results without wasting too much time or resources.

So stay tuned! In upcoming posts, I'll share the exact tools I'm using, lay out my game plan, and keep a close update on what I discover. If things go well, I'll dive into planning the actual short film!

Continue to the next article:

2: Exploring the Tools for my AI-Assisted Production Pipeline
Back to blog

References