Runway Act One and the Dream of Seamless Storytelling

What Act One Brings to the Table

Act One opened up a whole new world for GenAI filmmakers like myself. It gave us the power to move beyond random outputs and into the realm of dynamic, expressive animation with full control. Suddenly, characters could tilt their heads, raise an eyebrow, or smirk—all without fumbling through endless text prompts. The result? Subtle, lifelike emotions that breathe life into digital creations.

Take my short film Price of Nice as an example. The story thrives on its characters’ sarcastic and exaggerated expressions, which Act One made possible. Renee, the star of the film, delivers biting comebacks and witty one-liners with a range of nuanced facial expressions that amplify her dry humour. This wouldn’t have been achievable without tools like Act One allowing us to craft moments of emotional clarity and relatability in our characters.

The Next Frontier: What’s Missing

But filmmaking isn’t just about control over facial expressions. To tell a story, you need movement, gesture, and scene-wide movement. This is where current tools, including Act One, still fall short.

Here’s what we’re dreaming of for the next phase of AI filmmaking:

🎥 Full-Body Acting: Imagine characters whose arms, gestures, and full-body movements match the same precision and fluidity as facial controls. Renee’s exasperated sighs and eye rolls in Price of Nice would feel even more alive if she could lean into the frustration with matching body language—crossing her arms or shrugging her shoulders.

🌍 Scene-Wide Dynamics: The ability to retexture an entire video while preserving seamless camera motion and character actions. It’s one thing to move a character with motion capture like a puppet; it’s another to make an entire animated world feel immersive. Imagine if Price of Nice could incorporate dynamic camera movements with controllable full body character movements so Renee could walk to the micro-kitchen to grab a coffee as she's on a call.

The Power of Video-to-Video

Runway’s current Video-to-Video tool is already impressive. By using a driving video and a text prompt, it transforms footage into stylised animations with surprising quality. But what if we could take this further?

Picture this: Instead of relying solely on text prompts, filmmakers could upload a keyframe image alongside a driving video. The output? A seamless animation that blends the style and aesthetic of the keyframe with the natural motion of the driving video.

This concept essentially brings the functionality of full-body motion capture without the need for expensive equipment or the complex workflows of 3D modelling and rendering software. Traditionally, creating animated characters with lifelike motion involves intricate rigs, expensive cameras, and time-consuming post-production processes in software like Maya or Blender.

With GenAI, this process could become seamless, easy, and fast. Filmmakers could shoot a simple reference video on a phone and turn it into a polished, stylised animation in minutes—all while preserving the original style and feel of the project. This democratises animation and brings high-quality storytelling tools to creators of all skill levels.

For Price of Nice, this would mean Renee’s sarcastic head tilts and exaggerated expressions could be paired with equally expressive hand gestures or body language, enhancing the humour and making her character even more relatable. The potential here is limitless.

Act Two: What’s Next?

We’re on the cusp of something extraordinary. With tools like Act One leading the charge, AI filmmaking is rapidly transforming into a space where storytellers can bring their visions to life without the constraints of traditional animation. But to truly revolutionise filmmaking, we need these next steps: full-body performance, scene-wide retexturing, and a more integrated approach to blending style and motion.

In Price of Nice, the humour shines through because the story, dialogue, and characters take centre stage. With advancements like those imagined for Act Two, we could push this even further—allowing AI creators to focus solely on storytelling while the tools handle the complexity of animation.

So, what are you wishing for in Act Two? Let’s keep dreaming big. 🎥✨