This AI Model Creates Video in Real Time

Runway has demonstrated a new AI video technology in which HD imagery is created in real time. While the technology could be an exciting development in the world of entertainment, others say it will make it even harder to spot fakes.

AI, along with misinformation and bad actors, is already undermining what people see online. But AI video in real time is an even bigger step as the technology could be used to scam people in the real world as someone controlling an AI avatar reacts and responds to a real person.

Runway showed off the technology at NVIDIA GTC last week. The as-yet-unnamed model can generate an image from a prompt in a literal blink of an eye — just 100 milliseconds. In case you’ve never used an AI video model before, it usually takes seconds, if not minutes, from entering a prompt to receiving an AI video on models by Google or OpenAI.

Interesting Engineering reports that the model runs on NVIDIA’s next-generation Vera Rubin architecture. The tech website says the model works closer to how a game engine renders graphics and is able to stream frames continuously.

It means that, in theory anyway, there could be live broadcasts and streams featuring AI characters or worlds that respond to whatever is happening.

It also has implications for gaming. In January, Google announced Project Genie, a world-building tool in which users create their own environments from a single photo or a text prompt that expands in real time.

Interesting Engineering notes that the technology still faces limitations: maintaining character consistency, for example, is a challenge as AI tends to wander from the source the more it iterates. But with companies like NVIDIA and its endless cash reserves behind it, big breakthroughs will keep coming.

Image credits: Runway