Runway Has Improved the Quality of Its Image-to-Video Generation Tech
In an update this week, Runway says its image-to-video artificial intelligence (AI) technology has been significantly improved, namely in quality and in the smoothness of outputs.
Runway, which describes itself as an AI research company, has created multiple tools that aim to either create images and video from nothing but text descriptions or enhance still photos by giving them motion. In March, it launched the first publicly-available text-to-video generator, a model it had been working on since the previous September.
It followed that up in April when it added support for 15-second videos generated by AI into its iOS app, which it billed as a way to let anyone turn videos originally shot on iPhone into anything they could imagine.
“Runway Research is at the forefront of these developments and we ensure that the future of content creation is both accessible, controllable and empowering for users. We believe that deep learning techniques applied to audiovisual content will forever change art, creativity, and design tools,” the company said at the time.
Runway’s latest isn’t a new feature, but is instead a refinement of an existing one. In an update launched yesterday, the company sayd that it has improved video generation that use images an input — that means no text required — to offer “significantly better quality, control, and latency, while providing much smoother output videos.”
The company, which does most of its marketing through Twitter (now known as X) and shared a few sample outputs that were generated using the updated image-to-video mode.
Generated greenery. Made with the new Image to Video mode in Gen-2. pic.twitter.com/Souui3kk87
— Runway (@runwayml) July 26, 2023
Photo experiments with the new Image to Video mode of Gen-2. pic.twitter.com/ZYebBRQnxB
— Runway (@runwayml) July 25, 2023
One of the problems that has plagued AI-generated video — including Runway’s early attempts — is a “creepy crawly” look to footage, since generators appear to have a problem understanding what part of video needs to remain constant and what part can move. The result looks like every piece of a video is somehow in motion, which is incredibly off-putting.
While that problem has not been entirely eliminated in Runway’s latest update, it has been toned down significantly. Several of the example outputs look very clean at first glance, although some scenes still exhibit the strange wobble to footage that comes as a result of the AI not properly understanding stationary objects. Still, this much advancement in just a few short months is nothing short of impressive.
As part of the update, Runway now offers an “unlimited” tier in its pricing plans, which costs $76 per user per month and allows for the creation of unlimited video generations.
