Runway AI announced the latest version of its video generator called Gen-2. Unlike Gen-1, it does not require sample images and videos. Instead, it is a text-to-video tool that produces motion pictures based on short descriptions. As a result, it could revolutionize how we produce videos and movies.
Runway Gen-2 also adds numerous new features to the video generator. For example, it can put a background and other objects that blend with your original image. More importantly, it lets you train custom AI video generation models for specific projects. Consequently, you could smoothly incorporate Runway AI for work and other purposes.
This article will discuss the announced features of Gen-2. Later, I will cover other text-to-video and similar AI programs. Note that it was not publicly available at the time of writing. Join the Runway AI Discord channel to know when it launches the latest version.
What are the features of Runway AI Gen-2?
Photo Credit: research.runwayml.com
- Text-to-video: Create videos by providing text descriptions.
- Text + Image to video: Generate videos based on an image and a text description.
- Image-to-video: Produce a video using only a sample picture.
- Stylization: Use a sample photo to inform how Runway AI will render your clip. For example, the demo video showed how it changed a clip of a man dancing. The program referred to a picture of a flaming monster so the dancing man would look like it.
- Storyboard: Create stylized, moving renders of mockups. For example, the Runway demo showed how it turned an overhead view of cereal boxes into a sweeping cityscape.
- Mask: Isolate subjects to ensure Runway AI edits them and nothing else. For example, the masking feature allowed a user to change a dog’s fur without affecting the rest of the video.
- Render: Turn untextured 3D renders into detailed outputs. For example, it can turn a crude 3D model of a diver into a near-realistic clip.
- Customization: You may provide more samples and detailed explanations for better results.
What are the limitations of AI video generators?
Photo Credit: digitaltrends.com
The apparent problem with AI video generators like Runway is that they do not produce realistic images. Check any Runway AI samples; you will see videos constantly shifting.
It seems their skin keeps “vibrating” as the video plays. The janky aesthetic is likely due to the AI video generator rendering each second of the clip.
Putting these AI renders together seem to produce unnatural textures in AI videos. The same is true for Meta’s text-to-video generator.
It announced the Make-A-Video tool in September 2022 but hasn’t publicly released the program. Meta’s samples show it can produce a “superhero dog flying” or “horses galloping.”
However, Meta and Runway AI outputs look too rudimentary for real-world use. Some look too basic, while most seem awkward and janky.
Soon, these text-to-video tools may produce better results after further research. Once that happens, they will change how we create videos.
In the meantime, you may try the new AI features of popular video editing software. For example, Adobe launched text-based editing for Premiere Pro.
You may also like: How To Use Midjourney’s Image-To-Text Feature
It automatically transcribes dialogue in videos into editable text and adds it to your Timeline. As a result, you can easily look for a specific portion of your video by searching specific keywords.
You may also copy and paste phrases in the sequence transcript, and Adobe AI will change your Timeline accordingly. As a result, you can get a rough cut and produce captions quickly.
Moreover, Adobe adds automatic tone mapping to help you work with different media and color spaces without custom color settings. It turns color gamuts from log formats to display standard sequences properly.
Conclusion
Runway AI announced its latest version, Gen-2, which adds new features. These include turning crude mockups into stylized renders and expanding customization options.
The AI firm hasn’t specified when it will release Gen-2 at the time of writing. However, you may join the Runway AI Discord at discord.com/invite/runwayml to know when it launches the new tool.
If you want the latest digital trends, follow Inquirer Tech. Moreover, it offers tips on how to use artificial intelligence, augmented reality, robots, and more.