Meta Says Its New Movie Gen AI Is an Industry First — But a Demo Shows It Isn’t Perfect

By temporary On Oct 5, 2024 1 0

Meta previewed new AI tools on Friday called Movie Gen that can create videos, edit them automatically, and layer on AI-generated sound for a cohesive video clip.

Movie Gen works with written text prompts, an image, or an existing video as input. There’s also an option to add a personal picture so users can see themselves in the video.

After the AI works its magic and generates a video, a user can type in a text prompt to create a custom audio soundtrack to play with the video.

^{A peek into a video Meta Movie Gen created from an image. Credit: Meta}

While Meta says Movie Gen’s high-definition videos are “the first of its kind in the industry,” when creating long videos at different aspect ratios, it doesn’t mean the AI is perfect. Right now the AI can only generate videos that last up to 16 seconds — and it doesn’t always get the assignment right.

In a demonstration to the New York Times, Meta’s AI tool made a mistake. Though it was able to create a video of a dog in a park talking into a phone, the AI messed up by placing a human hand around the phone instead of a dog’s paw.

? Today we’re premiering Meta Movie Gen: the most advanced media foundation models to-date.

Developed by AI research teams at Meta, Movie Gen delivers state-of-the-art results across a range of capabilities. We’re excited for the potential of this line of research to usher in… pic.twitter.com/NDOnyKOOyq

— AI at Meta (@AIatMeta) October 4, 2024

Chris Cox, chief product officer at Meta, stated in a Threads post that Movie Gen is “industry-leading” in video quality but that Meta isn’t prepared to release the tools because they’re too expensive and the videos currently take too long to generate.

Meta is sharing what it has right now because the outputs “are getting quite impressive,” Cox wrote.

Meta isn’t the first to show off a text-to-video AI generator tool — ChatGPT-maker OpenAI did it in February with its text-to-video model Sora.

In July, OpenAI published multiple YouTube videos in partnership with artists and entrepreneurs showing how Sora could create fantastical short films.