An example from Movie Gen by Meta which allows for video and image prompts as well as text ones. See video below. Meta has previewed its AI video generator called Movie Gen by showing some impressive-looking examples, but there’s no word on a release date. The AI video space is hotting up while simultaneously slowing down, it has now been seven months since OpenAI announced the much-hyped Sora, and yet we are still none the wiser as to when it will be released. Small AI video startups have been filling the gap releasing interesting models but it is still up in the air as to who will be the most successful player in the burgeoning AI video market. Part of the problem is AI video models are simply not good products yet. Users have to wait a long time for videos to appear after typing in a prompt, something acknowledged by Meta’s chief product officer Chris Cox. “We aren’t ready to release this as a product anytime soon — it’s still expensive and generation time is too long — but we wanted to share where we are since the results are getting quite impressive,” Cox writes on Threads. Nevertheless, Cox says the model is “industry-leading”. Videos on Movie Gen can be generated via a text prompt but also with a still image or moving footage.
The model also has the capability to match AI-generated audio with AI imagery and videos can be made in different aspect ratios — features that already exist on competitors’ models. One example Meta gave on the Movie Gen page is footage of a runner in the desert. It was tweaked so he had blue pom poms in one version; so that he was running through a cactus desert in another; and in another version his running clothes were replaced with an inflatable dinosaur costume. What About the Training Data? Meta, like all AI companies, was vague about what data Movie Gen was trained on. Saying only it was “a combination of licensed and publicly available datasets” in a blog post. In July it was revealed that AI video startup Runway scraped hundreds of YouTube video to make its latest Gen-3 model. It is believed that Meta has an advantage over the competition thanks to its insistence that it has the right to use the data created by its millions of social media users for the purposes of AI training.
