Meta Shows Sound-Capable AI Video Generation Tool

Facebook parent Meta has introduced an artificial intelligence (AI) video-generation tool with audio effects that it says rank favorably with competitors from the likes of OpenAI and ElevenLabs, as tech giants seek an edge in the emerging generative AI market.

The tool can generate video from user prompts as well as editing existing videos, for instance adding pom-poms into the hands of a man running in the desert.

In another example, a video of a man skateboarding in a parking lot was changed so that the surface was covered in splashing water.

Users can also use photos of themselves to make the tool create a video of themselves doing an action such as painting on a canvas.

Short videos

Meta also shared 10-second videos of animals swimming and surfing, and a DJ spinning records with a cheetah beside her.

The Movie Gen tool can be used to add sound effects and music, such as audible splashes accompanying a waterfall the engine and tyre screeches of a sports car, and hopeful or suspenseful orchestral music.

The tool, which is not yet available to the general public, was announced shortly after Meta’s annual Connect event at which it announced version 3.2 of its Llama large language model, which powers its Meta AI chatbot.

The feature could be introduced into Meta’s products such as Facebook and Instagram next year, the company said.

Movie Gen Video consists of 30 billion parameters, with the audio feature having 13 billion parameters, compared to Llama 3.1’s 405 billion, in a measure of how capable a model is.

It can produce videos of up to 16 seconds and Meta said it compared favourably to competitors from Runway, OpenAI, ElevenLabs and Kling in blind tests.

The models were trained on a combination of licensed and publicly available data sets, Meta said, without providing details.

Copyright controversy

The source of AI training data continues to be a source of controversy, with multiple lawsuits claiming AI training has violated copyright, while publishers such as Time and News Corp have reached licensing deals for such uses.

OpenAI introduced its Sora video-generation model earlier this year and has not yet made it public or disclosed a public launch date.

In September Google said it may make aspects of its Veo video tool available to YouTube Shorts creators next year.

Start-ups including Runway and Pika have made such technology available to the public, spurring a debate within Hollywood around how such technologies should be used to enhance moviemaking.

In May actor Scarlett Johansson accused OpenAI of creating a voice for its ChatGPT chatbot that was overly similar to hers, after she had declined to work with the company.