Meta Releases Open Source Llama 3.1 AI Model

Meta Platforms has released its openly accessible artificial intelligence (AI) model, which it claims is its “most capable” AI model to date.

Meta announced in a blog post that Llama 3.1 405B “is in a class of its own, with unmatched flexibility, control, and state-of-the-art capabilities that rival the best closed source models. Our new model will enable the community to unlock new workflows, such as synthetic data generation and model distillation.”

Last week Meta had confirmed it would withhold its upcoming multimodal Llama AI model from the European Union, citing the “unpredictable nature” of its regulatory environment, despite it being released under an open license in other regions.

Llama 3.1 405B

Meta said it is continuing to build out Llama to be a system by providing more components that work with the model, including a reference system.

It said it has bolstered this new AI model with new security and safety tools, including Llama Guard 3 and Prompt Guard, to help developers build responsibly.

It has also released a request for comment on the Llama Stack API, a standard interface Meta hopes will make it easier for third-party projects to leverage Llama models.

And Meta added that the ecosystem is primed and ready to go with over 25 partners, including AWS, Nvidia, Databricks, Groq, Dell, Azure, Google Cloud, and Snowflake offering services on day one.

“Llama 3.1 405B is the first openly available model that rivals the top AI models when it comes to state-of-the-art capabilities in general knowledge, steerability, math, tool use, and multilingual translation,” said Meta.

“With the release of the 405B model, we’re poised to supercharge innovation – with unprecedented opportunities for growth and exploration,” it added. “We believe the latest generation of Llama will ignite new applications and modeling paradigms, including synthetic data generation to enable the improvement and training of smaller models, as well as model distillation – a capability that has never been achieved at this scale in open source.”

Meta said that Llama 3.1 405B is its largest model yet, training on over 15 trillion tokens, which was a challenge.

“To enable training runs at this scale and achieve the results we have in a reasonable amount of time, we significantly optimised our full training stack and pushed our model training to over 16 thousand H100 GPUs, making the 405B the first Llama model trained at this scale,” said Meta.

Open source

In a separate blog post, CEO Mark Zuckerberg explained Meta’s decision to continue with open sourcing the technology, pointing to how open source Linux gained popularity to become “the industry standard foundation for both cloud computing and the operating systems that run most mobile devices.”

“I believe that AI will develop in a similar way,” said Zuckerberg. “Today, several tech companies are developing leading closed models. But open source is quickly closing the gap. Last year, Llama 2 was only comparable to an older generation of models behind the frontier. This year, Llama 3 is competitive with the most advanced models and leading in some areas.”

“Starting next year, we expect future Llama models to become the most advanced in the industry,” said Zuckerberg. “Today we’re taking the next steps towards open source AI becoming the industry standard. We’re releasing Llama 3.1 405B, the first frontier-level open source AI model, as well as new and improved Llama 3.1 70B and 8B models. In addition to having significantly better cost/performance relative to closed models, the fact that the 405B model is open will make it the best choice for fine-tuning and distilling smaller models.”

Zuckerberg stated that Meta is committed to open source AI, but stated that he believed “that open source is necessary for a positive AI future.”

“AI has more potential than any other modern technology to increase human productivity, creativity, and quality of life – and to accelerate economic growth while unlocking progress in medical and scientific research,” said Zuckerberg. “Open source will ensure that more people around the world have access to the benefits and opportunities of AI, that power isn’t concentrated in the hands of a small number of companies, and that the technology can be deployed more evenly and safely across society.”

Zuckerberg conceded that “at some point in the future, individual bad actors may be able to use the intelligence of AI models to fabricate entirely new harms from the information available on the internet.”

“At this point, the balance of power will be critical to AI safety,” he stated. “I think it will be better to live in a world where AI is widely deployed so that larger actors can check the power of smaller bad actors. This is how we’ve managed security on our social networks – our more robust AI systems identify and stop threats from less sophisticated actors who often use smaller scale AI systems. More broadly, larger institutions deploying AI at scale will promote security and stability across society. As long as everyone has access to similar generations of models – which open source promotes – then governments and institutions with more compute resources will be able to check bad actors with less compute.”