Intel Unveils Latest AI Chip To Counter Nvidia

Tom Jowitt,
Linkedin

Nvidia AI chip challenger? Intel takes wraps off its latest artificial intelligence chip, called the Gaudi 3 AI accelerator

Intel Corp is seeking to challenge Nvidia’s stranglehold on the AI chip sector, after it unveiled its latest AI processor dubbed Gaudi 3.

Intel in its Tuesday announcement made clear it is seeking to tackle Nvidia’s domination of the AI chip market, after saying that its Gaudi 3 “breaks down proprietary walls to bring choice to enterprise GenAI market.”

Last week Intel’s share price had fallen after it disclosed a huge operating loss for its foundry contract chip-making business. A regulatory filing showed the unit had posted operating losses of $7 billion in 2023, compared with a $5.2 billion loss in 2022.

An Intel chip fab in New Mexico. Image credit: Intel
An Intel chip fab in New Mexico. Image credit: Intel

Gaudi 3 AI accelerator

Intel’s Gaudi 3 AI accelerator builds on the performance and scalability of its Gaudi 2 predecessor, by delivering “4x AI compute for BF16, 1.5x increase in memory bandwidth, and 2x networking bandwidth for massive system scale out compared to its predecessor.”

Intel said Gaudi 3 is a significant leap in performance and productivity for AI training and inference on popular large language models (LLMs) and multimodal models.

“In the ever-evolving landscape of the AI market, a significant gap persists in the current offerings. Feedback from our customers and the broader market underscores a desire for increased choice,” stated Justin Hotard, Intel executive VP and GM of the Data Center and AI Group.

“Enterprises weigh considerations such as availability, scalability, performance, cost, and energy efficiency,” said Hotard. “Intel Gaudi 3 stands out as the GenAI alternative presenting a compelling combination of price performance, system scalability, and time-to-value advantage.”

Intel introduced the Intel Gaudi 3 AI accelerator on April 9, 2024, at the Intel Vision event in Phoenix, Arizona. Image credit Intel

Intel said the Gaudi 3 accelerator is architected for efficient large-scale AI compute, and is manufactured on a 5 nanometer (nm) process and offers significant advancements over its predecessor.

Intel says it is designed to allow activation of all engines in parallel – with the Matrix Multiplication Engine (MME), Tensor Processor Cores (TPCs) and Networking Interface Cards (NICs) – enabling the acceleration needed for fast, efficient deep learning computation and scale.

Among its key features are:

  • AI-Dedicated Compute Engine
  • Memory Boost for LLM Capacity Requirements
  • Efficient System Scaling for Enterprise GenAI
  • Open Industry Software for Developer Productivity
  • Gaudi 3 PCIe

Intel claims the Gaudi 3 accelerator will deliver significant performance improvements for training and inference tasks on leading GenAI models.

Intel introduced the Intel Gaudi 3 AI accelerator on April 9, 2024, at the Intel Vision event in Phoenix, Arizona.
Image credit Intel

Specifically, the Intel Gaudi 3 accelerator is projected to deliver on average versus Nvidia H100:

  • 50 percent faster time-to-train across Llama 7B and 13B parameters, and GPT-3 175B parameter models.
  • 50 percent faster inference throughput2 and 40 percent greater inference power-efficiency across Llama 7B and 70B parameters, and Falcon 180B parameter models.
  • 30 percent faster inferencing on Llama 7B and 70B parameters, and Falcon 180B parameter models against Nvidia H200.

The Intel Gaudi 3 accelerator will apparently be available to original equipment manufacturers (OEMs) in the second quarter of 2024.

Developers can now get started with access to Intel Gaudi 2-based instances on the developer cloud.

Nvidia competition

It is clear that Intel is seeking to challenge Nvidia’s domination of the AI chip market, and meet growing demand for suitable AI processors.

Nvidia is estimated to have a 80 percent share of the AI chip market, and in February 2024 the GPU powerhouse overtook Amazon in terms of market value.

It should be remembered that Nvidia’s H100 and A100 chips serve as a generalised, all-purpose AI processor for many AI customers.

And these AI chips are not cheap.

While Nvidia does not disclose H100 prices, each chip can reportedly sell from $16,000 to $100,000 depending on the volume purchased and other factors.

Meta for example apparently plans to bring its total stock to 350,000 H100s chips in 2024, demonstrating the hefty financial investment required to compete in this sector, and the value of organisation supplying these semiconductors.

And Nvidia is not sitting still.

Last month Nvidia introduced the next-generation AI accelerator chips.

The new platform, “Blackwell”, powers the B200 GPU that is the successor to the current generation of “Hopper”-based H100.