Groq and Nvidia in non-exclusive AI Inference technology licensing agreement; top Groq execs joining Nvidia

AI chip startup Groq [1.] today announced that it has entered into a non-exclusive licensing agreement with Nvidia for Groq’s AI inference technology [2.]. The agreement reflects a shared focus on expanding access to high-performance, low cost inference. As part of this agreement, Jonathan Ross, Groq’s Founder, Sunny Madra, Groq’s President, and other members of the Groq team will join Nvidia to help advance and scale the licensed technology. Groq will continue to operate as an independent company with Simon Edwards stepping into the role of Chief Executive Officer. GroqCloud will continue to operate without interruption. It remains to be seen how Groq’s new collaboration with Nvidia will effect its recent partnership with IBM.

Note 1. Founded in 2016, Groq specializes in what is known as inference, where artificial intelligence (AI) models that have already been trained respond to requests from users. While Nvidia dominates the market for training AI models (see Note 2.), it faces much more competition in inference, where traditional rivals such as Advanced Micro Devices have aimed to challenge it as well as startups such as Groq and Cerebras Systems.

Note 2. Training AI models (used by Nvidia GPUs) involves teaching a model to learn patterns from large amounts of data, while AI “inferencing” refers to using that trained model to generate outputs. Both processes demand massive computing power from AI chips.

…………………………………………………………………………………………………………………………………………………………………………………………………….

Groq has achieved a significant financial milestone, elevating its post-money valuation to $6.9 billion from $2.8 billion following a successful $750 million funding round in September. The company distinguishes itself in the competitive AI chip landscape by employing a unique architectural approach that does not rely on external high-bandwidth memory (HBM) chips. This design choice, leveraging on-chip static random-access memory (SRAM), mitigates the supply chain constraints currently impacting the global HBM market.

The LPU (Language Processing Unit) architecture, while enhancing inference speed for applications like chatbots, currently presents limitations regarding the maximum size of AI models that can be efficiently served. Groq’s primary competitor utilizing a similar architectural philosophy is Cerebras Systems, which has reportedly commenced preparations for an initial public offering (IPO) as early as next year. Both companies have strategically secured substantial contracts in the Middle Eastern market.

Nvidia’s investments in AI firms span the entire AI ecosystem, ranging from large language model developers such as OpenAI and xAI to “neoclouds” like Lambda and CoreWeave, which specialize in AI services and compete with its Big Tech customers. Nvidia has also invested in chipmakers Intel and Enfabrica. The company made a failed attempt around 2020 to acquire British chip architecture designer Arm Ltd. Nvidia’s wide-ranging investments — many of them in its own customers — have led to accusations that it’s involved in circular financing schemes reminiscent of the dot-com bubble. The company has vehemently denied those claims.

…………………………………………………………………………………………………………………………………………………………………..

This deal follows a familiar pattern in recent years where the world’s biggest technology firms pay large sums in deals with promising startups to take their technology and talent but stop short of formally acquiring the target.

A great example of that was Meta which in June invested ~$14.3 billion in Scale AI for a 49% stake in the company. That move valued the startup at around $29 billion. As part of that deal, 28 year old Alexandr Wang resigned as CEO of Scale AI to become Meta’s first-ever Chief AI Officer. He will remain on Scale AI’s board of directors. Wang’s team at the new “superintelligence” lab is tasked with building advanced AI systems that can surpass human-level intelligence.
In a similar but smaller scale deal, Microsoft agreed to pay AI startup Inflection about $650 million in cash in an unusual arrangement that would allow Microsoft to use Inflection’s models and hire most of the startup’s staff including its co-founders, a person familiar with the matter told Reuters. The high-profile AI startup’s models will be available on Microsoft’s Azure cloud service, the source said. Inflection is using the licensing fee to pay Greylock, Dragoneer and some other investors, the source added, saying the investors will get a return of 1.5 times what they invested.

Bernstein analyst Stacy Rasgon wrote in a note to clients on Wednesday after Groq’s announcement:

“The Nvidia-Groq deal appears strategic in nature for Nvidia as they leverage their increasingly powerful balance sheet to maintain dominance in key areas…..Antitrust would seem to be the primary risk here, though structuring the deal as a non-exclusive license may keep the fiction of competition alive (even as Groq’s leadership and, we would presume, technical talent move over to Nvidia). Nvidia CEO Jensen Huang’s “relationship with the Trump administration appears among the strongest of the key U.S. tech companies.”

…………………………………………………………………………………………………………………………………………………………………………………………………………….

Nvidia CEO Jensen Huang recently dedicated a significant portion of his 2025 GTC 2025 conference keynote speech to emphasize that Nvidia intends to maintain its market dominance as the AI sector increasingly transitions its focus from model training to inference workloads. Huang delivered two major GTC keynotes in 2025: The primary annual conference held in San Jose, California, in March 2025 and a second GTC in Washington, D.C., in October 2025. At these events, he emphasized the rise of “reasoning AI” and “agentic AI” as the drivers for an unprecedented 100x surge in demand for inference computing in just a couple of years. Huang announced that the new Blackwell system, designed as a “thinking machine” for reasoning, was in full production and optimized for both training and large-scale inference workloads.

Huang shared a vision of moving from traditional data centers to “AI factories“—ultra-high-performance computing environments designed specifically to generate intelligence at scale, positioning investment in Nvidia’s infrastructure as an economic necessity.