Buyers of Nvidia GPUs
AI cloud start-up Vultr valued at $3.5B; Hyperscalers gorge on Nvidia GPUs while AI semiconductor market booms
Over the past two years, AI model builders OpenAI, Anthropic and Elon Musk’s xAI have raised nearly $40bn between them. Other sizeable investment rounds this week alone included $500mn for Perplexity, an AI-powered search engine, and $333mn for Vultr, part of a new band of companies running specialized cloud data centers to support AI.
Cloud AI startup Vultr raised $333 million in a financing round this week from Advanced Micro Devices (AMD) and hedge fund LuminArx Capital Management. That’s a sign of the super hot demand for AI infrastructure. West Palm Beach, Fla.-based Vultr said it is now valued at $3.5 billion and plans to use the financing to acquire more graphics processing units (GPUs) which process AI models. The funding is Vultr’s first injection of outside capital. That’s unusually high for a company that had not previously raised external equity capital. The average valuation for companies receiving first-time financing is $51mn, according to PitchBook.
Vultr said its AI cloud service, in which it leases GPU access to customers, will soon become the biggest part of its business. Earlier this month, Vultr announced plans to build its first “super-compute” cluster with thousands of AMD GPUs at its Chicago-area data center. Vultr said its cloud platform serves hundreds of thousands of businesses, including Activision Blizzard, the Microsoft-owned videogame company, and Indian telecommunications giant Bharti Airtel. Vultr’s customers also use its decade-old cloud platform for their core IT systems, said Chief Executive J.J. Kardwell. Like most cloud platform providers, Vultr isn’t using just one GPU supplier. It offers Nvidia and AMD GPUs to customers, and plans to keep doing so, Kardwell said. “There are different parts of the market that value each of them,” he added.
Vultr’s plan to expand its network of data centers, currently in 32 locations, is a bet that customers will seek greater proximity to their computing infrastructure as they move from training to “inference” — industry parlance for using models to perform calculations and make decisions.
Vultr runs a cloud computing platform on which customers can run applications and store data remotely © Vultr
……………………………………………………………………………………………………………………………………………………………………………………………..
The 10 biggest cloud companies — dubbed hyperscalers — are on track to allocate $326bn to capital expenditure in 2025, according to analysts at Morgan Stanley. While most depend heavily on chips made by Nvidia, large companies including Google, Amazon and Facebook are designing their own customized silicon to perform specialized tasks. Away from the tech mega-caps, emerging “neo-cloud” companies such as Vultr, CoreWeave, Lambda Labs and Nebius have raised billions of dollars of debt and equity in the past year in a bet on the expanding power and computing needs of AI models.
AI chip market leader Nvidia, which alongside other investors, provided more than $400 million to AI cloud provider CoreWeave [1.] in 2023. CoreWeave last year also secured $2.3 billion in debt financing by using its Nvidia GPUs as collateral.
Note 1. CoreWeave is a New Jersey-based company that got its start in cryptocurrency mining.
The race to train sophisticated AI models has inspired the commissioning of increasingly large “supercomputers” (aka AI Clusters) that link up hundreds of thousands of high-performance GPU chips. Elon Musk’s start-up xAI built its Colossus supercomputer in just three months and has pledged to increase it tenfold. Meanwhile, Amazon is building a GPU cluster alongside Anthropic, developer of the Claude AI models. The ecommerce group has invested $8bn in Anthropic.
Hyperscalers are big buyers of Nvidia GPUs:
Analysts at market research firm Omdia (an Informa company) estimate that Microsoft bought 485,000 of Nvidia’s “Hopper” chips this year. With demand outstripping supply of Nvidia’s most advanced graphics processing units for much of the past two years, Microsoft’s chip hoard has given it an edge in the race to build the next generation of AI systems.
This year, Big Tech companies have spent tens of billions of dollars on data centers running Nvidia’s latest GPU chips, which have become the hottest commodity in Silicon Valley since the debut of ChatGPT two years ago kick-started an unprecedented surge of investment in AI.
- Microsoft’s Azure cloud infrastructure was used to train OpenAI’s latest o1 model, as they race against a resurgent Google, start-ups such as Anthropic and Elon Musk’s xAI, and rivals in China for dominance of the next generation of computing. Omdia estimates
- ByteDance and Tencent each ordered about 230,000 of Nvidia’s chips this year, including the H20 model, a less powerful version of Hopper that was modified to meet U.S. export controls for Chinese customers.
- Meta bought 224,000 Hopper chips.
- Amazon and Google, which along with Meta are stepping up deployment of their own custom AI chips as an alternative to Nvidia’s, bought 196,000 and 169,000 Hopper chips, respectively, the analysts said. Omdia analyses companies’ publicly disclosed capital spending, server shipments and supply chain intelligence to calculate its estimates.
The top 10 buyers of data center infrastructure — which now include relative newcomers xAI and CoreWeave — make up 60% of global investment in computing power. Vlad Galabov, director of cloud and data center research at Omdia, said some 43% cent of spending on compute servers went to Nvidia in 2024. “Nvidia GPUs claimed a tremendously high share of the server capex,” he said.
What’s telling is that the biggest buyers of Nvidia GPUs are the hyperscalers who design their own compute servers and outsource the detailed implementation and manufacturing to Taiwan and China ODMs! U.S. compute server makers Dell and HPE are not even in the ball park!
What about #2 GPU maker AMD?
Dave McCarthy, a research vice president in cloud and edge services at research firm International Data Corp (IDC). “For AMD to be able to get good billing with an up-and-coming cloud provider like Vultr will help them get more visibility in the market.” AMD has also invested in cloud providers such as TensorWave, which also offers an AI cloud service. In August, AMD bought the data-center equipment designer ZT Systems for nearly $5 billion. Microsoft, Meta Platforms and Oracle have said they use AMD’s GPUs. A spokesperson for Amazon’s cloud unit said the company works closely with AMD and is “actively looking at offering AMD’s AI chips.”
Promising AI Chip Startups:
Nuvia: Founded by former Apple engineers, Nuvia is focused on creating high-performance processors tailored for AI workloads. Their chips are designed to deliver superior performance while maintaining energy efficiency, making them ideal for data centers and edge computing.
SambaNova Systems: This startup is revolutionizing AI with its DataScale platform, which integrates hardware and software to optimize AI workloads. Their unique architecture allows for faster training and inference, catering to enterprises looking to leverage AI for business intelligence.
Graphcore: Known for its Intelligence Processing Unit (IPU), Graphcore is making waves in the AI chip market. The IPU is designed specifically for machine learning tasks, providing significant speed and efficiency improvements over traditional GPUs.
Market for AI semiconductors:
- IDC estimates it will reach $193.3 billion by the end of 2027 from an estimated $117.5 billion this year. Nvidia commands about 95% of the market for AI chips, according to IDC.
- Bank of America analysts forecast the market for AI chips will be worth $276 billion by 2027.
References:
https://www.ft.com/content/946069f6-e03b-44ff-816a-5e2c778c67db
https://www.restack.io/p/ai-chips-answer-top-ai-chip-startups-2024-cat-ai