AI silicon
Custom AI Silicon: OpenAI, Anthropic, Hyperscalers, Huawei and the Battle with Nvidia, AMD and Broadcom
Disclaimer: ChatGPT was used extensively to write this article.
Executive Summary:
Custom AI silicon is entering a new phase. Google, Amazon, Microsoft and Meta have established proprietary accelerator programs, while OpenAI has introduced its first custom inference chip, Jalapeño, and Anthropic is assembling an in-house chip-design team.
These developments extend the industry trend examined in earlier IEEE Techblog articles, Custom AI Chips: Powering the Next Wave of Intelligent Computing and Google’s TPU Business Outpaces Rivals as Hyperscalers Accelerate Custom AI Silicon Strategies (see References for more).
The important new development is the entry of AI-model developers themselves into chip architecture and design. OpenAI is attempting to optimize hardware around the demands of its models and inference services, while Anthropic plans to pursue closer hardware-software co-design for Claude.
The competitive consequences extend beyond Nvidia and AMD. Broadcom and Marvell stand to benefit from demand for custom application-specific integrated circuits (ASICs), while Intel and specialized accelerator developers pursue alternative architectures.
The central question is whether custom silicon can deliver a sustained advantage in cost per AI token, energy efficiency and latency without sacrificing the flexibility and software ecosystem of merchant GPUs.
OpenAI – From Chip Partnership to Working Silicon:
OpenAI’s October 2025 agreement with Broadcom established a multiyear $10 billion custom-chip program for 10 gigawatts of OpenAI-designed accelerators and associated networking systems. Since then, the project has advanced from a development partnership to working silicon. On June 24, 2026, the companies unveiled Jalapeño, OpenAI’s first custom AI inference accelerator, designed specifically for large language models (LLMs). OpenAI designed the architecture, while Broadcom provides silicon implementation and networking expertise, and Celestica supports board and rack integration. TSMC will manufacture the chips. Engineering samples are running machine-learning workloads in the laboratory at targeted operating frequency and power. Initial deployment is planned for the end of 2026, followed by expansion over multiple generations.
In August 2026, OpenAI published initial benchmark results claiming substantial performance-per-watt and latency advantages over competing systems. These are company-reported results, not yet independent proof of a sustained production-scale advantage over Nvidia’s latest accelerators. Jalapeño is optimized for inference rather than general-purpose AI training, and OpenAI’s immediate priority is to meet its own rapidly growing computing demand.
The strategic significance is that OpenAI is extending its capabilities beyond AI models and software into custom hardware. By co-designing the chip, software kernels, memory movement, networking and serving systems, it aims to lower inference costs and improve responsiveness across ChatGPT, Codex and its API services. For Broadcom, the partnership reinforces the opportunity to supply custom silicon and networking technology to major AI developers without having to compete directly with Nvidia across the entire GPU market.
The distinction is significant: this is no longer simply a plan to develop proprietary silicon. OpenAI reported that engineering samples were running machine-learning workloads in its laboratories at production-target frequency and power.
Jalapeño is designed around large language model (LLM) inference rather than adapting a general-purpose GPU architecture. Its design emphasizes minimizing data movement and balancing compute, memory and networking resources. These considerations become increasingly important when serving interactive AI applications and agents that must generate responses quickly and execute multiple steps.
OpenAI designed the architecture, while Broadcom contributes silicon implementation and networking technology, and Celestica supports board and rack integration. OpenAI plans an initial deployment in small volumes near the end of 2026, with more substantial deployment expected in 2027.
The business rationale is straightforward. OpenAI operates ChatGPT, Codex and its API services at considerable computing scale. Even relatively modest improvements in inference efficiency could translate into meaningful savings if deployed across a large installed base.
However, Jalapeño’s initial benchmarks do not establish that it will outperform Nvidia’s latest systems across all workloads. Nor do they yet demonstrate its production-scale cost advantage. The relevant comparison must include complete systems, utilization, software development, power, cooling and operating costs.
The critical test is whether Jalapeño can deliver better economics at production scale. OpenAI must demonstrate that its claimed efficiency gains translate into lower total costs per AI token after accounting for manufacturing, memory, networking, power, cooling, utilization and software engineering. The initial results are promising, but the competitive outcome remains to be established.
Sources: OpenAI’s Jalapeño announcement and August benchmark results.
Author’s Note: The claims about Jalapeño’s performance are explicitly attributed to OpenAI.
……………………………………………………………………………………………………………………………………………
Anthropic – Building an Internal Design Capability:
Anthropic announced in August 2026 that it was forming an internal chip-design team to develop hardware optimized for Claude. The initiative reflects the same strategic logic as OpenAI’s: understanding a model’s architecture, inference patterns and software stack can help guide the design of more efficient processors.
Anthropic is not starting from a position of hardware independence. It already uses accelerators from Nvidia, Google and Amazon, among others. Its planned custom-silicon effort is an additional element of a multivendor strategy, not evidence that it is abandoning external suppliers.
The distinction between the two AI developers is important. OpenAI has demonstrated working silicon and published initial benchmarks; Anthropic is building a design organization and has not announced a production-ready proprietary accelerator or deployment timetable.
Designing advanced chips also requires substantial capital, specialized engineering talent and access to manufacturing and packaging capacity. Anthropic must ultimately demonstrate that the benefits justify those investments.
Source: Reuters on Anthropic’s chip-design team.
……………………………………………………………………………………………………………………………………………………..
Hyperscalers: Established Programs, Different Economics:
The major hyperscalers have already developed proprietary accelerators, as discussed in our September article. Their competitive positions differ according to the maturity of their hardware programs, their internal workloads and their ability to sell or rent computing capacity to external customers.
| Company | Custom silicon | Strategic emphasis |
|---|---|---|
| Tensor Processing Units (TPUs) | Training and inference, including external cloud customers | |
| Amazon | Trainium and Inferentia | AWS computing economics and customer choice |
| Microsoft | Maia | Azure AI inference and internal services |
| Meta | MTIA | Recommendation systems and generative AI inference |
| OpenAI | Jalapeño | LLM inference optimized around its models and serving software |
| Anthropic | In-house design initiative | Future hardware-software co-design for Claude |
Google’s position is particularly notable because it has an established TPU ecosystem and sells access to its accelerators beyond its own operations. Amazon similarly offers Trainium through AWS, giving customers an alternative to Nvidia-based infrastructure.
Microsoft and Meta are expanding their own programs, although proprietary silicon does not eliminate their need for merchant GPUs. Different workloads, development schedules and capacity requirements favor a mix of processors.
The broader point is that these companies are not necessarily trying to replace Nvidia everywhere. They are attempting to optimize selected workloads, diversify supply and improve the economics of operating AI infrastructure.
……………………………………………………………………………………………………………………………………………….
Huawei: Building an Alternative AI Silicon and Networking Ecosystem:
Huawei represents a different competitive challenge to NVIDIA, AMD and U.S. hyperscalers. Through its Ascend AI processors, HiSilicon semiconductor-design subsidiary, CANN software stack and high-speed interconnect technologies, Huawei is developing an integrated AI computing platform intended to reduce China’s dependence on foreign chips and software.
Huawei’s strategy extends beyond individual accelerators to cluster-scale infrastructure. Its September 2026 roadmap includes the Ascend 960DT, planned for the first quarter of 2027, and Ascend 960PR, planned for the third quarter 2026, followed by Ascend 970 and 980 generations in 2028 and 2029. Its UnifiedBus interconnect architecture is designed to connect processors, memory and storage across tightly coupled SuperPoD systems and larger AI clusters, potentially scaling to hundreds of thousands or even one million neural processing units (NPUs), according to Huawei’s announced architecture. These are roadmap and design targets, not independently verified production-scale performance results.
Manufacturing constraints: Huawei has also reported insufficient production capacity to meet domestic demand. That is a practical deployment constraint beyond the chip-design and software challenges.
Huawei’s principal challenge is translating this ambitious system architecture into competitive real-world performance. Advanced-chip manufacturing access, high-bandwidth memory (HBM) supply, power efficiency and the maturity of its CANN software ecosystem remain important constraints. NVIDIA benefits from a substantial lead in accelerator performance and a deeply established CUDA ecosystem, while AMD and hyperscalers pursue their own hardware and software strategies.
Nevertheless, Huawei is strategically significant because it combines AI silicon, interconnects, networking, storage and software within a broader computing and telecommunications portfolio. Its progress could reshape China’s domestic AI infrastructure market even if its chips do not match NVIDIA’s leading products on a per-accelerator basis. The competitive comparison must therefore consider total cluster performance, system availability, software compatibility, power consumption and cost—not chip specifications alone.
Huawei’s roadmap underscores a broader point: the AI-silicon competition is not simply a contest among chip designers. It is also a race to build complete, scalable computing platforms with the networking and software required to make thousands of accelerators work efficiently together.
Other AI Developers and Specialized Chip Companies:
The custom-silicon movement extends beyond OpenAI and Anthropic, but announced interest should not be confused with established production programs.
Alibaba’s T-Head and Baidu’s Kunlunxin are examples of Chinese semiconductor-development efforts associated with domestic AI and cloud infrastructure. Their importance includes reducing reliance on foreign suppliers amid restrictions on advanced semiconductor exports.
Among smaller AI-model developers, the strategic choices vary. A company can optimize models, compilers and kernels for existing processors without designing a chip of its own. Developing proprietary silicon becomes more attractive when workloads are large and predictable enough to justify substantial upfront investment.
Specialist chip companies pursue different alternatives to conventional GPUs:
- Cerebras uses a wafer-scale architecture designed to place substantial computing resources on a single silicon wafer.
- Groq developed a specialized architecture targeting low-latency inference and has entered a broader ecosystem that includes partnerships and technology integration.
- Intel offers its Gaudi accelerator family as an alternative AI-computing platform.
These vendors compete through different combinations of architecture, software, throughput, latency and system economics. They should not be treated as interchangeable with hyperscaler ASICs or general-purpose merchant GPUs.
……………………………………………………………………………………………………………………………………………..
Image from ChatGPT:
…………………………………………………………………………………………………………………………………………………………………………
Merchant Market AI Accelerator Chips from Nvidia and AMD :
Nvidia’s competitive advantage extends well beyond its GPU hardware. CUDA, optimized libraries, developer tools, networking and integrated rack-scale systems make its platform difficult to displace. Customers also benefit from a broad software ecosystem and the ability to use the same general platform across many workloads.
Nvidia’s vulnerability is that large AI operators have incentives to reduce costs and dependence on a single supplier. Custom ASICs can be attractive when workloads are sufficiently stable and deployment volumes justify specialized engineering.
AMD provides a second major merchant-GPU platform through its Instinct accelerators and ROCm software ecosystem. It offers customers another source of advanced computing capacity and an alternative to Nvidia’s platform.
AMD faces the challenge of expanding software adoption and competing against Nvidia’s established ecosystem. Nevertheless, the availability of multiple accelerator suppliers is strategically valuable to cloud providers and AI developers.
Neither company is likely to be displaced solely because a few major customers introduce proprietary chips. Merchant GPUs remain useful for new models, changing workloads and applications for which custom silicon is not yet economically justified.
Broadcom and Marvell – Beneficiaries of the Custom-ASIC Trend:
Broadcom occupies a particularly important position because it can benefit from the growth of customer-designed silicon without relying exclusively on sales of a proprietary general-purpose AI accelerator.
Its custom-ASIC capabilities, high-speed connectivity and networking products support the development of customer-specific AI systems. Its work with OpenAI and Meta illustrates how chip designers can retain control of their architectures while relying on semiconductor partners for implementation and infrastructure.
Marvell offers another set of custom-ASIC and data-infrastructure capabilities, including high-speed interfaces and connectivity technologies. It can benefit as companies develop specialized processors and build the systems needed to connect them.
The distinction from Nvidia and AMD is fundamental:
- Nvidia and AMD sell merchant accelerators to a broad customer base.
- Broadcom and Marvell help customers develop customized silicon and the associated data infrastructure.
- Hyperscalers and AI developers use their knowledge of workloads to decide where proprietary hardware can improve performance or economics.
These roles can overlap. Broadcom’s custom-chip business, for example, can benefit from a customer moving some workloads away from merchant GPUs. But custom silicon does not eliminate the need for advanced manufacturing, memory, packaging, networking or other semiconductor suppliers.
…………………………………………………………………………………………………………………………………………
What Will Determine the Winners?
The decisive metric is not peak computing performance alone. AI operators need to know how much useful work a complete system can deliver at an acceptable cost and latency.
Five factors will be especially important:
- Cost per token: Total cost of generating useful AI output at specified quality, throughput and response time.
- Performance per watt: The ability to serve more workloads without proportionally increasing electricity consumption.
- Memory and networking: The ability to keep model data accessible and minimize communication delays.
- Software maturity: The availability of compilers, optimized kernels, debugging tools and support for changing models.
- Production economics: Manufacturing yields, system utilization, cooling, reliability, deployment scale and engineering costs.
Custom ASICs are most compelling when a company has large, repeatable workloads and can optimize the model, software and hardware together. Merchant GPUs remain attractive when flexibility, rapid deployment and broad software support outweigh the potential benefits of specialization.
Power availability is an additional constraint. A more efficient processor can increase the useful computing capacity delivered by a given data-center power budget, although total electricity consumption may still rise as AI demand expands.
………………………………………………………………………………………………………………………………………
Conclusions:
Custom AI silicon is becoming a more important part of the industry’s computing strategy. The entry of OpenAI and Anthropic into chip design extends a trend already established by Google, Amazon, Microsoft and Meta.
OpenAI’s Jalapeño is a meaningful milestone because it has progressed from a partnership announcement to working silicon and published initial performance results. Anthropic’s initiative remains at an earlier stage. Both illustrate the growing interest in co-designing AI models, software and hardware.
The market is likely to remain heterogeneous. Nvidia and AMD will continue supplying merchant accelerators, while Broadcom and Marvell can benefit from demand for customer-specific silicon and high-speed networking. Specialized architectures will compete for workloads where their performance and cost advantages are compelling.
The ultimate competitive advantage will come from delivering more effective AI computation per dollar and per watt—not simply from designing a proprietary chip.
……………………………………………………………………………………………………………………………………
References:
Google’s TPU Business Outpaces Rivals as Hyperscalers Accelerate Custom AI Silicon Strategies
Custom AI Chips: Powering the next wave of Intelligent Computing
OpenAI and Broadcom in $10B deal to make custom AI chips
Meta’s “Iris” AI Chip for MTIA: Implications for Telecom-Grade Optical Networking, DCI and High Capacity Ethernet Fabrics
2026 OCP Summit: Full-Stack AI Networking Silicon from Nvidia/Cisco and Broadcom; Ethernet vs InfiniBand Interconnects
The AI Infrastructure Build-Out: A $10 Trillion Bet on Compute, Power, and Networks
Will Google Cloud’s AI and data analytics revenue +TPU IP licensing income offset huge AI CAPEX to produce a decent ROI?
Custom AI Silicon: OpenAI, Anthropic, Hyperscalers and the Battle with Nvidia, AMD and Broadcom
Microsoft Maia 300 Accelerator Chip: Co-Designing AI Inference for Improved Performance per Watt
Microsoft’s third-generation custom AI accelerator silicon (made by TSMC) combines a multi-process-node chiplet architecture with higher-density racks, targeting improved compute efficiency and lower infrastructure costs.
Executive Summary:
Microsoft announced its Maia 300 AI accelerator on October 12, 2026, outlining plans for multi-gigawatt deployments beginning in 2027. The company is positioning the third generation of its custom AI silicon as a key component of Azure’s infrastructure for inference workloads, including coding, long-context reasoning, and multi-step AI agent tasks.
Maia 300 reflects a broader shift in AI infrastructure design: optimizing the complete computing system rather than maximizing accelerator performance in isolation. Its chiplet architecture, high-bandwidth memory subsystem, and system-level integration are intended to improve throughput, power efficiency, and the economics of delivering AI services at scale.
Microsoft claims that Maia 300 delivers approximately 40% more tokens per megawatt than leading GPU-based systems. However, independent validation will require comparable workloads, clearly defined power boundaries, and measurements of throughput, latency, and output quality. The accelerator’s significance will ultimately depend on whether its architectural choices translate into lower costs per useful AI task under real-world operating conditions.
Azure Maia 200 now in production; Azure Maia 300 coming in 2027. Image credit: Microsoft
…………………………………………………………………………………………………………………………………………………………………………………………….
At a Glance- Key capabilities of the Maia 300 system:
Less data movement, more useful compute: Moving data efficiently, rather than raw compute power, is increasingly the bottleneck in large inference systems. Maia 300’s software-defined dataflow architecture and AI-assisted kernel optimization work together to maximize locality, reduce data movement and improve utilization across compute, memory and networking resources. By treating dataflow as a first-class design principle, Maia 300 delivers a significant performance-per-watt advantage over traditional GPU architectures.
Higher memory bandwidth within every watt: As AI workloads become increasingly context and memory-bound, Maia 300 delivers industry-leading HBM bandwidth per watt. Eight HBM4 stacks provide 18.0 TB/s of peak memory bandwidth within a 950-watt TDP, enabling efficient execution of large models and long-context workloads. Specialized kernel support uses on-die SRAM as a software-managed scratchpad, reducing pressure on HBM.
Chiplet architecture for enhanced efficiency: A new multi-chiplet architecture separates compute, memory-controller and I/O functions, improving efficiency, scalability and cost structure. Compute chiplets are built on TSMC’s N2P process node to maximize performance-per-watt.
Larger context, richer intelligence: By designing for advanced low-precision formats, including enhanced MXFP4, Maia 300 reduces memory and bandwidth requirements for both computation and KV-cache storage. The result is larger context windows, higher throughput and improved model efficiency within the same power envelope.
Scale-up networking at datacenter scale: Maia 300 treats networking as a seamless extension of the software-defined dataflow architecture. Rather than moving data through separate communication layers, dataflow operations span compute, memory and network resources as a single system. High-bandwidth interconnects between Maia accelerators are optimized for tensor parallelism, while a custom AI transport delivers performant and reliable scale-up communication across accelerators and racks. The AI-optimized transport leverages Ethernet-based networking infrastructure to enable a unified topology that scales efficiently from a single rack to datacenter-scale deployments.
Higher AI throughput from every rack: Maia 300 rack systems bring together 96 accelerators within a 167-kilowatt power envelope, delivering 27.6 TB of HBM capacity and 1.7 PB/s of aggregate HBM bandwidth across a high-speed scale-up domain. That density allows Microsoft to serve more simultaneous AI interactions and larger workloads from each rack, improving datacenter capacity and the economics of expanding AI services.
………………………………………………………………………………………………………………………………………………………………………………………….
Chiplet Architecture and Process Technology:
Maia 300 uses a chiplet architecture that partitions functions across multiple dies manufactured using different semiconductor process technologies. According to the reported design, the primary compute chiplets use TSMC’s advanced 2 nm-class technology, while other components use a 3 nm process. Other components, including those handling data movement between the processor and the rest of the system, use TSMC’s less expensive 3-nanometer process.
This approach differs from the Nvidia AI accelerator architecture described in the Reuters article, which combines a large central compute die with surrounding high-bandwidth memory chips. Microsoft’s design assigns different functions to different process nodes rather than manufacturing all components using the most advanced process.
This approach reflects an important principle in accelerator design: not every component benefits equally from the most advanced manufacturing process. Allocating compute-intensive functions to leading-edge technology while implementing other functions on different process nodes can improve manufacturing flexibility and potentially reduce overall costs.
However, process-node selection is only one aspect of the design. The performance of a chiplet-based accelerator also depends on die-to-die interconnect bandwidth, communication latency, packaging efficiency, power delivery, and thermal management. These factors determine how effectively the individual components operate as a unified processor.
For AI inference, this system-level perspective is particularly important. Increasing arithmetic throughput provides limited benefit when memory access or data movement becomes the dominant bottleneck. The architectural challenge is therefore to balance compute resources, memory bandwidth, and communication paths to sustain high utilization across representative workloads.
Memory Bandwidth and Inference Performance:
Memory architecture is central to the performance of large language model inference. Maia 300 incorporates eight high-bandwidth memory (HBM4) stacks, with Microsoft reporting peak memory bandwidth of 18.0 TB/s and a thermal design power of 950 W per accelerator.
These specifications provide useful indicators of the accelerator’s intended operating envelope, but peak bandwidth alone does not establish application performance. Actual results depend on memory access patterns, model architecture, batch size, sequence length, cache behavior, and the efficiency of the software stack.
Inference also encompasses distinct computational phases. During prompt processing, or prefill, accelerators process input tokens and may be constrained primarily by compute throughput. During autoregressive token generation, or decode, performance can become more sensitive to memory bandwidth, model-weight movement, and the size of the key-value (KV) cache used to retain context.
Long-context reasoning and agentic workloads introduce additional demands on memory capacity, data movement, and repeated model execution. Consequently, evaluating Maia 300 requires more than peak compute specifications: it requires sustained measurements across realistic inference workloads, including both throughput and latency.
Microsoft’s emphasis on inference-oriented design is therefore significant. The practical advantage will depend on how effectively the architecture converts available compute and memory resources into useful output tokens while meeting application-level latency requirements.
Rack Density, Power, and System-Level Efficiency:
Microsoft plans to deploy Maia 300 in racks containing 96 accelerators, with reported power consumption of approximately 167 kW per rack. For comparison, Nvidia’s GB300 rack configuration has been reported to contain 72 GPUs and consume approximately 142 kW. These figures illustrate the increasing power density of AI infrastructure, but they do not establish comparative performance or energy efficiency. Accelerator counts, rack configurations, and power measurements must be interpreted in the context of the complete system, including memory, networking, power conversion, and cooling.
Dividing reported rack power by accelerator count yields approximately 1.74 kW per Maia 300 accelerator position and 1.97 kW per GB300 GPU position. These are normalized rack-level figures, not measurements of individual accelerator power consumption. Differences in system configuration and measurement boundaries also limit their value as a direct comparison.
Microsoft separately claims that Maia 300 delivers approximately 40% more tokens per megawatt than leading GPU-based systems. This is a potentially important improvement because energy consumption is a major component of the cost of operating large-scale AI infrastructure. Nevertheless, the claim cannot be evaluated rigorously without identifying the comparison systems, workloads, software configurations, and power boundaries.
“It’s basically delivering more intelligence for every watt, every dollar, and every byte of memory by optimizing across the entire system,” Rani Borkar, president of Azure hardware systems and infrastructure at Microsoft, told Reuters.
A meaningful comparison should report aggregate token throughput alongside time to first token, inter-token latency, and sustained utilization. It should also distinguish accelerator or rack power from total facility power, which includes cooling and other datacenter overheads.
Ultimately, the relevant metric is not simply how many accelerators fit into a rack or how much power each rack consumes. It is how much useful inference work the complete system delivers for a given energy budget, while satisfying latency and quality requirements.
Hardware–Software Co-Design and Infrastructure Economics:
Custom AI silicon gives Microsoft greater control over the interaction between accelerator architecture, inference software, networking, and Azure’s datacenter infrastructure. This control can enable workload-specific optimization, improved resource utilization, and greater flexibility in deploying models across different hardware platforms.
However, custom silicon does not automatically produce lower costs. Its economics depend on manufacturing yields, packaging costs, software maturity, deployment scale, utilization, and the engineering effort required to optimize models and inference runtimes. Hardware specialization can also introduce trade-offs in flexibility as models and workloads evolve.
Microsoft’s reported plans for Maia 300 deployments spanning multiple gigawatts beginning in 2027 indicate the scale at which these considerations become important. The company also expects capital expenditures to exceed $175 billion in calendar year 2026, reflecting a broader investment in AI infrastructure rather than spending attributable solely to Maia 300.
At this scale, small improvements in sustained throughput per watt can have substantial operational consequences. Higher throughput may increase service capacity within a fixed power envelope, while lower energy consumption per token may reduce operating costs. Yet the financial benefit depends on actual utilization, workload mix, hardware lifecycle, and the total cost of operating the infrastructure.
The strategic objective is therefore broader than reducing dependence on third-party accelerators. It is to improve control over the cost, capacity, and performance of AI services through coordinated hardware and software design.
Deployment Scale and Validation Challenges:
Microsoft’s planned Maia 300 deployment raises several engineering questions that cannot be resolved by specifications alone.
First, how does the accelerator perform across different model sizes, context lengths, and inference patterns? Second, can its reported efficiency advantage be sustained under production concurrency and latency constraints? Third, how do networking, memory utilization, and software optimization affect performance when hundreds or thousands of accelerators operate as part of a larger service?
Independent evaluation should compare Maia 300 against clearly identified competing systems using equivalent models, output-quality requirements, precision settings, and inference software configurations. Measurements should include tokens per second, time to first token, inter-token latency, energy per generated token, and cost per million tokens. Agentic workloads may additionally require cost and latency measurements per successfully completed task.
These results should be reported with explicit power boundaries and sufficient methodological detail to permit reproduction. Without such evidence, peak specifications and vendor-reported efficiency figures remain useful indicators of design objectives, but not definitive proof of superior system performance.
Conclusions:
Microsoft Maia 300 illustrates how AI accelerator development is moving beyond the pursuit of peak compute performance toward system-level optimization. Its chiplet architecture, HBM4 memory subsystem, and integration with Azure’s inference infrastructure reflect the importance of balancing computation, memory bandwidth, data movement, and power consumption.
The central question is whether these architectural choices deliver sustained improvements in useful inference throughput, latency, and cost under real production conditions. Microsoft’s reported 40% advantage in tokens per megawatt is a promising claim, but its significance depends on transparent and comparable measurements.
As AI deployments grow, the competitive advantage of custom silicon will increasingly be determined by the efficiency of the complete system—not the specifications of the accelerator alone. Maia 300’s long-term importance will depend on whether Microsoft can translate hardware–software co-design into measurable gains in service capacity, energy efficiency, and the cost of delivering useful AI results.
…………………………………………………………………………………………………………………………………………………………………………………………………………………………………………..
References:
Reuters, “Microsoft plans huge deployment of its own AI chips, with an eye on saving costs,” Oct. 12, 2026.
Microsoft, official Maia 300 announcement and technical architecture materials, Oct. 12, 2026.
Nvidia, GB300 rack architecture and system power specifications. Cite the specific product documentation used to establish the 72-GPU configuration and 142 kW power figure.
TSMC, semiconductor process technology documentation. Cite the relevant process-node documentation to substantiate the manufacturing-node assignments reported for Maia 300.
Google’s TPU Business Outpaces Rivals as Hyperscalers Accelerate Custom AI Silicon Strategies
AWS to deploy AI inference chips from Cerebras in its data centers; Anapurna Labs/Amazon in-house AI silicon products
Huawei’s Ascend Silicon Roadmap Extends From AI Chips to Cluster-Scale Infrastructure
Custom AI Chips: Powering the next wave of Intelligent Computing
The AI Infrastructure Build-Out: A $10 Trillion Bet on Compute, Power, and Networks
Analysis: Nvidia–MediaTek Deepen AI Platform Collaboration -What it Means for AI RAN & DCI
Will Google Cloud’s AI and data analytics revenue +TPU IP licensing income offset huge AI CAPEX to produce a decent ROI?
Google’s TPU Business Outpaces Rivals as Hyperscalers Accelerate Custom AI Silicon Strategies
Executive Summary:
Google parent Alphabet’s emerging business of selling artificial intelligence (AI) accelerator chips is twice as large as a cloud computing rival, Google executive Thomas Kurian claimed Tuesday at a Goldman Sachs investors conference.
On July 22, Google reported second-quarter cloud-computing revenue of $24.77 billion, up 82% year over year, driven by artificial intelligence workloads, handily beating estimates of $22.46 billion. For the first time, Google included third-party sales of AI accelerator chips, called tensor-processing units, in cloud revenue.
Kurian, head of Google’s cloud business, made these remarks at Goldman Sachs’ Communacopia conference:
“We offer the best computational infrastructure for AI, and we offer 3 types of silicon. NVIDIA GPUs, our own Tensor Processing Units (TPUs), custom ARM silicon. [As AI models generate code awe also offer our own Arm processors to run that code.] We offer 2.7x better price performance for training, 80% better price performance for inference, 30% better price performance for CPUs. All of that allows us to differentiate our portfolio from other providers. It allows us to offer solutions to financial markets and capital markets.”
“The size of our accelerator business, our TPU business, is more than twice that of the next-largest hyperscaler.”
“Our platform is called Gemini Enterprise. It is used by over 90% of the Fortune 100 and thousands of small businesses. It’s used in a very specific way. People want to use it as a reasoning agent. So break the plan, understand the steps that are needed, reason on it and execute the steps. So it uses a reasoning agent to understand all the information in the company to then automate that workflow process. And when it does it, you want strong controls. What kinds of controls? Companies are worried about security. They’re worried about auditing, what these agents are doing. They want to manage costs and set budget caps. We have all those controls. And we allow people to use the right model for the right task. So you don’t have to always use the most expensive model, saving people a lot of money in doing so. We have a range of companies from insurance.”
–>You can read the entire transcript here. For more on Google’s TPUs please see:
Will Google Cloud’s AI and data analytics revenue +TPU IP licensing income offset huge AI CAPEX to produce a decent ROI?
Google’s TPU photo
……………………………………………………………………………………………………………………………………………………
Kurian added that Google monetizes TPU systems through three business models. One is letting companies rent TPU processing at its own cloud business. Also, Google sells TPU systems directly for deployment in customers’ data centers. One such customer is Anthropic. Third, Google also sells TPUs through a Blackstone cloud computing joint venture.
Google’s AI Chip Business:
In May, Google introduced Ironwood, its eighth-generation of TPUs. The Ironwood TPUs target both training of AI models and “inferencing” — processing AI workloads.
In a report published Aug. 24, Morgan Stanley analyst Brian Nowak estimated that Google cloud could garner $84 billion in “first party” — meaning non-cloud rental — TPU sales in 2027.
“We are raising our TPU sale estimates to $27 billion selling at a 30% gross margin,” Nowak said. “In all, we now expect Google to sell 0.3 gigawatts of TPU systems in the second half of 2026, 3.2 gigawatts in 2027 and 4.2 gigawatts in 2028. This translates into $84 billion/$108 billion of TPU-related Google cloud revenue in 2027 and 2028.”
In Q2-2026, Google said its cloud computing order backlog jumped to $514 billion, up from $460 billion in Q1. The backlog is converted into realized revenue as new data centers come online and crunch artificial intelligence-related workloads — training AI models and processing AI apps.
Google has increased its 2026 capital spending guidance to a range of $195 billion to $205 billion. Most of the spendings is going toward AI data centers and AI model development. In Q2, capital spending jumped 100% from a year earlier to $44.9 billion.
Kurian, a former top executive at Oracle, took over as the cloud-computing unit’s CEO in November 2018. When Kurian arrived, Google’s cloud customers were mostly other tech companies. Under Kurian, Google has targeted enterprise customers with cloud-based data-analytics and artificial intelligence tools.
Hyperscaler Custom Silicon: Meta, Microsoft, Oracle, and the Shift to In-House AI Accelerators:
While Google’s TPU business has reached a scale that Kurian says is more than twice that of the next-largest hyperscaler, other cloud and platform operators are rapidly expanding their own custom AI silicon programs to reduce dependence on Nvidia GPUs and optimize cost, power, and workload-specific performance.
Amazon.com has developed in-house Trainium AI accelerators while Microsoft has developed Maia AI chips. Amazon is further ahead than Microsoft in selling AI chips to outside customers, analysts say.
Meta – MTIA Family Targets Inference at Scale:
Meta has moved aggressively into custom silicon with its Meta Training and Inference Accelerator (MTIA) family, announcing four new chips — MTIA 300, 400, 450, and 500 — in March 2026 as part of a strategy to diversify hardware sources and lower AI infrastructure costs. The MTIA 300 entered production in mid-2026, with subsequent generations rolling out on an approximately six-month cadence through 2027.
Meta’s MTIA chips are manufactured by TSMC and co-developed with Broadcom under a multi-year partnership extending through 2029. The roadmap spans ranking and recommendation training (MTIA 300), combined generative AI and ranking workloads (MTIA 400), and decode-optimized generative AI inference (MTIA 450 and 500), with mass deployment of the flagship MTIA 500 planned for late 2027. Meta plans to put its own AI chip into production in September and is aiming to roughly double the computing capacity across its data centres.
By mid-2026, Meta, Amazon, Microsoft, and OpenAI have each closed the gap on the three key AI inputs — custom chips, power, and models — that only Google held in 2021.
Microsoft: Maia 200 and the Push to External Customers:
Microsoft unveiled its first custom AI accelerator, Maia 100, at Hot Chips 2024, followed by the inference-optimized Maia 200 in January 2026. Maia 200, built on TSMC’s 3 nm process with more than 140 billion transistors, 216 GB of HBM3e, and over 10 PFLOPS of FP4 compute within a 750 W SoC TDP, is designed to deliver 30% better performance per dollar for AI token generation.blogs.
Maia 200 will serve multiple models, including OpenAI’s GPT-5.2, and support Microsoft Foundry, Microsoft 365 Copilot, and reinforcement learning workflows. Microsoft plans to unveil next-gen Maia 300 AI chip in September, aiming to lower costs for in-house and OpenAI models while actively courting major enterprise customers. Anthropic is reportedly in talks with Microsoft to rent the company’s custom AI server chips as it looks to expand computing capacity.blogs.
Oracle: Partner-Led AI Clusters Rather Than Custom Silicon:
Oracle has taken a different path, opting not to develop its own AI accelerator but instead building large-scale AI clusters using third-party chips from Nvidia and AMD. Oracle plans to install the first MI450-equipped Helios racks in its OCI data centers during the third quarter of 2026, with an initial deployment targeting 50,000 MI450 processors.
Oracle’s AI strategy emphasizes rapid deployment of massive GPU-based clusters to serve anchor tenants like OpenAI under a reported $300 billion, five-year cloud computing contract beginning in 2027. In parallel, OpenAI is diversifying its supply of compute by designing its own chips with partners like Broadcom, with the first custom AI inference chips expected to deploy in the second half of 2026.
Market Implications:
By 2026, five of six major AI players — Google, Meta, Amazon, Microsoft, and OpenAI — now control at least two of the three critical AI inputs (chips, power, models), down from only Google in 2021. This vertical integration trend is reshaping the AI infrastructure market, with hyperscalers increasingly using custom silicon to optimize cost and performance for specific workloads while maintaining strategic flexibility through multi-vendor GPU procurement.
Google’s TPU v7, Amazon’s Trainium 3, Microsoft’s Maia 2, and Meta’s MTIA 2 all ramped into volume production in 2025–2026, signaling a maturation of the hyperscaler custom silicon ecosystem. Meta is also the first commercial gigawatt AMD MI450 deployment in H2 2026, illustrating a hybrid approach that combines in-house accelerators with third-party GPUs.
…………………………………………………………………………………………………………………………………………………..
References:
https://www.investors.com/news/technology/google-stock-cloud-kurian-ai-chip-business/
Will Google Cloud’s AI and data analytics revenue +TPU IP licensing income offset huge AI CAPEX to produce a decent ROI?
Google announces Gemini: it’s most powerful AI model, powered by TPU chips
Google Cloud and Verizon Expand Strategic Partnership to Scale Enterprise AI and Autonomous Network Operations
AWS to deploy AI inference chips from Cerebras in its data centers; Anapurna Labs/Amazon in-house AI silicon products
Meta’s “Iris” AI Chip for MTIA: Implications for Telecom-Grade Optical Networking, DCI and High Capacity Ethernet Fabrics
Custom AI Chips: Powering the next wave of Intelligent Computing
AI Compute Has a Switchboard Problem: Orchestration & Data Center Fabric Explained
South Korean startup Rebellions to use open source software for carriers to quickly build AI stacks with its AI inferencing chips
Analysis: Nvidia–MediaTek Deepen AI Platform Collaboration -What it Means for AI RAN & DCI
Executive Summary:
Nvidia has expanded its partnership with Taiwan based semiconductor firm MediaTek, committing $3.5 billion to a bond investment the two companies describe as a deepening of their “longstanding collaboration to build the next generations of AI computing platforms.” The move extends Nvidia’s pattern of strategic capital deployment across its compute ecosystem, this time reaching into custom silicon and advanced packaging.
According to the announcement, the collaboration couples Nvidia’s accelerated computing, AI, graphics, and software stack with MediaTek’s capabilities in custom silicon, high-performance computing (HPC), system-on-chip (SoC) design, and advanced packaging. MediaTek vice chairman and CEO Rick Tsai framed the investment as reinforcing a relationship spanning “cloud AI infrastructure, local AI computing and automotive in the era of physical AI,” combining Nvidia’s accelerated-computing and AI software ecosystem with MediaTek’s AI technology portfolio and custom-silicon resources.
Over the past several years, MediaTek has repositioned itself from a supplier of predominantly mid- and low-tier handset chipsets into a presence across smartphones, AI data centers, automotive, and IoT. Securities firm Yuanta Financial Holdings expects the closer alignment to accelerate MediaTek’s transition from edge AI into data-center infrastructure, device-based AI computing, and automotive platforms. Trading in MediaTek shares was suspended on the Taipei exchange Tuesday after the stock hit its daily 10% upside limit. The shares closed at NT$4,315 (about US$136.05), reflecting investor enthusiasm for the deepening of the collaboration.
NVIDIA and MediaTek are collaborating in three major areas:
- AI infrastructure: MediaTek will work with NVIDIA’s NVLink Fusion ecosystem to enable customers to develop custom AI infrastructure designed to integrate with NVIDIA rack-scale systems and AI factories.
- Local AI computing: The companies will continue to collaborate on multiple generations of NVIDIA RTX Spark™ and DGX Spark™ PC chips, powering consumer PCs, AI developer supercomputers and enterprise-class workstations, that integrate NVIDIA GPUs with MediaTek SoCs.
- Automotive: MediaTek and NVIDIA will continue developing platforms for AI-powered, software-defined vehicles in the era of physical AI.
“AI is transforming every computing platform — from the world’s largest AI factories to the PC and the car,” said Jensen Huang, founder and CEO of NVIDIA. “MediaTek is one of the world’s great semiconductor companies, with exceptional expertise in system-on-chip design, connectivity, leading performance and power efficiency. Together, we’re building platforms that bring NVIDIA accelerated computing to new markets and give customers the freedom to create differentiated AI systems at enormous scale.”
“MediaTek and NVIDIA share a vision for making advanced AI computing pervasive across the technology landscape,” said Rick Tsai, vice chairman and CEO of MediaTek. “NVIDIA’s investment strengthens a collaboration that spans cloud AI infrastructure, local AI computing and automotive in the era of physical AI. By combining NVIDIA’s leadership in accelerated computing and AI software ecosystem with MediaTek’s expertise in a diverse AI technology portfolio from edge to cloud, and our leadership position in custom silicon, we can accelerate innovation for our customers.”
Custom AI Infrastructure and NVLink Fusion:
Yuanta identified MediaTek’s participation in Nvidia’s NVLink Fusion as the most consequential element of the arrangement. Through that linkage, MediaTek could support the design of custom AI chips for clients while connecting them directly into Nvidia’s AI infrastructure — a complementary rather than competing role alongside the GPU vendor’s proprietary interconnect fabric.
The scale of the bond purchase also signals Nvidia’s long-term commitment to the relationship. Nvidia, currently the world’s most valuable company at roughly a $5 trillion market capitalization, has been deploying capital aggressively across both upstream and downstream partners. This latest deal, however, inevitably risks overlapping with other Nvidia investments — notably its $2 billion partnership with Marvell, which is also heavily oriented around NVLink Fusion and pitch-for the RAN with a one-chip architecture.
NVLink Fusion vs. Ethernet Scale-Out Interconnect:
NVLink Fusion and Ethernet-based scale-out interconnects target fundamentally different layers of the AI fabric, and the distinction turns on memory semantics rather than raw link rate. NVLink Fusion sits in the scale-up domain: it delivers memory-coherent, tightly coupled attachment for custom XPUs into Nvidia’s rack-scale NVLink fabric, with point-to-point GPU/XPU-to-GPU/XPU connectivity and an integrated high-bandwidth-memory interface (NVHBM) that trades die area and power for latency and bandwidth. It is proprietary, closed, and full-coherency by design — precisely the properties that make it hard for open standards to clone. Ethernet scale-out, by contrast, is the scale-out workhorse governing traffic between racks, pods, and clusters; it is loss-based, coarser-grained, and standardized through Ultra Ethernet and the broader IEEE 802.3 ecosystem.
The real contest is not NVLink against Ethernet so much as how much of the memory-coherent scale-up domain Ethernet-based challengers can capture. Standards-backed alternatives — UALink 2.0, Ultra Ethernet, and Broadcom’s Scale-Up Ethernet (SUE) — are pressing into scale-up territory, but they lack NVLink’s memory coherency and direct XPU-to-XPU connectivity, and SUE explicitly does not wire GPUs directly to GPUs. Meanwhile Nvidia’s strategy with NVLink Fusion is to co-opt custom XPUs (AWS Trainium4, and now MediaTek- and Marvell-designed silicon) into its fabric rather than let them migrate to those open scale-up paths. The practical outcome in the data center is a layered model — Ethernet surging in scale-out, and NVLink Fusion consolidating the latency-sensitive, memory-coherent scale-up spine — with Nvidia working to keep its proprietary layer the mandatory gateway that custom accelerators must pass through to reach the wider Ethernet-based world.
The $3.5 billion convertible-bond investment is best read not as a portfolio decision but as a defensive move within the AI rack-scale interconnect stack. The technical centerpiece is MediaTek’s adoption of the NVLink Fusion platform. Nvidia positions it as the connectivity technology and intellectual property that lets hyperscalers and AI-native operators drop custom accelerators (“XPUs”) and CPUs directly into its NVLink scale-up fabric.
MediaTek will now build against the three subsystems that define NVLink Fusion: the NVLink Fusion chiplet, which bridges a customer’s custom XPU to the NVLink scale-up domain over photonic or electrical interconnects (the chiplet itself carries up to ~1.8 TB/s of bidirectional bandwidth); NVLink-C2C, the high-bandwidth die-to-die link between XPUs, Nvidia’s Vera/Rosa CPUs, and other compatible processors; and NVHBM, a customized high-bandwidth-memory interface that trades off die area and power efficiency. The strategic significance is that Nvidia is effectively licensing its interconnect and memory layer as a licensable subsystem rather than enforcing a closed, all-Nvidia island — which lets it absorb “frenemy” silicon into the fabric instead of fighting it.
Two implications stand out for interconnect strategy:
-
A hedge on hyperscaler in-house silicon. AWS has already integrated its Trainium4 custom ASICs with NVLink Fusion for rack-scale deployment, and Google, Meta, Microsoft, and OpenAI are all advancing private accelerators. By standardizing the interface (NVLink Fusion, NVLink-C2C, NVHBM, chiplets, and rack-scale integration) that these custom XPUs must speak, Nvidia keeps chiplets, packages, memory, and racks—the connective infrastructure itself—inside its own revenue funnel. MediaTek, in turn, becomes the design conduit that helps hyperscalers and frontier model firms get to that tape-out without building an independent scale-up stack.
-
The scale-up vs. scale-out contest sharpens. NVLink remains essentially the only mature, memory-coherent scale-up fabric in production, whereas Ethernet is consolidating its dominance in scale-out (front-end and increasingly AI back-end) fabrics, and standards like Ultra Ethernet, UALink, and Broadcom’s Scale-Up Ethernet (SUE) are pressing into the scale-up domain. UALink and SUE, notably, lack NVLink’s memory coherency and point-to-point GPU-to-GPU connectivity, so NVLink Fusion’s positioning — co-opting custom XPUs rather than competing with them — lowers the attack surface those open standards could otherwise exploit.
The buy-in therefore consolidates Nvidia’s grip on the two layers that are hardest to commoditize: the latency-sensitive scale-up interconnect and the custom-silicon design flow that feeds it. What operators and chip partners give up is architectural independence — the bond structure and NVLink Fusion licensing effectively bind MediaTek into Nvidia’s ecosystem rather than allowing it to become an independent AI data-center silicon house.
What It Means for AI-RAN:
For the RAN, this is arguably a more consequential signal than the data-center economics. Nvidia’s AI-RAN play, via the AI Aerial software-defined accelerated-computing platform and its founding role in the AI-RAN Alliance, is built on having vRAN and AI workloads share the same GPU-accelerated infrastructure — what the community variously calls “AI for RAN,” “AI on RAN,” and “AI-and-RAN” on common hardware.nvidia+1
Here the MediaTek deal intersects directly with Nvidia’s earlier $2 billion Marvell partnership, which last year wired Marvell into the same NVLink Fusion platform for both the AI factory and AI-RAN ecosystem — Marvell supplying custom XPUs and NVLink Fusion-compatible scale-up networking, while Nvidia contributes the Vera CPU, ConnectX NICs, BlueField DPUs, Spectrum-X switches, and the rack-scale AI compute. The two arrangements are complementary in practice but overlapping in intent: Marvell owns the scale-up networking and the “one-chip-to-rule-the-RAN” custom silicon angle, while MediaTek brings SoC design, advanced packaging, and stronger reach into edge, automotive, and device-grade silicon. Together they give Nvidia two design houses capable of producing custom XPUs that plug into both data-center racks and AI-RAN infrastructure.
The operative question for IEEE readers is whether this broadens the AI-RAN supplier base going forward. Ericsson and Samsung have indicated their vRAN software built for Intel could be ported to AMD or Arm silicon with relatively modest effort, and Dell’Oro projections already have AI-RAN taking a significant share of the broader RAN market. Nvidia’s strategy is to ensure that, however the RAN silicon shakes out, the scale-up fabric and the custom-XPU design path remain NVLink Fusion-based — extending its data-center interconnect franchise into the radio access domain itself. The risk, flagged in the news report, is over-concentration: stacking Marvell and MediaTek onto the same NVLink Fusion foundation concentrates architectural leverage in Nvidia’s hands and could crowd out the open, multi-vendor fabric choices that O-RAN advocates would prefer to see served by standard Ethernet and UALink-style paths.
References:
https://www.lightreading.com/finance/nvidia-tips-3-5b-into-mediatek-in-latest-tech-partnership
Analysis: Nvidia’s $2 billion investment in Marvell; NVLink Fusion ecosystem & RAN vendor silicon strategy
Ericsson and MediaTek Demonstrate 3GPP-Based GNSS RTK Positioning with Sub-30cm Accuracy Over a Commercial 5G Network
5G infrastructure moves from coverage and speeds to cloud-native, orchestration, automation and AI-assisted networks
MediaTek overtakes Qualcomm in 5G smartphone chip market
South Korean startup Rebellions to use open source software for carriers to quickly build AI stacks with its AI inferencing chips
South Korean chip startup Rebellions builds purpose-engineered Al accelerators to redefine energy-efficiency and scale in the age of large-scale Al. It aims to deliver the best performance per dollar per watt possible for inferencing and lower both capex and opex associated with running AI infrastructure. The company has been very selective about where it has established office locations and staffing: Korea, Japan, Singapore, Saudi Arabia and the United States.
Their flagship semiconductor product is the Rebel100, which uses a predictive, software-controlled DMA engine tightly coupled with an on-chip mesh to prefetch KV data proactively. This enbales 2.7TB/s effective bandwidth and reduces token-level latency in 32K+ context LLMs.
Image Credit: Rebellions
Gaining valuable experience in building an AI stack with SK Telecom, it’s pairing its field experience with an open source-first software strategy it said can help carriers move faster without locking scarce engineering talent into yet another proprietary AI stack. The company says it’s dedicated to using open-source software – everything from vLLM and OpenShift to PyTorch – with a “no forks” rule. The rule is designed to help customers avoid skills issues and ensure engineers don’t have to learn non-transferrable skills just to use its equipment.
Rebellions’ work with SK Telecom (SKT) has served as a foundational pillar of its engagement with other operators, according to Marshall Choy, Rebellions’ Chief Business Officer (CBO). The company is in conversations with between 10 and 20 operators around the world.
“Our systems have been deployed at SKT for nearly three years,” Choy said. “What does that mean? It means three years of lessons learned, institutional knowledge gain, product improvement and deep engagement with an end user customer working on real problems… It’s three years of blood sweat and tears that has become institutional knowledge,” he added.
The network operators in its pipeline are in “different stages of engagement and deployment” with Rebellions, Choy said, saying there will be “more to come on that.” While SK Telecom has been its most publicized partner to date, Choy said over the next 12 months Rebellions plans to highlight more of what has been going on under cover. That includes work with other telecom operators as well as neocloud operators, enterprises and governments.
According to Choy, the architecture prioritizes low-latency, high-throughput compute infrastructure. From a cost-efficiency perspective, the company targets a price-to-performance ratio that is two to three times more cost-effective than comparable Nvidia hardware. Furthermore, Rebellions delivers significantly higher energy efficiency. While an Nvidia DGX GB200 NVL72 system consumes upwards of 120 kilowatts per rack, Rebellions averages 4 kilowatts per system, or approximately 20 kilowatts per rack.
Choy noted that this power reduction yields a 6x savings in operational expenditure (OpEx). It also enables telecommunications providers to deploy Rebellions infrastructure in edge environments with stringent power constraints, such as legacy central offices. By retrofitting these distributed facilities with modern inference compute to serve LLM tokens, operators can maximize the lifecycle and return on investment (ROI) of their existing physical assets.
“Our goal is to reduce the unit economics of AI inferencing to near zero,” Choy said. While he admitted that sounds strange coming from an AI inferencing chip company, Choy explained that it’s all about making AI accessible for even more use cases – the ones where the math doesn’t work out today. “If you’re a telephone operator and you have an existing line card of services you provide, I can make you more profitable because I can lower your costs. But more strategically, what I can do is I can enable you to introduce a lower tier of services at a lower cost, which then makes AI inferencing accessible to a whole set of applications where it was previously too expensive,” he added.
It should be noted that Groq and SambaNova (see References below) are similarly working on chips to reduce the cost of inferencing. OpenAI appears to be moving in a similar direction with its Jalapeño chip.
Through its work with SK Telecom and others (unnamed), Choy said Rebellions has learned that customers aren’t just looking for hardware or software but for fully optimized infrastructure that cuts across both. That’s where Rebellions’ open-source ethos comes into play. “We didn’t want to have this mainframe model where everything is custom and bespoke and we’re this weird thing off in the side of the data center that doesn’t get touched by anything else,” Choy said. “It’s all about interoperability and integration.”
In conclusion, Choy opined, “Let’s be honest, the telcos don’t necessarily have all the right skills in place. So, being able to spread that across more of an open-source ecosystem means they can be in service and productive faster.”
References:
https://rebellions.ai/company/about/
https://rebellions.ai/category/blogs/
https://rebellions.ai/rebellions-product/rebel100/
SambaNova targets AI inference boom with chips built for existing data centers
Intel and AI chip startup SambaNova partner; SN50 AI inferencing chip max speed said to be 5X faster than competitive AI chips
AWS to deploy AI inference chips from Cerebras in its data centers; Anapurna Labs/Amazon in-house AI silicon products
Custom AI Chips: Powering the next wave of Intelligent Computing
Huawei to Double Output of Ascend AI chips in 2026; OpenAI orders HBM chips from SK Hynix & Samsung for Stargate UAE project
Huawei to Double Output of Ascend AI chips in 2026; OpenAI orders HBM chips from SK Hynix & Samsung for Stargate UAE project
Huawei to Double Output of Ascend AI chips in 2026; OpenAI orders HBM chips from SK Hynix & Samsung for Stargate UAE project
Superclusters of Nvidia GPU/AI chips combined with end-to-end network platforms to create next generation data centers
