Data Center Networking Market
Inside Amazon’s new data center network architecture: quasi random network topology and passive optical devices
Amazon Web Services (AWS) claims it recently achieved a major breakthrough in Data Center Network (DCN) architecture and has been quietly deploying the new technology in its data centers since late last year. Amazon detailed its new networking design in a paper published May 21st titled “RNG: Flat Data Center Networks at Scale.” RNG, or “resilient network graphs,” is built around a quasi-random topology and new passive optical hardware. It’s a “quasi-random” design that combines elements of traditional, structured data networks with the performance advantages of more random architectures.
The goal is to move off conventional hierarchical “fat-tree” designs toward a flatter, more mesh-like fabric that uses far fewer routers and switches, offers more parallel paths, and therefore delivers higher effective throughput at lower power and capex.
“By essentially flattening the network, we eliminated the bottlenecks that come with traditional networking designs,” Matt Rehder, vice president of AWS Network Engineering, said in an exclusive interview with WIRED. “We think we’re the only ones who have done this at scale. RNG is a great fit for our core demands, but AI training data patterns are far more coordinated and centrally orchestrated, so they don’t approximate a random graph.”
The fact that Amazon is using this in the real world is “remarkable,” said Brighten Godfrey, a computer science professor at the University of Illinois Urbana-Champaign and an expert in networking, who was not involved in Amazon’s research. Godfrey coauthored a seminal 2012 paper on random network graphs, which he says are a “mind-bending problem to solve, in general.”
Classic cloud DCNs use structured topologies (Clos/fat-tree) where paths are highly regular and layered (Top of Rack (ToR)–aggregation–core). By contrast, random-graph theory says the most efficient routing networks are flat random graphs: each node connects to a small random subset of others, creating many short, diverse paths and graceful degradation under failures. The problem has always been practical: random cabling at scale is unmanageable, and routing across a huge random graph is nontrivial.
AWS’s “quasi-random” design essentially mixes determinism with randomness: key structural elements are fixed to keep the cabling and deployment manageable, while enough randomness is retained in the interconnect pattern to get the performance and resilience benefits of random graphs. The physical enabler is a new passive optical device called a ShuffleBox that standardizes how switches connect and internally permutes links so that, when many ShuffleBoxes are wired together, the resulting global topology is quasi-random without having to hand-design every link.
Image Credit: Amazon
………………………………………………………………………………………………………………………….
Key architectural pieces and claimed gains:
AWS reports that RNG-based fabrics now serve as the default network architecture for most new AWS data centers, after initial deployments beginning in 2024. The company claims the design:
-
Uses roughly 69% fewer routers/switches than traditional fat-tree DCNs, because the network is flatter and relies more on passive optical fanout.
-
Delivers up to about 33% higher throughput, due to more independent paths and better load spreading.
-
Cuts network equipment power consumption by on the order of 40%, with associated reductions in cooling and operational overhead.
On the control-plane side, AWS developed a routing scheme called Spraypoint. Instead of always following a strict shortest path from source to destination, Spraypoint first “sprays” traffic randomly to neighbors, then directs it via preselected “waypoints” using more conventional shortest-path routing. This hybrid behavior exploits the quasi-random topology to open many more independent paths than standard ECMP-style shortest-path routing would, which in turn improves utilization and resilience under congestion or failures.
Strategic implications:
For AWS’s cloud and AI build-out, this is positioned as a foundational infrastructure advantage: higher bisection bandwidth and lower network energy per bit directly benefit large-scale AI training clusters, storage backends, and multi-tenant cloud workloads. Fewer active devices and more passive optics also translate into lower capex and opex at hyperscale, so AWS is framing this as both a performance and cost/sustainability play that could save billions of dollars and reduce CO₂ emissions over time.
From a networking-theory standpoint, this is notable as one of the first reported at-scale, production deployments of a flat random-graph-inspired topology in a hyperscale DCN, rather than a purely academic or lab system.
In a quasi-random topology like AWS’s RNG fabric, the impact on latency and jitter comes from three main effects: path length distribution, load spreading, and failure behavior.
Baseline latency: path lengths and device count:
In a traditional Clos/fat-tree, average latency is dominated by a fixed number of stages (ToR → agg → core → agg → ToR), so hop count is tightly controlled but you pay for many active devices. A quasi-random, flat graph replaces that rigid hierarchy with many short, irregular paths; on average, shortest paths between any two switches are similar or slightly shorter in hop count than in a fat-tree, and there are fewer active routers in the path because the architecture offloads fanout to passive optics. That tends to keep or slightly reduce median/mean latency per flow, especially under moderate load, because packets traverse fewer serialized queueing points even if the physical graph looks “messier.”
Jitter: congestion and path diversity:
Jitter is driven much more by variable queueing delay than by fixed propagation or serialization. In a quasi-random fabric with many alternate paths and a load-balancing scheme like Spraypoint (random spray + waypoint-based shortest paths), flows can be spread more evenly across the network, reducing hot spots and thus reducing the variance of queueing delay across packets. That can lower jitter compared with a Clos under the same aggregate load, because the system is less likely to funnel many flows through the same few congested uplinks or spine devices.
However, because the routing intentionally uses many different paths, per-flow packet reordering becomes more likely unless constrained by per-flow hashing or waypointing, which can show up as effective jitter at higher layers. AWS’s description of Spraypoint suggests they mitigate this by using waypoints and policy to preserve some path structure, so you get the diversity benefits without unconstrained per-packet spraying.
Under failure and high load:
Where quasi-random really helps latency/jitter is under failure and partial congestion. In a Clos, link or spine failures can force large sets of flows to converge on a smaller subset of remaining equal-cost paths, driving up queueing delay and jitter nonlinearly. In a resilient random-graph-style fabric, node/edge failures simply remove a few edges from a highly connected graph; there are typically many alternative short paths, so the increase in hop count and queueing pressure is smaller and more diffuse. That tends to keep tail latency and jitter (P99, P99.9) better behaved, even if median latency looks similar to a Clos at low load.
So, qualitatively: median latency is roughly comparable to a well-designed Clos, sometimes better due to fewer active stages; jitter and tail latency should improve under realistic, bursty load and failure scenarios, provided the routing stack is designed to limit packet reordering.
Summary and Conclusions:
Quasi-random data center topologies like AWS’s RNG fabric replace rigid Clos/fat-tree hierarchies with a flatter, graph-like network that preserves short path lengths while dramatically increasing path diversity, which tends to hold median latency roughly steady or slightly better by reducing the number of active, queueing devices per path and offloading fanout to passive optics. They primarily improve jitter and tail latency by spreading flows across many alternative routes so congestion is less concentrated, making queueing delays less bursty and keeping P99/P99.9 behavior more stable under failures and hot spots, provided the routing layer (for example, AWS’s Spraypoint approach) constrains packet reordering through way pointing or per-flow consistency.
In conclusion, quasi-random fabrics are less about shaving a few microseconds off baseline latency and more about delivering more predictable end-to-end performance—especially for east–west, latency-sensitive cloud and AI workloads—by trading rigid structure for statistically robust, highly connected graphs that degrade more gracefully when links, nodes, or traffic patterns become pathological.
…………………………………………………………………………………………………………………………………………………………………….
References:
https://arxiv.org/pdf/2604.15261
https://www.wired.com/story/amazon-aws-ceo-matt-garman-ai-agents/
AWS to deploy AI inference chips from Cerebras in its data centers; Anapurna Labs/Amazon in-house AI silicon products
Amazon’s Jeff Bezos at Italian Tech Week: “AI is a kind of industrial bubble”
Data Center Networking Market to grow at a CAGR of 6.22% during 2022-2027 to reach $35.6 billion by 2027
TMR: Data Center Networking Market sees shift to user-centric & data-oriented business + CoreSite DC Tour
Big Fiber’s $250M financing deal to buildout dark fiber routes for AI Data Center expansion
Executive Summary:
Big Fiber [1.] has secured $250 million in financing from Stonepeak and Caisse de dépôt et placement du Québec (CDPQ) to expand its dark fiber footprint and increase network capacity in response to accelerating hyperscaler and large-scale data center investments in AI-driven workloads.
Note 1. Sunnyvale, CA headquartered Big Fiber was previously known as Bandwidth IG, which was originally established in 2019 as a telecom and dark-fiber infrastructure company. The rebrand to BIG Fiber was announced on May 1, 2025 when the company described it as a shift to better reflect its focus on privately owned, newly constructed dark fiber networks. The company has built privately owned metro dark fiber networks from its inception, primarily in the SF Bay Area and the Greater Portland, OR and Atlanta, GA areas.
BIG Fiber structures its dark fiber portfolio around high‑strand‑count, single‑mode, low‑loss fiber deployed in purpose‑built, underground metro and regional routes, rather than a carrier‑specific “technology” stack of its own. The company’s public materials emphasize:
-
Single‑mode fiber (SMF) for metro and long‑haul connectivity, consistent with standard dark‑fiber infrastructure designed for multi‑wavelength and DWDM‑based upgrades.
-
High‑density, high‑fiber‑count cables in metro corridors (often hundreds of strands) to support dense data‑center and interconnect demand, which is typical of “new‑build” dark‑fiber operators entering AI‑and‑cloud‑centric markets.
-
Point‑to‑point and ring‑style topologies engineered for extreme route diversity (tri‑/quad‑versity) and low latency, rather than a legacy long‑haul backbone that relies on older fiber types or managed wavelengths.
To complement Big Fiber’s dark‑fiber infrastructure; the customer provides the optical PHY layer (e.g., coherent DWDM, 400ZR/ZR+, or other high‑speed optics), which is how dark‑fiber providers typically position their offerings.
–>More about Big Fiber at the end of this article from the company itself.
……………………………………………………………………………………………………………………………………………………………………………………………………………………………………….
Proceeds of the facility will be used to refinance existing debt, provide new capital and facilitate the necessary headroom for major fiber optic network expansions already underway. This includes a significant multi-market buildout in Greater Atlanta, adding over 205 route miles and 165,000 fiber miles to BIG Fiber’s existing market-leading footprint.
“Our partnership with Stonepeak Credit and La Caisse marks a pivotal moment in our mission to empower our customers with highly scalable and purpose-built dark fiber solutions,” said Bruce Garrison, CEO of BIG Fiber. “This financing ensures we have the scale to stay ahead of the escalating demand for modernized infrastructure enabling the AI ecosystem and the necessary digital highways for decades to come.”
“BIG Fiber’s infrastructure delivers critical bandwidth to meet the insatiable demand for both data and compute capacity across its key markets,” said Arun Varanasi, Managing Director at Stonepeak Credit. “We are proud to partner with Columbia Capital, SDC Capital Partners, and La Caisse to support the company’s next leg of growth as it positions itself as one of the preeminent dark fiber operators in the country.”
“BIG Fiber is well positioned to meet the growing connectivity needs of enterprises and data centers seeking new, high-quality infrastructure options,” said Jérôme Marquis, Managing Director and Head of Private Credit at La Caisse. “Its resilient business model, underpinned by long-term contracts and strong structural demand, positions the company well for growth. Together with Stonepeak Credit, we’re providing a tailored financing solution that supports the continued buildout of essential digital infrastructure.”
The latest expansion will bring BIG Fiber’s Atlanta and San Francisco Bay Area network capacity to 850 route miles and over 3 million fiber miles. Projects are currently under construction or contract, with phased Ready for Service (RFS) dates expected in early 2027.
………………………………………………………………………………………………………………………………………………………………………………………………………………………………
According to Big Fiber Chief Commercial Officer Patton Lochridge, demand signals are particularly strong in key U.S. metros including the San Francisco Bay Area, Hillsboro, and Atlanta, where new fiber routes are being deployed to support AI-centric data center expansion. “We’re seeing customers require extreme route diversity, often moving toward triversity or quadversity networks to connect metro assets and long-haul routes,” Lochridge said. He added that inference workloads are increasing the demand for dense metro connectivity: “Traditional telecommunications networks are often too congested or lack the latency and loss tolerances required for stringent AI workloads, making purpose-built metro fiber essential.” Lochridge indicated that the majority of the new capital will be directed toward greenfield build-outs and targeted overbuilds of “exhausted legacy telecommunications corridors that need more scale.”
Industry analysts highlight a parallel geographic shift in AI infrastructure deployment. Sterling Perrin, senior principal analyst for optical networks and transport at Omdia, noted that AI campuses are expanding beyond traditional connectivity hubs such as Ashburn, Dallas, and Northern California into power-advantaged regions including West Texas, Ohio, Tennessee, Louisiana, and Georgia. “They all require massive fiber optic connectivity,” Perrin said.
Power availability is emerging as a primary constraint shaping network topology. Ron Westfall, vice president and analyst at HyperFrame Research, emphasized that grid limitations are driving hyperscalers toward distributed AI campus architectures interconnected via metro and long-haul dark fiber. “Power grid constraints have forced a material shift toward metro and long-haul dark fiber infrastructure to stitch together distributed regional data center campuses,” Westfall said. “Because this relentless GPU-to-GPU communication demands near-zero latency and unprecedented bandwidth, infrastructure planners are prioritizing the deployment of ultra-high-strand dark fiber corridors that directly link distributed, power-rich data centers.”
AI Workloads Reshape Optical Demand:
AI-driven traffic growth is now materially impacting the optical supply chain. In its April 2026 post-OFC analysis, CRU Group reported that AI-related data center demand “has overtaken traditional telecom as the primary growth engine for optical [fiber] and cable,” contributing to tightening supply conditions for high-fiber-count cables and upstream preform materials.
Despite this surge, the majority of AI traffic remains intra-data-center. Omdia estimates indicate that up to 90% of AI traffic does not exit the facility during GPU cluster operations. However, the emergence of distributed AI architectures is beginning to increase requirements for high-capacity inter-data-center interconnect (DCI).
At the Optica Executive Forum, Cisco SVP and Fellow Rakesh Chopra highlighted the scale differential between AI and conventional traffic profiles. As cited by Perrin, AI “scale-up” traffic within data centers can generate 504 times more traffic than traditional DCI flows, while “scale-out” traffic can produce 56 times DCI bandwidth requirements. “With AI training models at the limits of what can be processed within a data center, distributed AI clusters are inevitable,” Perrin said.
This architectural transition is reflected in NVIDIA’s AI factory designs, which decouple east-west GPU compute traffic from traditional north-south enterprise flows, leveraging low-latency leaf-spine topologies optimized for continuous GPU synchronization.
Westfall further noted that these evolving traffic patterns are fundamentally altering network design assumptions. Operators are increasingly optimizing for persistent machine-to-machine synchronization rather than burst-oriented enterprise traffic models.
Fiber as a Core AI Infrastructure Asset:
The Big Fiber’s latest financing aligns with broader trends in AI infrastructure investment, where capital is being deployed across integrated stacks including energy, land, connectivity, and compute infrastructure. Utilities are expanding transmission capacity, while developers are co-locating generation resources near emerging AI hubs.
Within this context, fiber infrastructure is being revalued based on its strategic proximity to power-rich data center clusters. “Infrastructure monetization is shifting away from historical metrics such as per-megabit pricing toward asset-level valuations built around proximity to power-rich data centers,” Westfall said.
If current deployment trajectories persist, the resulting topology will consist of a dense, high-capacity mesh of metro and long-haul fiber routes interconnecting geographically distributed, power-optimized AI campuses with hyperscale cloud and interconnection ecosystems.
………………………………………………………………………………………………………………………………
About BIG Fiber:
BIG Fiber is a metro dark fiber provider that offers high capacity, strategically placed, dark fiber networks to mission critical data centers, Hyperscalers and enterprises throughout the San Francisco Bay Area, Greater Portland and Greater Atlanta areas. BIG Fiber’s 100% underground network meets critical data needs for enterprises and data centers that require new, quality infrastructure options. BIG Fiber’s San Francisco Bay Area network offers more than 320 route miles and 65 data centers. The Greater Portland network has more than 20 route miles and 15 data centers, and the Greater Atlanta network has more than 550 route miles and 30 data centers. BIG Fiber was founded in 2019 and is headquartered in Sunnyvale, California. Visit www.bigfiber.com to learn more.
………………………………………………………………………………………………………………………………
References:
BIG Fiber Secures $250 Million Financing Led by Stonepeak Credit and La Caisse
Analysis: Fiber Broadband Association (FBA) whitepaper: Upgrading MSO Networks to Fiber to the Home (FTTH): A Technical Perspective
Fiber Broadband Association Middle Mile WG: how to use “Digital Infrastructure Networks” for coordinated fiber backbone investments
Analysis: AT&T 1Q-2026 results: increased fiber penetration, FWA momentum, D2D deals, and mobile/home internet bundles
Fiber Optic Boost: Corning and Meta in multiyear $6 billion deal to accelerate U.S data center buildout
Fiber Optic Networks & Subsea Cable Systems as the foundation for AI and Cloud services
How will fiber and equipment vendors meet the increased demand for fiber optics in 2026 due to AI data center buildouts?
Automating Fiber Testing in the Last Mile: An Experiment from the Field
AI wireless and fiber optic network technologies; IMT 2030 “native AI” concept
Big tech spending on AI data centers and infrastructure vs the fiber optic buildout during the dot-com boom (& bust)
Big Tech plans to spend between $364 billion and $400 billion on AI data centers, purchasing specialized AI hardware like GPUs, and supporting cloud computing/storage capacity. The final 2Q 2025 GDP report, released last week, reveals a surge in data center infrastructure spending from $9.5 billion in early 2020 to $40.4 billion in the second quarter of 2025. It’s largely due to an unprecedented investment boom driven by artificial intelligence (AI) and cloud computing. The increase highlights a monumental shift in capital expenditure by major tech companies.
Yet there are huge uncertainties about how far AI will transform scientific discovery and hypercharge technological advance. Tech financial analysts worry that enthusiasm for AI has turned into a bubble that is reminiscent of the mania around the internet’s infrastructure build-out boom from 1998-2000. During that time period, telecom network providers spent over $100 billion blanketing the country with fiber optic cables based on the belief that the internet’s growth would be so explosive that such massive investments were justified. The “talk of the town” during those years was the “All Optical Network,” with ultra-long haul optical transceiver, photonic switches and optical add/drop multiplexers. 27 years later, it still has not been realized anywhere in the world.
The resulting massive optical network overbuilding made telecom the hardest hit sector of the dot-com bust. Industry giants toppled like dominoes, including Global Crossing, WorldCom, Enron, Qwest, PSI Net and 360Networks.
However, a key difference between then and now is that today’s tech giants (e.g. hyperscalers) produce far more cash than the fiber builders in the 1990s. Also, AI is immediately available for use by anyone that has a high speed internet connection (via desktop, laptop, tablet or smartphone) unlike the late 1990s when internet users (consumers and businesses) had to obtain high-speed wireline access via cable modems, DSL or (in very few areas) fiber to the premises.
……………………………………………………………………………………………………………………………………………………………………………………………………………………………….
Today, the once boring world of chips and data centers has become a raging multi-hundred billion dollar battleground where Silicon Valley giants attempt to one up each other with spending commitments—and sci-fi names. Meta CEO Mark Zuckerberg teased his planned “Hyperion” mega-data center with a social-media post showing it would be the size of a large chunk of Manhattan.
OpenAI’s Sam Altman calls his data-center effort “Stargate,” a reference to the 1994 movie about an interstellar time-travel portal. Company executives this week laid out plans that would require at least $1 trillion in data-center investments, and Altman recently committed the company to pay Oracle an average of approximately $60 billion a year for AI compute servers in data centers in coming years. That’s despite Oracle is not a major cloud service provider and OpenAI will not have the cash on hand to pay Oracle.
In fact, OpenAI is on track to realize just $13 billion in revenue from all its paying customers this year and won’t be profitable till at least 2029 or 2030. The company projects its total cash burn will reach $115 billion by 2029. The majority of its revenue comes from subscriptions to premium versions of ChatGPT, with the remainder from selling access to its models via its API. Although ~ 700 million people—9% of the world’s population—are weekly users of ChatGPT (as of August, up from 500 million in March), its estimated that over 90% use the free version. Also this past week:
- Nvidia plans to invest up to $100 billion to help OpenAI build data center capacity with millions GPUs.
- OpenAI revealed an expanded deal with Oracle and SoftBank , scaling its “Stargate” project to a $400 billion commitment across multiple phases and sites.
- OpenAI deepened its enterprise reach with a formal integration into Databricks — signaling a new phase in its push for commercial adoption.
Nvidia is supplying capital and chips. Oracle is building the sites. OpenAI is anchoring the demand. It’s a circular economy that could come under pressure if any one player falters. And while the headlines came fast this week, the physical buildout will take years to deliver — with much of it dependent on energy and grid upgrades that remain uncertain.
Another AI darling is CoreWeave, a company that provides GPU-accelerated cloud computing platforms and infrastructure. From its founding in 2017 until its pivot to cloud computing in 2019, Corweave was an obscure cryptocurrency miner with fewer than two dozen employees. Flooded with money from Wall Street and private-equity investors, it has morphed into a computing goliath with a market value bigger than General Motors or Target.
…………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………….
Massive AI infrastructure spending will require tremendous AI revenue for pay-back:
David Cahn, a partner at venture-capital firm Sequoia, estimates that the money invested in AI infrastructure in 2023 and 2024 alone requires consumers and companies to buy roughly $800 billion in AI products over the life of these chips and data centers to produce a good investment return. Analysts believe most AI processors have a useful life of between three and five years.
This week, consultants at Bain & Co. estimated the wave of AI infrastructure spending will require $2 trillion in annual AI revenue by 2030. By comparison, that is more than the combined 2024 revenue of Amazon, Apple, Alphabet, Microsoft, Meta and Nvidia, and more than five times the size of the entire global subscription software market.
Morgan Stanley estimates that last year there was around $45 billion of revenue for AI products. The sector makes money from a combination of subscription fees for chatbots such as ChatGPT and money paid to use these companies’ data centers. How the tech sector will cover the gap is “the trillion dollar question,” said Mark Moerdler, an analyst at Bernstein. Consumers have been quick to use AI, but most are using free versions, Moerdler said. Businesses have been slow to spend much on AI services, except for the roughly $30 a month per user for Microsoft’s Copilot or similar products. “Someone’s got to make money off this,” he said.
…………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………….
Why this time is different (?):
AI cheerleaders insist that this boom is different from the dot-com era. If AI continues to advance to the point where it can replace a large swath of white collar jobs, the savings will be more than enough to pay back the investment, backers argue. AI executives predict the technology could add 10% to global GDP in coming years.
“Training AI models is a gigantic multitrillion dollar market,” Oracle chairman Larry Ellison told investors this month. The market for companies and consumers using AI daily “will be much, much larger.”
The financing behind the AI build-out is complex. Debt is layered on at nearly every level. his “debt-fueled arms race” involves large technology companies, startups, and private credit firms seeking innovative ways to fund the development of data centers and acquire powerful hardware, such as Nvidia GPUs. Debt is layered across different levels of the AI ecosystem, from the large tech giants to smaller cloud providers and specialized hardware firms.
Alphabet, Microsoft, Amazon, Meta and others create their own AI products, and sometimes sell access to cloud-computing services to companies such as OpenAI that design AI models. The four “hyperscalers” alone are expected to spend nearly $400 billion on capital investments next year, more than the cost of the Apollo space program in today’s dollars. Some build their own data centers, and some rely on third parties to erect the mega-size warehouses tricked out with cooling equipment and power.
…………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………….
Echoes of bubbles past:
History is replete with technology bubbles that pop. Optimism over an invention—canals, electricity, railroads—prompts an investor stampede premised on explosive growth. Overbuilding follows, and investors eat giant losses, even when a new technology permeates the economy. Predicting when a boom turns into a bubble is notoriously hard. Many inflate for years. Some never pop, and simply stagnate.
The U.K.’s 19th-century railway mania was so large that over 7% of the country’s GDP went toward blanketing the country with rail. Between 1840 and 1852, the railway system nearly quintupled to 7,300 miles of track, but it only produced one-fourth of the revenue builders expected, according to Andrew Odlyzko,PhD, an emeritus University of Minnesota mathematics professor who studies bubbles. He calls the unbridled optimism in manias “collective hallucinations,” where investors, society and the press follow herd mentality and stop seeing risks.
He knows from firsthand experience as a researcher at Bell Labs in the 1990s. Then, telecom giants and upstarts raced to speculatively plunge tens of millions of miles of fiber cables into the ground, spending the equivalent of around 1% of U.S. GDP over half a decade.
Backers compared the effort to the highway system, to the advent of electricity and to discovering oil. The prevailing belief at the time, he said, was that internet use was doubling every 100 days. But in reality, for most of the 1990s boom, traffic doubled every year, Odlyzko found.
The force of the mania led executives across the industry to focus on hype more than unfavorable news and statistics, pouring money into fiber until the bubble burst.
“There was a strong element of self interest,” as companies and executives all stood to benefit financially as long as the boom continued, Odlyzko said. “Cautionary signs are disregarded.”
Kevin O’Hara, a co-founder of upstart fiber builder Level 3, said banks and stock investors were throwing money at the company, and executives believed demand would rocket upward for years. Despite worrying signs, executives focused on the promise of more traffic from uses like video streaming and games.
“It was an absolute gold rush,” he said. “We were spending about $110 million a week” building out the network.
When reality caught up, Level 3’s stock dropped 95%, while giants of the sector went bust. Much of the fiber sat unused for over a decade. Ultimately, the growth of video streaming and other uses in the early 2010s helped soak up much of the oversupply.
Worrying signs:
There are growing, worrying signs that the optimism about AI won’t pan out.
- MIT Media Lab (2025): The “State of AI in Business 2025” report found that 95% of custom enterprise AI tools and pilots fail to produce a measurable financial impact or reach full-scale production. The primary issue identified was a “learning gap” among leaders and organizations, who struggle to properly integrate AI tools and redesign workflows to capture value.
- A University of Chicago economics paper found AI chatbots had “no significant impact on workers’ earnings, recorded hours, or wages” at 7,000 Danish workplaces.
- Gartner (2024–2025): The research and consulting firm has reported that 85% of AI initiatives fail to deliver on their promised value. Gartner also predicts that by the end of 2025, 30% of generative AI projects will be abandoned after the proof-of-concept phase due to issues like poor data quality, lack of clear business value, and escalating costs.
- RAND Corporation (2024): In its analysis, RAND confirmed that the failure rate for AI projects is over 80%, which is double the failure rate of non-AI technology projects. Cited obstacles include cost overruns, data privacy concerns, and security risks.
OpenAI’s release of ChatGPT-5 in August was widely viewed as an incremental improvement, not the game-changing thinking machine many expected. Given the high cost of developing it, the release fanned concerns that generative AI models are improving at a slower pace than expected. Each new AI model—ChatGPT-4, ChatGPT-5—costs significantly more than the last to train and release to the world, often three to five times the cost of the previous, say AI executives. That means the payback has to be even higher to justify the spending.
Another hurdle: The chips in the data centers won’t be useful forever. Unlike the dot-com boom’s fiber cables, the latest AI chips rapidly depreciate in value as technology improves, much like an older model car. And they are extremely expensive.
“This is bigger than all the other tech bubbles put together,” said Roger McNamee, co-founder of tech investor Silver Lake Partners, who has been critical of some tech giants. “This industry can be as successful as the most successful tech products ever introduced and still not justify the current levels of investment.”
Other challenges include the growing strain on global supply chains, especially for chips, power and infrastructure. As for economy-wide gains in productivity, few of the biggest listed U.S. companies are able to describe how AI was changing their businesses for the better. Equally striking is the minimal euphoria some Big Tech companies display in their regulatory filings. Meta’s 10k form last year reads: “[T]here can be no assurance that the usage of AI will enhance our products or services or be beneficial to our business, including our efficiency or profitability.” That is very shaky basis on which to conduct a $300bn capex splurge.
………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………
Conclusions:
Big tech spending on AI infrastructure has been propping up the U.S. economy, with some projections indicating it could fuel nearly half of the 2025 GDP growth. However, this contribution primarily stems from capital expenditures, and the long-term economic impact is still being debated. George Saravelos of Deutsche Bank notes that economic growth is not coming from AI itself but from building the data centers to generate AI capacity.
Once those AI factories have been built, with needed power supplies and cooling, will the productivity gains from AI finally be realized? How globally disseminated will those benefits be? Finally, what will be the return on investment (ROI) for the big spending AI companies like the hyperscalers, OpenAI and other AI players?
………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………
References:
https://www.wsj.com/tech/ai/ai-bubble-building-spree-55ee6128
https://www.ft.com/content/6c181cb1-0cbb-4668-9854-5a29debb05b1
https://www.cnbc.com/2025/09/26/openai-big-week-ai-arms-race.html
Gartner: AI spending >$2 trillion in 2026 driven by hyperscalers data center investments
AI Data Center Boom Carries Huge Default and Demand Risks
AI spending is surging; companies accelerate AI adoption, but job cuts loom large
Will billions of dollars big tech is spending on Gen AI data centers produce a decent ROI?
Canalys & Gartner: AI investments drive growth in cloud infrastructure spending
AI wave stimulates big tech spending and strong profits, but for how long?
AI Echo Chamber: “Upstream AI” companies huge spending fuels profit growth for “Downstream AI” firms
OpenAI partners with G42 to build giant data center for Stargate UAE project
Big Tech and VCs invest hundreds of billions in AI while salaries of AI experts reach the stratosphere
Superclusters of Nvidia GPU/AI chips combined with end-to-end network platforms to create next generation data centers
Analysis: Cisco, HPE/Juniper, and Nvidia network equipment for AI data centers
Networking chips and modules for AI data centers: Infiniband, Ultra Ethernet, Optical Connections
OpenAI and Broadcom in $10B deal to make custom AI chips
Lumen Technologies to connect Prometheus Hyperscale’s energy efficient AI data centers
Proposed solutions to high energy consumption of Generative AI LLMs: optimized hardware, new algorithms, green data centers
Liquid Dreams: The Rise of Immersion Cooling and Underwater Data Centers
Lumen: “We’re Building the Backbone for the AI Economy” – NaaS platform to be available to more customers
Initiatives and Analysis: Nokia focuses on data centers as its top growth market
Cisco CEO sees great potential in AI data center connectivity, silicon, optics, and optical systems
It’s no surprise to IEEE Techblog readers that Cisco’s networking business – still its biggest unit, generating nearly half its total sales – reported <$6.9 billion in revenue for the three-month period ending in January (Cisco’s second fiscal quarter). That was down 3% compared with the same quarter the year before. For its first half year, networking sales dropped 14% year-over-year, to about $13.6 billion.
However, total second-quarter revenues grew 9% year-over-year, to just less than $14 billion, boosted by the Splunk (security company) acquisition in March 2024. Thanks to that deal, Cisco’s security revenues more than doubled for the first half, to about $4.1 billion. But net income fell 8%, to roughly $2.4 billion, due partly to higher costs for research and development, as well as sales and marketing expenses.
Cisco groused about an “inventory correction” as networking customers digested stock they had already bought, but that surely is not the case now as that inventory has been worked off by its customers (ISPs, telcos, enterprise & government end users). Cisco CFO Richard Scott Herren now says “The demand that we’re seeing today a function of extended lead times like we saw a couple of years ago. That’s not the case. Our lead times are not extending.”
Currently, Cisco firmly believes that Ethernet connectivity sales to owners of AI data centers is an “emerging opportunity.” That refers to Cisco’s data center switching solutions for “web-scale” and enterprise customer intra-data center communications. The company’s AI strategy is described here.
Image Courtesy of Cisco Systems
………………………………………………………………………………………………………………………………………
AI investments “will lead to our networking equipment being combined with Nvidia GPUs, and that’s how we’ll accomplish that in the future,” CEO Chuck Robbins told industry analysts on a call to discuss second-quarter results, according to a Motley Fool transcript. “There’s so much change going on right now from a technology perspective that there’s both excitement about the opportunity, and candidly, there’s a little bit of fear of slowing down too much and letting your competition get too much ahead of you. So, we saw solid demand,” he said.
However, Cisco will face mighty competition in that space.
- Nokia is targeting the same opportunity and last month said it would spend an additional €100 million (US$104 million) on its Internet Protocol unit annually with the goal of generating another €1 billion ($1.04 billion) in data center revenues by 2028.
- Arista Networks is another rival in this market, selling high performance Ethernet switches to cloud service providers like Microsoft.
- Nvidia, whose $7 billion acquisition of Mellanox in 2019 gave it effective control of InfiniBand, an alternative to Ethernet that had represented the main option for connecting GPU clusters when analysts published research on the topic in August 2023. Just as important, the Mellanox division of Nvidia also is a leader in Ethernet connectivity within data centers as described in this IEEE Techblog post.
- Juniper Networks (being acquired by HPC) is also focusing on networking the AI data center as per a white paper you can download after filling out this form.
During the Q & A, Robbins elaborated: “On the $700 million in AI orders, it’s a combination of systems, silicon, optics, and optical systems. And I think if you break it down, it’s about half is in silicon and systems. And it continues to accelerate. And I’d say the teams have done a great job on the silicon front. We’ve invested heavily in more resources there. The team is running parallel development efforts for multiple chips that are staggered in their time frames. They’ve worked hard. They were increasing the yield, which is a positive thing. And so, we feel good about it, but it’s a combination of all those things that we’re selling to the customers.”
…………………………………………………………………………………………………………………………………………………………………………………………
Enterprise AI:
“What we’re seeing on the enterprise side relative to AI is it’s still — customers are still in the very early days, and they all realize they need to figure out exactly what their use cases are. We’re starting to see some spending though on specific AI-driven infrastructure. And we think as we get AI pods out there — we got Hyperfabric coming. We got AI defense coming.
We have Hypershield in the market. And we got this new DPU switch, they are all going to be a part of the infrastructure to support these AI applications. So, we’re beginning to see it happen, but I think it’s also really important to understand that as the enterprises leverage their private data, their proprietary data, and they’ll do some training on that and then they’ll run inference obviously against that. We believe that opportunity is an order of magnitude higher than what we’ve seen in training today. We’re going to continue to innovate and build capabilities to put ourselves in a better position to be a real beneficiary as this continues to accelerate. But as of today, we feel like we’re in pretty good shape.”
“If you look at AI defense with the AI Summit that we did recently, there’s — I think there’s about 20-some-odd customers who are interested in going to proof of concept with us right now on it. We had almost half the Fortune 100 there for that event. So, I feel good about where we are. It will turn into greater demand as we just continue to scale these products.”
Telco use of AI Edge Applications:
“We see some of the European network operators are looking at delivering AI as a service,” said Robbins. “We see a lot of them planning for AI edge applications that are sitting at the edge of their networks that they’re managing for customers.”
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,………………………………………………
Cisco raised its guidance and now expects revenues for the full year of between $56 billion and $56.5 billion, up from its earlier range of $55.3 billion to $56.3 billion.
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,………………………………………………
References:
https://www.cisco.com/site/uk/en/solutions/artificial-intelligence/index.html
https://www.juniper.net/content/dam/www/assets/white-papers/us/en/networking-the-ai-data-center.pdf
Nokia selects Intel’s Justin Hotard as new CEO to increase growth in IP networking and data center connections
Initiatives and Analysis: Nokia focuses on data centers as its top growth market
Nvidia enters Data Center Ethernet market with its Spectrum-X networking platform
Lumen Technologies to connect Prometheus Hyperscale’s energy efficient AI data centers
The need for more cloud computing capacity and AI applications has been driving huge investments in data centers. Those investments have led to a steady demand for fiber capacity between data centers and more optical networking innovation inside data centers. Here’s the latest example of that:
Prometheus Hyperscale has chosen Lumen Technologies to connect its energy-efficient data centers to meet growing AI data demands. Lumen network services will help Prometheus with the rapid growth in AI, big data, and cloud computing as they address the critical environmental challenges faced by the AI industry.
Rendering of Prometheus Hyperscale flagship Data Center in Evanston, Wyoming:
……………………………………………………………………………….
Prometheus Hyperscale, known for pioneering sustainability in the hyperscale data center industry, is deploying a Lumen Private Connectivity Fabric℠ solution, including new network routes built with Lumen next generation wavelength services and Dedicated Internet Access (DIA) [1.] services with Distributed Denial of Service (DDoS) protection layered on top.
Note 1. Dedicated Internet Access (DIA) is a premium internet service that provides a business with a private, high-speed connection to the internet.
This expanded network will enable high-density compute in Prometheus facilities to deliver scalable and efficient data center solutions while maintaining their commitment to renewable energy and carbon neutrality. Lumen networking technology will provide the low-latency, high-performance infrastructure critical to meet the demands of AI workloads, from training to inference, across Prometheus’ flagship facility in Wyoming and four future data centers in the western U.S.
“What Prometheus Hyperscale is doing in the data center industry is unique and innovative, and we want to innovate alongside of them,” said Ashley Haynes-Gaspar, Lumen EVP and chief revenue officer. “We’re proud to partner with Prometheus Hyperscale in supporting the next generation of sustainable AI infrastructure. Our Private Connectivity Fabric solution was designed with scalability and security to drive AI innovation while aligning with Prometheus’ ambitious sustainability goals.”
Prometheus, founded as Wyoming Hyperscale in 2020, turned to Lumen networking solutions prior to the launch of its first development site in Aspen, WY. This facility integrates renewable energy sources, sustainable cooling systems, and AI-driven energy optimization, allowing for minimal environmental impact while delivering the computational power AI-driven enterprises demand. The partnership with Lumen reinforces Prometheus’ dedication to both technological innovation and environmental responsibility.
“AI is reshaping industries, but it must be done responsibly,” said Trevor Neilson, president of Prometheus Hyperscale. “By joining forces with Lumen, we’re able to offer our customers best-in-class connectivity to AI workloads while staying true to our mission of building the most sustainable data centers on the planet. Lumen’s network expertise is the perfect complement to our vision.”
Prometheus’ data center campus in Evanston, Wyoming will be one of the biggest data centers in the world with facilities expected to come online in late 2026. Future data centers in Pueblo, Colorado; Fort Morgan, Colorado; Phoenix, Arizona; and Tucson, Arizona, will follow and be strategically designed to leverage clean energy resources and innovative technology.
About Prometheus Hyperscale:
Prometheus Hyperscale, founded by Trenton Thornock, is revolutionizing data center infrastructure by developing sustainable, energy-efficient hyperscale data centers. Leveraging unique, cutting-edge technology and working alongside strategic partners, Prometheus is building next-generation, liquid-cooled hyperscale data centers powered by cleaner energy. With a focus on innovation, scalability, and environmental stewardship, Prometheus Hyperscale is redefining the data center industry for a sustainable future. This announcement follows recent news of Bernard Looney, former CEO of bp, being appointed Chairman of the Board.
To learn more visit: www.prometheushyperscale.com
About Lumen Technologies:
Lumen uses the scale of their network to help companies realize AI’s full potential. From metro connectivity to long-haul data transport to edge cloud, security, managed service, and digital platform capabilities, Lumenn meets its customers’ needs today and is ready for tomorrow’s requirements.
In October, Lumen CTO Dave Ward told Light Reading that a “fundamentally different order of magnitude” of compute power, graphics processing units (GPUs) and bandwidth is required to support AI workloads. “It is the largest expansion of the Internet in our lifetime,” Ward said.
Lumen is constructing 130,000 fiber route miles to support Meta and other customers seeking to interconnect AI-enabled data centers. According to a story by Kelsey Ziser, the fiber conduits in this buildout would contain anywhere from 144 to more than 500 fibers to connect multi-gigawatt data centers.
REFERENCES:
https://www.lightreading.com/data-centers/2024-in-review-data-center-shifts
Will billions of dollars big tech is spending on Gen AI data centers produce a decent ROI?
Superclusters of Nvidia GPU/AI chips combined with end-to-end network platforms to create next generation data centers
Initiatives and Analysis: Nokia focuses on data centers as its top growth market
Proposed solutions to high energy consumption of Generative AI LLMs: optimized hardware, new algorithms, green data centers
Deutsche Telekom with AWS and VMware demonstrate a global enterprise network for seamless connectivity across geographically distributed data centers
Initiatives and Analysis: Nokia focuses on data centers as its top growth market
Telco is no longer the top growth market for Nokia. Instead, it’s data centers, said Nokia’s CEO Pekka Lundmark on the company’s Q3 2024 earnings call last week. “Across Nokia, we are investing to create new growth opportunities outside of our traditional communications service provider market,” he said. “We see a significant opportunity to expand our presence in the data center market and are investing to broaden our product portfolio in IP Networks to better address this. There will be others as well, but that will be the number one. This is obviously in the very core of our strategy.”
Lundmark said Nokia’s telco total addressable market (TAM) is €84 billion, while its data center total addressable market is currently at €20 billion. “I mean, telco TAM will never be a significant growth market,” he added to no one’s surprise.
Nokia’s recent deal to acquire fiber optics equipment vendor Infinera for $2.3 Billion might help. The Finland based company said the combination with Infinera is expected to accelerate its path to double-digit operating margins in its optical-networks business unit (which was inherited from Alcatel-Lucent) . The transaction (expected to close in the first half of 2025) and the recent sale of submarine networks will reshape Nokia’s Network Infrastructure business to be built around fixed networks, internet-protocol networks and optical networks, the company said. Data centers not only require GPUs, but they also require optical networking to support their AI workloads. Lundmark said the role of optics will increase, not only in connections between data centers, but also inside data centers to connect servers to each other. “Once we get there, that market will be of extremely high volumes,” he said.
Pekka Lundmark, Nokia CEO– Photo: Arno Mikkor
- In September, Nokia announced the availability of its AI era, Event-Driven Automation (EDA) platform. Nokia EDA raises the bar on data center network operations with a modern approach that builds on Kubernetes to bring highly reliable, simplified, and adaptable lifecycle management to data center networks. Aimed at driving human error in network operations to zero, Nokia’s new platform reduces network disruptions and application downtime while also decreasing operational effort up to 40%. Nokia says its new EDA platform helps data center operators reduce errors in network operations. Nokia said it hopes to remove the risk of human error and reduce network disruptions and application downtime.
- A highlight of the recent quarter is a September deal with self proclaimed “AI hyperscalar” CoreWeave [1.] which selected Nokia to deploy its IP routing and optical transport equipment globally as part of its extensive backbone build-out, with immediate roll-out across its data centers in the U.S. and Europe. Raymond James analyst Simon Leopold said the CoreWeave win was good for Nokia to gain some exposure to AI, and he wondered if Nokia had a long-term strategy of evolving customers away from its telco base into more enterprise-like opportunities. “The reason why CoreWeave is so important is that they are now the leading GPU-as-a- service company,” said Lundmark. “And they have now taken pretty much our entire portfolio, both on the IP side and optical side. And as we know, AI is driving new business models, and one of the business models is clearly GPU-as-a-service,” he added.
Note 1. CoreWeave rents graphical processing units (GPUs) to artificial intelligence (AI) developers. A modern, Kubernetes native cloud that’s purpose-built for large scale, GPU-accelerated workloads. Designed with engineers and innovators in mind, CoreWeave claims to offer unparalleled access to a broad range of compute solutions that are up to 35x faster and 80% less expensive than legacy cloud providers.
……………………………………………………………………………………………………………………………………………………………
Nokia says its IP Interconnection can provide attractive business benefits to data center customers including:
- Improved security – Applications and services can be accessed via private direct connections to the networks of cloud providers collocated in the same facility without traversing the internet.
- Reduced transport costs – Colocated service providers, alternative network providers and carrier neutral network operators offer a wide choice of connections to remote destinations at a lower price.
- Higher performance and lower latency – As connections are direct and are often located closer to the person or thing they are serving, there is a reduction in latency and an increase in reliability as they bypass multiple hops across the public internet.
- More control – Through network automation and via customer portals, cloud service providers can gain more control of their cloud connectivity.
- Greater flexibility – With a wider range of connectivity options, enterprises can distribute application workloads and access cloud applications and services globally to meet business demands and to gain access to new markets.
……………………………………………………………………………………………………………………………………………………………
Nokia’s Data Center Market Risks:
The uncertainty is whether spending on GPUs and optical network equipment in the data center will produce the traffic growth to justify a decent ROI for Nokia. Also, the major cloud vendors (Amazon, Google, Microsoft and Facebook) design, develop, and install their own fiber optic networks. So it will likely be the new AI Data Center players that Nokia will try to sell to. William Webb, an independent consultant and former executive at Ofcom told Light Reading, “There may be substantially more traffic between data centers as models are trained but this will flow across high-capacity fiber connections which can be expanded easily if needed.” Text-based AI apps like ChatGPT generate “minuscule amounts of traffic,” he said. Video-based AI will merely substitute for the genuinely intelligent form.
References:
https://www.datacenterdynamics.com/en/news/nokia-eyes-data-center-market-growth-as-q3-sales-fall/
https://www.nokia.com/blog/enhance-cloud-services-with-high-capacity-interconnection/
https://www.lightreading.com/5g/telecom-glory-days-are-over-bad-news-for-nokia-worse-for-ericsson
AI adoption to accelerate growth in the $215 billion Data Center market
Market Overview:
Data Centers are a $215bn global market that grew 18% annually between 2018-2023. AI adoption is expected to accelerate data center growth as AI chips require 3-4x more electrical power versus traditional central processing units (CPUs).
AI adoption is poised to accelerate this growth meaningfully over coming years. BofA‘s US Semis analyst, Vivek Arya, forecasts the AI chip market to reach ~$200bn in 2027, up from $44bn in 2023. This has positive implications for the broader data center industry.
AI workloads are bandwidth-intensive, connecting hundreds of processors with gigabits of throughput. As these AI models grow, the number of GPUs required to process them grows, requiring larger networks to interconnect the GPUs. See Network Equipment market below.
The electrical and thermal equipment within a data center is sized for maximum load to ensure reliability and uptime. For electrical and thermal equipment manufacturers, AI adoption drives faster growth in data center power loads. AI chips require 3-4x more electrical power versus traditional CPUs (Central Processing Units).
BofA estimates data center capex was $215bn globally in 2023. The majority of this spend is for compute servers, networking and storage ($160bn) with data center infrastructure being an important, but smaller, piece ($55bn). For perspective, data center capex represented ~1% of global fixed capital formation, which includes all private & public sector spending on equipment and structures.
Networking Equipment Market:
BofA estimates a $20bn market size for Data Center networking equipment. Cisco is the market share leader, with an estimated 28% market share.
- Ethernet switches which communicate within the data center via local area networks. Typically, each rack would have a networking switch.
- Routers handle traffic between buildings, typically using internet protocol (IP). Some cloud service providers use “white box“ networking switches (e.g., manufactured by third parties, such as Taiwanese ODMs, to their specifications).
Data center speeds are in a state of constant growth. The industry has moved from 40G speeds to 100G speeds, and those are quickly giving way to 400G speeds. Yet even 400G speeds won’t be fast enough to support some emerging applications which may require 800G and 1.6TB data center speeds.
…………………………………………………………………………………………………………………………………….
Data Centers are also a bright spot for the construction industry. BofA notes that construction spending for data centers is approaching $30bn (vs $2bn in 2014) and accounts for nearly 21% of data center capex. At 4% of private construction spending (vs 2% five years ago), the data center category has surpassed retail, and could be a partial offset in a construction downturn.
Source: BofA Global Research
………………………………………………………………………………………………………………………..
References:
https://www.belden.com/blogs/smart-building/faster-data-center-speeds-depend-on-fiber-innovation#
Proposed solutions to high energy consumption of Generative AI LLMs: optimized hardware, new algorithms, green data centers
Nvidia enters Data Center Ethernet market with its Spectrum-X networking platform
Co-Packaged Optics to play an important role in data center switches
EdgeCore Digital Infrastructure and Zayo bring fiber connectivity to Santa Clara data center
Deutsche Telekom with AWS and VMware demonstrate a global enterprise network for seamless connectivity across geographically distributed data centers
Nvidia enters Data Center Ethernet market with its Spectrum-X networking platform
Nvidia is planning a big push into the Data Center Ethernet market. CFO Colette Kress said the Spectrum-X Ethernet-based networking solution it launched in May 2023 is “well on track to begin a multi-billion-dollar product line within a year.” The Spectrum-X platform includes: Ethernet switches, optics, cables and network interface cards (NICs). Nvidia already has a multi-billion-dollar play in this space in the form of its Ethernet NIC product. Kress said during Nvidia’s earnings call that “hundreds of customers have already adopted the platform.” And that Nvidia plans to “launch new Spectrum-X products every year to support demand for scaling compute clusters from tens of thousands of GPUs today to millions of DPUs in the near future.”
- With Spectrum-X, Nvidia will be competing with Arista, Cisco, and Juniper at the system level along with “bare metal switches” from Taiwanese ODMs running DriveNets network cloud software.
- With respect to high performance Ethernet switching silicon, Nvidia competitors include Broadcom, Marvell, Microchip, and Cisco (which uses Silicon One internally and also sells it on the merchant semiconductor market).
Image by Midjourney for Fierce Network
…………………………………………………………………………………………………………………………………………………………………………..
In November 2023, Nvidia said it would work with Dell Technologies, Hewlett Packard Enterprise and Lenovo to incorporate Spectrum-X capabilities into their compute servers. Nvidia is now targeting tier-2 cloud service providers and enterprise customers looking for bundled solutions.
Dell’Oro Group VP Sameh Boujelbene told Fierce Network that “Nvidia is positioning Spectrum-X for AI back-end network deployments as an alternative fabric to InfiniBand. While InfiniBand currently dominates AI back-end networks with over 80% market share, Ethernet switches optimized for AI deployments have been gaining ground very quickly.” Boujelbene added Nvidia’s success with Spectrum-X thus far has largely been driven “by one major 100,000-GPU cluster, along with several smaller deployments by Cloud Service Providers.” By 2028, Boujelbene said Dell’Oro expects Ethernet switches to surpass InfiniBand for AI in the back-end network market, with revenues exceeding $10 billion.
………………………………………………………………………………………………………………………………………………………………………………
In a recent IEEE Techblog post we wrote:
While InfiniBand currently has the edge in the data center networking market, but several factors point to increased Ethernet adoption for AI clusters in the future. Recent innovations are addressing Ethernet’s shortcomings compared to InfiniBand:
- Lossless Ethernet technologies
- RDMA over Converged Ethernet (RoCE)
- Ultra Ethernet Consortium’s AI-focused specifications
Some real-world tests have shown Ethernet offering up to 10% improvement in job completion performance across all packet sizes compared to InfiniBand in complex AI training tasks. By 2028, it’s estimated that: 1] 45% of generative AI workloads will run on Ethernet (up from <20% now) and 2] 30% will run on InfiniBand (up from <20% now).
………………………………………………………………………………………………………………………………………………………………………………
References:
https://www.fierce-network.com/cloud/data-center-ethernet-nvidias-next-multi-billion-dollar-business
https://www.nvidia.com/en-us/networking/spectrumx/
Will AI clusters be interconnected via Infiniband or Ethernet: NVIDIA doesn’t care, but Broadcom sure does!
Data Center Networking Market to grow at a CAGR of 6.22% during 2022-2027 to reach $35.6 billion by 2027
LightCounting: Optical Ethernet Transceiver sales will increase by 40% in 2024
Microsoft choses Lumen’s fiber based Private Connectivity Fabric℠ to expand Microsoft Cloud network capacity in the AI era
Lumen Technologies and Microsoft Corp. announced a new strategic partnership today. Microsoft has chosen Lumen to expand its network capacity and capability to meet the growing demand on its datacenters due to AI (i.e. huge processing required for Large Language Models, including data collection, preprocessing, training, and evaluation). Datacenters have become critical infrastructure that power the compute capabilities for the millions of people and organizations who rely on and trust the Microsoft Cloud.
Microsoft claims they are playing a leading role in ushering in the era of AI, offering tools and platforms like Azure OpenAI Service, Microsoft Copilot and others to help people be more creative, more productive and to help solve some of humanity’s biggest challenges. As Microsoft continues to evolve and scale its ecosystem, it is turning to Lumen as a strategic supplier for its network infrastructure needs and is investing with Lumen to support its next generation of applications for Microsoft platform customers worldwide.
Lumen’s Private Connectivity Fabric℠ is a custom network that includes dedicated access to existing fiber in the Lumen network, the installation of new fiber on existing and new routes, and the use of Lumen’s new digital services. This AI-ready infrastructure will strengthen the connectivity capabilities between Microsoft’s datacenters by providing the network capacity, performance, stability and speed that customers need as data demands increase.
Art by Midjourney for Fierce Network
…………………………………………………………………………………………………………………………………………………………………………………..
“AI is reshaping our daily lives and fundamentally changing how businesses operate,” said Erin Chapple, corporate vice president of Azure Core Product and Design, Microsoft. “We are focused both on the impact and opportunity for customers relative to AI today, and a generation ahead when it comes to our network infrastructure. Lumen has the network infrastructure and the digital capabilities needed to help support Azure’s mission in creating a reliable and scalable platform that supports the breadth of customer workloads—from general purpose and mission-critical, to cloud-native, high-performance computing, and AI, plus what’s on the horizon. Our work with Lumen is emblematic of our investments in our own cloud infrastructure, which delivers for today and for the long term to empower every person and every organization on the planet to achieve more.”
“We are preparing for a future where AI is the driving force of innovation and growth, and where a powerful network infrastructure is essential for companies to thrive,” said Kate Johnson, president and CEO, Lumen Technologies (a former Microsoft executive). “Microsoft has an ambitious vision for AI and this level of innovation requires a network that can make it reality. Lumen’s expansive network meets this challenge, with unique routes, unmatched coverage, and a digital platform built to give companies the flexibility, access and security they need to create an AI-enabled world.”
Lumen has launched an enterprise-wide transformation to simplify and optimize its operations. By embracing Microsoft’s cloud and AI technology, Lumen can reduce its overall technology costs, remove legacy systems and silos, improve its offerings, and create new solutions for its global customer base. Lumen will migrate and modernize its workloads to Microsoft Azure, use Microsoft Entra solutions to safeguard access and prevent identity attacks and partner with Microsoft to create and deliver new telecom industry-specific solutions. This element alone is expected to improve Lumen’s cash flow by more than $20 million over the next 12 months while also improving the company’s customer experience.
“Azure’s advanced global infrastructure helps customers and partners quickly adapt to changing economic conditions, accelerate technology innovation, and transform their business with AI,” said Chapple. “We are committed to partnering with Lumen to help deliver on their transformation goals, reimagine cloud connectivity and AI synergies, drive business growth, and help customers achieve more.”
This collaboration expands upon the longstanding relationship between Lumen Technologies and Microsoft. The companies have worked together for several years, with Lumen leveraging Copilot to automate routine tasks and reduce employee workloads and enhance Microsoft Teams.
……………………………………………………………………………………………………………………………………………………………………………………………………..
Lumen’s CMO Ryan Asdourian hinted the deal could be the first in a series of such partnerships, as network infrastructure becomes the next scarce resource in the era of AI. “When the world has talked about what’s needed for AI, you usually hear about power, space and cooling…[these] have been the scarce resources,” Asdourian told Fierce Telecom. Asdourian said Lumen will offer Microsoft access to a combination of new and existing routes in the U.S., and will overpull fiber where necessary. However, he declined to specify the speeds which will be made available or exactly how many of Microsoft’s data centers it will be connecting.
Microsoft will retain full control over network speeds, routes and redundancy options through Lumen’s freshly launched Private Connectivity Fabric digital interface. “That is not something traditional telecom has allowed,” Asdourian said.
Asdourian added that Lumen isn’t just looking to enable AI, but also incorporate it into its own operations. Indeed, part of its partnership deal with Microsoft involves Lumen’s adoption of Azure cloud and other Microsoft services to streamline its internal and network systems. Asdourian said AI could be used to make routing and switching on its network more intelligent and efficient.
…………………………………………………………………………………………………………………………………………………………………………………..
About Lumen Technologies:
Lumen connects the world. We are igniting business growth by connecting people, data, and applications – quickly, securely, and effortlessly. Everything we do at Lumen takes advantage of our network strength. From metro connectivity to long-haul data transport to our edge cloud, security, and managed service capabilities, we meet our customers’ needs today and as they build for tomorrow. For news and insights visit news.lumen.com, LinkedIn: /lumentechnologies, Twitter: @lumentechco, Facebook: /lumentechnologies, Instagram: @lumentechnologies and YouTube: /lumentechnologies.
About Microsoft:
Microsoft (Nasdaq “MSFT” @microsoft) creates platforms and tools powered by AI to deliver innovative solutions that meet the evolving needs of our customers. The technology company is committed to making AI available broadly and doing so responsibly, with a mission to empower every person and every organization on the planet to achieve more.
…………………………………………………………………………………………………………………………………………………………………………………..
References:
https://news.lumen.com/2024-07-24-Microsoft-and-Lumen-Technologies-partner-to-power-the-future-of-AI-and-enable-digital-transformation-to-benefit-hundreds-of-millions-of-customers
https://fierce-network.com/cloud/microsoft-taps-lumens-fiber-network-help-it-meet-ai-demand
AI Frenzy Backgrounder; Review of AI Products and Services from Nvidia, Microsoft, Amazon, Google and Meta; Conclusions
Lumen, Google and Microsoft create ExaSwitch™ – a new on-demand, optical networking ecosystem
ACSI report: AT&T, Lumen and Google Fiber top ranked in fiber network customer satisfaction
Lumen to provide mission-critical communications services to the U.S. Department of Defense
Dell’Oro: Optical Transport market to hit $17B by 2027; Lumen Technologies 400G wavelength market
Light Source Communications Secures Deal with Major Global Hyperscaler for Fiber Network in Phoenix Metro Area
Light Source Communications is building a 140-mile fiber middle-mile network in the Phoenix, AZ metro area, covering nine cities: Phoenix, Mesa, Tempe, Chandler, Gilbert, Queen Creek, Avondale, Coronado and Cashion. The company already has a major hyperscaler as the first anchor tenant.
There are currently 70 existing and planned data centers in the area that Light Source will serve. As one might expect, the increase in data centers stems from the boom in artificial intelligence (AI).
The network will include a big ring, which will be divided into three separate rings. In total, Light Source will be deploying 140 miles of fiber. The company has partnered with engineering and construction provider Future Infrastructure LLC, a division of Primoris Services Corp., to make it happen.
“I would say that AI happens to be blowing up our industry, as you know. It’s really in response to the amount of data that AI is demanding,” said Debra Freitas [1.], CEO of Light Source Communications (LSC).
Note 1. Debra Freitas has led LSC since co-founding in 2014. Owned and operated network with global OTT as a customer. She developed key customer relationships, secured funding for growth. Currently sits on the Executive Board of Incompas.
……………………………………………………………………………………………………..
Light Source plans for the entire 140-mile route to be underground. It’s currently working with the city councils and permitting departments of the nine cities as it goes through its engineering and permit approval processes. Freitas said the company expects to receive approvals from all the city councils and to begin construction in the third quarter of this year, concluding by the end of 2025.
Primoris delivers a range of specialty construction services to the utility, energy, and renewables markets throughout the United States and Canada. Its communications business is a leading provider of critical infrastructure solutions, including program management, engineering, fabrication, replacement, and maintenance. With over 12,700 employees, Primoris had revenue of $5.7 billion in 2023.
“We’re proud to partner with Light Source Communications on this impactful project, which will exceed the growing demands for high-capacity, reliable connectivity in the Phoenix area,” said Scott Comley, president of Primoris’ communications business. “Our commitment to innovation and excellence is well-aligned with Light Source’s cutting-edge solutions and we look forward to delivering with quality and safety at the forefront.”
Light Source is a carrier neutral, owner-operator of networks serving enterprises throughout the U.S. In addition to Phoenix, several new dark fiber routes are in development in major markets throughout the Central and Western United States. For more information about Light Source Communications, go to lightsourcecom.net.
The city councils in the Phoenix metro area have been pretty busy with fiber-build applications the past couple of years because the area is also a hotbed for companies building fiber-to-the-premises (FTTP) networks. In 2022 the Mesa City Council approved four different providers to build fiber networks. AT&T and BlackRock have said their joint venture would also start deploying fiber in Mesa.
Light Source is focusing on middle-mile, rather than FTTP because that’s where the demand is, according to Freitas. “Our route is a unique route, meaning there are no other providers where we’re going. We have a demand for the route we’re putting in,” she noted.
The company says it already has “a major, global hyperscaler” anchor tenant, but it won’t divulge who that tenant is. Its network will also touch Arizona State University at Tempe and the University of Arizona.
Light Source doesn’t light any of the fiber it deploys. Rather, it is carrier neutral and sells the dark fiber to customers who light it themselves and who may resell it to their own customers.
Light Source began operations in 2014 and is backed by private equity. It did not receive any federal grants for the new middle-mile network in Arizona.
………………………………………………………………………………………………………..
Bill Long, Zayo’s chief product officer, told Fierce Telecom recently that data centers are preparing for an onslaught of demand for more compute power, which will be needed to handle AI workloads and train new AI models.
…………………………………………………………………………………………………………
About Light Source Communications:
Light Source Communications (LSC) is a carrier neutral, customer agnostic provider of secure, scalable, reliable connectivity on a state-of-the-art dark fiber network. The immense amounts of data businesses require to compete in today’s global market requires access to an enhanced fiber infrastructure that allows them to control their data. With over 120 years of telecom experience, LSC offers an owner-operated network for U.S. businesses to succeed here and abroad. LSC is uniquely positioned and is highly qualified to build the next generation of dark fiber routes across North America, providing the key connections for business today and tomorrow.
References:
https://www.lightsourcecom.net/services/
https://www.fiercetelecom.com/ai/ai-demand-spurs-light-source-build-middle-mile-network-phoenix
Proposed solutions to high energy consumption of Generative AI LLMs: optimized hardware, new algorithms, green data centers
AI sparks huge increase in U.S. energy consumption and is straining the power grid; transmission/distribution as a major problem
AI Frenzy Backgrounder; Review of AI Products and Services from Nvidia, Microsoft, Amazon, Google and Meta; Conclusions
