Ethernet switch/router market
1.6T Ethernet Packet Loss Problem Solved: PHY-Layer Buffer Exhaustion with MAC-Layer Credit-Based Flow Control
Introduction:
As network fabrics scale to support next-generation AI clusters and workloads, the industry is confronting a harsh physical-layer reality. Recent laboratory benchmarks evaluating 1.6 Terabit Ethernet (1.6T) data streams confirm that under saturated, wire-speed workloads, traditional packet-forwarding logic triggers instantaneous Physical layer (PHY) memory buffer exhaustion.
Recently released laboratory validation data provides the first definitive, real-world proof that the 1.6 Terabit Ethernet (1.6T) PHY layer packet loss problem has been solved in hardware. That’s enabled the networking industry to go from theoretical design to proven physical deployment for next-generation multi-terabit fabrics. While the Open Compute Project (OCP) outlined the architectural blueprint for Ethernet for Scale-Up Networks (ESUN) in February 2026, the live telemetry published from production-ready merchant silicon testbeds establishes a critical milestone: under saturated, wire-speed workloads, hardware-enforced Credit-Based Flow Control (CBFC) at the Media Access Control (MAC) layer successfully prevents physical layer (PHY) memory buffer exhaustion. In synchronized, collective AI training workloads (such as All-Reduce), dropping a single packet introduces a catastrophic tail-latency penalty that leaves expensive GPU clusters sitting idle.
This solution is primarily based on the data link-layer protocol stack defined by the Open Compute Project’s (OCP’s) Ethernet for Scale-Up Networks (ESUN) specification which was successfully tested at the OCP Summit last week by Keysight.
Last week’s hardware test results confirm that by combining the physical signaling foundation of the IEEE P802.3dj Task Force (within the IEEE 802.3 Working Group) with the active pacing blocks of the OCP ESUN specification, silicon architects can guarantee zero-packet-discard performance at full 1.6T line rates without relying on physically impractical silicon buffer footprints.
The Root Cause – Synchronized Bursts vs. PHY Buffers:
Traditional Ethernet was designed for asynchronous, distributed internet traffic where transient congestion could be absorbed by deep-buffer switches or managed gracefully by top-of-stack TCP/IP windows. AI scale-up environments flip this paradigm. Thousands of processing nodes push traffic simultaneously in synchronized, bursty phases. When a 1.6T pipe—operating on 8 channels of 224G SerDes using PAM4 modulation—receives an unpaced burst, the sheer data density overflows the tight SRAM allocation of the physical coding and routing components.
-
- Priority Flow Control (PFC / Pause Frames): Operates too coarsely. Sending a pause frame halts an entire traffic class across the link, creating head-of-line blocking, buffer bloat, and potential deadlock cascades across the data center fabric [Keysight Technologies Protocol Analysis].
- TCP/RoCEv2 Retransmissions: Re-sending a dropped packet introduces millisecond-level delays. In a tightly coupled cluster, the entire training job stalls until that single packet is recovered, destroying overall cluster efficiency.
| Architectural Dimension | IEEE P802.3dj Draft Standard | OCP ESUN Open Specification |
|---|---|---|
| Primary Layer Focus | Physical Layer (PHY) & Electrical Interfaces | Media Access Control (MAC) & Link Sublayer |
| Signaling & Plumbing | 8 lanes of 224G SerDes via PAM4 modulation | Inherits IEEE physicals, modifies packet frame encapsulation |
| Error Handling | Concatenated Forward Error Correction (Inner/Outer RS-FEC) | Link-Level Retry (LLR) for microsecond-range hardware recovery |
| Congestion Control | None (Best-effort link transport) | Credit-Based Flow Control (CBFC) per virtual channel |
| Header Efficiency | Retains standard 28–48 byte IP/UDP network headers | Strips network overhead down to a lightweight 4-byte L2 header |
-
- Short-Range Data Center Clusters (Up to 500 meters):
-
- Standard designation:
1.6TBASE-DR8 - Fiber Type:OS2 Single-Mode Fiber (SMF)
- Cabling Infrastructure: Uses a 16-fiber MPO/MTP connector layout. It utilizes a parallel structure with 8 fibers for transmitting (Tx) and 8 fibers for receiving (Rx) to run eight concurrent 200 Gbps PAM4 pathways.
- Standard designation:
-
- Campus and Intra-Data Center Links (Up to 2 kilometers):
-
- Standard designation:
1.6TBASE-DR8-2 - Fiber Type:OS2 Single-Mode Fiber (SMF)
- Cabling Infrastructure: Also utilizes parallel fibers over a 16-fiber MPO/MTP connector.
- Note: Non-IEEE multi-source agreements (MSAs) have also developed Wavelength Division Multiplexing (WDM) variants like 2×FR4 to hit this 2 km reach. These multiplex eight channels down onto a simpler, standard LC or dual mini-MPO/MTP duplex fiber layout over a standard OS2 cable.
- Standard designation:
-
- Short-Range Data Center Clusters (Up to 500 meters):
-
- Credit Allocation: The downstream receiving node advertises explicit buffer availability to the upstream transmitting node in the form of “credits,” segmented down to granular virtual channels
- Strict Transmission Gate: The upstream MAC transmitter is prohibited from sending a frame unless it possesses a valid credit for that specific virtual channel [Synopsys Architecture Blog].
- Hardware-Enforced Backpressure: As the receiving node’s physical buffers fill, it stops issuing credits. The transmitting node immediately pauses transmission at the silicon boundary, holding packets safely in scale-out network queues before they ever touch the physical wire [Keysight Technologies Protocol Analysis].
-
- Hyperscale Cloud Operators: Meta, Microsoft
- Silicon Architecture & Chip Design: AMD, Broadcom, Cisco, Intel, Marvell, NVIDIA
- Network & Optical Systems: Arista, Hewlett Packard Enterprise (HPE)
- Design & Automation Software: Cadence, Synopsys
+-------------------------------------------------------------------------+
| AI WORKLOAD LAYER (All-Reduce / Collective Communications) |
+-------------------------------------------------------------------------+
| OCP ESUN LINK LAYER (Base Specification 1.0) |
| - 4-Byte Segmented Frame Header (Stripped of traditional IP overhead) |
| - Credit-Based Flow Control (Proactive virtual-channel buffering) |
| - Link-Level Retry (Microsecond-range local hardware error recovery) |
+-------------------------------------------------------------------------+
| IEEE P802.3dj PHYSICAL LAYER (Draft Amendment) |
| - Multi-rate MAC Configuration (1.6 Terabits per second throughput) |
| - Concatenated Forward Error Correction (Inner & Outer RS-FEC) |
| - Physical Media Dependent Signaling (8 Lanes x 224G SerDes via PAM4) |
+-------------------------------------------------------------------------+
| PHYSICAL LAYER MEDIA (Data Center Infrastructure) |
| - OS2 Single-Mode Fiber (Max 500m via DR8 / Max 2km via 2xFR4 WDM) |
| - MPO-16 / MTP-16 High-Density Connectors |
+-------------------------------------------------------------------------+References:
OCP ESUN – Network Operator Requirements – Base Specification 1.0. https://www.opencompute.org/news/the-ocp-esun-10-specification-has-been-released/
Nvidia enters Data Center Ethernet market with its Spectrum-X networking platform
Nvidia is planning a big push into the Data Center Ethernet market. CFO Colette Kress said the Spectrum-X Ethernet-based networking solution it launched in May 2023 is “well on track to begin a multi-billion-dollar product line within a year.” The Spectrum-X platform includes: Ethernet switches, optics, cables and network interface cards (NICs). Nvidia already has a multi-billion-dollar play in this space in the form of its Ethernet NIC product. Kress said during Nvidia’s earnings call that “hundreds of customers have already adopted the platform.” And that Nvidia plans to “launch new Spectrum-X products every year to support demand for scaling compute clusters from tens of thousands of GPUs today to millions of DPUs in the near future.”
- With Spectrum-X, Nvidia will be competing with Arista, Cisco, and Juniper at the system level along with “bare metal switches” from Taiwanese ODMs running DriveNets network cloud software.
- With respect to high performance Ethernet switching silicon, Nvidia competitors include Broadcom, Marvell, Microchip, and Cisco (which uses Silicon One internally and also sells it on the merchant semiconductor market).
Image by Midjourney for Fierce Network
…………………………………………………………………………………………………………………………………………………………………………..
In November 2023, Nvidia said it would work with Dell Technologies, Hewlett Packard Enterprise and Lenovo to incorporate Spectrum-X capabilities into their compute servers. Nvidia is now targeting tier-2 cloud service providers and enterprise customers looking for bundled solutions.
Dell’Oro Group VP Sameh Boujelbene told Fierce Network that “Nvidia is positioning Spectrum-X for AI back-end network deployments as an alternative fabric to InfiniBand. While InfiniBand currently dominates AI back-end networks with over 80% market share, Ethernet switches optimized for AI deployments have been gaining ground very quickly.” Boujelbene added Nvidia’s success with Spectrum-X thus far has largely been driven “by one major 100,000-GPU cluster, along with several smaller deployments by Cloud Service Providers.” By 2028, Boujelbene said Dell’Oro expects Ethernet switches to surpass InfiniBand for AI in the back-end network market, with revenues exceeding $10 billion.
………………………………………………………………………………………………………………………………………………………………………………
In a recent IEEE Techblog post we wrote:
While InfiniBand currently has the edge in the data center networking market, but several factors point to increased Ethernet adoption for AI clusters in the future. Recent innovations are addressing Ethernet’s shortcomings compared to InfiniBand:
- Lossless Ethernet technologies
- RDMA over Converged Ethernet (RoCE)
- Ultra Ethernet Consortium’s AI-focused specifications
Some real-world tests have shown Ethernet offering up to 10% improvement in job completion performance across all packet sizes compared to InfiniBand in complex AI training tasks. By 2028, it’s estimated that: 1] 45% of generative AI workloads will run on Ethernet (up from <20% now) and 2] 30% will run on InfiniBand (up from <20% now).
………………………………………………………………………………………………………………………………………………………………………………
References:
https://www.fierce-network.com/cloud/data-center-ethernet-nvidias-next-multi-billion-dollar-business
https://www.nvidia.com/en-us/networking/spectrumx/
Will AI clusters be interconnected via Infiniband or Ethernet: NVIDIA doesn’t care, but Broadcom sure does!
Data Center Networking Market to grow at a CAGR of 6.22% during 2022-2027 to reach $35.6 billion by 2027
LightCounting: Optical Ethernet Transceiver sales will increase by 40% in 2024
Dell’Oro: Campus Ethernet Switch Revenues dropped 23% YoY in 1Q-2024
Worldwide Campus Ethernet Switch [1.] revenues plummeted by 23% YoY in the 1st quarter of 2024 to a 2-year low, according to the Dell’Oro Group.
- The only two vendors that grew campus switch revenues year over year were Santa Clara, CA based Arista Networks and Ubiquiti [2.].
- Cisco’s campus switch revenues fell more than the worldwide average.
Note 1. With higher demand from new use cases for wired connectivity such as automation, analytics, and network visibility, and the need for new access switches to aggregate Wireless LAN access points, Dell’Oro Group is offering in-depth analysis specifically on Ethernet Switches built and optimized for deployment outside the data center, for the purpose of connecting users and things to the corporate Local Area Network (LAN).
“Basically, campus switches are really the networking gear to connect users and devices and laptops,” said Sameh Boujelbene, VP with Dell’Oro Group. “Access points are probably the number one application.”
………………………………………………………………………………………………………..
Note 2. Although headquartered in New York, NY, Ubiquiti wrote in a 2019 SEC filing:
“We use contract manufacturers, primarily located in China, Vietnam and Taiwan, to manufacture our products. Our relationships with contract manufacturers allow us to conserve working capital, reduce manufacturing costs and minimize delivery lead times while maintaining high product quality and the ability to scale quickly to handle increased order volume. Over the long term, our contract manufacturers are not required to manufacture our products for any specific period or in any specific quantity. If necessary, we expect that it would take approximately 3 to 6 months to transition manufacturing, quality assurance and shipping services to new providers.”
………………………………………………………………………………………………………………………
“Vendor backlogs of campus switch orders have now been completely run down, and the market is in a multi-quarter digestion cycle,” said Siân Morgan, Research Director at Dell’Oro Group. “The shipments of most port speeds declined, and the Average Sales Price (ASP) also dropped on a YoY basis.
“However, in 1Q 2024, Arista had its third sequential quarter of (market) share gain, growing Campus Switch sales to large enterprises.
On the downside, Cisco’s Campus Switch shipments contracted sharply. This reduction contrasts with their shipments in 2023, when Cisco opened the “floodgate” for Catalyst and Meraki port shipments which had been on backorder,” added Morgan.
Additional highlights from the 1Q 2024 Ethernet Switch – Campus Report:
- The contraction in campus switch sales was broad-based across all regions, with the exception of Central America-Latin America (CALA).
- Some vendors bucked the price trend and were able to grow port ASPs thanks to richer product mixes.
- 5/5.0 Gbps switch ports are expected to return to growth as shipments of Wi-Fi 7 Access Points accelerate.
The Dell’Oro Group Ethernet Switch – Campus Quarterly Report offers a detailed view of Ethernet switches built and optimized for deployment outside the data center, to connect users and things to the Local Area Networks. The report contains in-depth market and vendor-level information on manufacturers’ revenue, ports shipped and average selling prices for both Modular and Fixed, and Fixed Managed and Unmanaged Ethernet Switches (100 Mbps, 1/2.5/5/10/25/40/50/100/400 Gbps), Power-over-Ethernet, plus regional breakouts as well as split by customer size (Enterprise vs. SMB) and vertical segments.
To purchase these reports, email [email protected]
References:
Campus Ethernet Switch Revenues Crash to a Two-Year Low in 1Q 2024, According to Dell’Oro Group
Dell’Oro: Optical Transport, Mobile Core Network & Cable CPE shipments all declined in 1Q-2024
Dell’Oro: Broadband network equipment spending to drop again in 2024 to ~$16.5 B
Dell’Oro: Ethernet adapter port shipments declined in Q3-2021 but expected to grow in 2022
After declining in the 3rd quarter of 2021, Ethernet adapter port shipments are forecast to return to double-digit growth in 2022, as supply restrictions ease, and as Smart NICs create growth opportunities.
“Ethernet adapter port shipments declined 7% year-over-year in 3Q 2021, as vendors faced various component sourcing challenges, with lead-times extending beyond 52 weeks in some extreme cases,” said Baron Fung, Research Director at Dell’Oro Group.
“In contrast, Ethernet controller shipments have approached record levels, as we believe server vendors are increasing their inventories of controllers in anticipation of stronger cloud and enterprise demand ahead,” added Fung.
Additional highlights from the 3Q 2021 Ethernet Controller and Adapter report include:
- Total Ethernet controller and adapter revenue forecast to grow 27 percent in 2021.
- Major cloud service providers are upgrading server connectivity to 100 and 200 Gbps port speeds in conjunction with network upgrades.
- Smart NICs developed internally by cloud service providers, such as Amazon and Microsoft, for their data centers have accounted for the majority of the shipments.
- Vendors such as Marvell and Nvidia, are expected to increase their share in 2022 as customer qualifications make progress.
“Outside of the hyperscale cloud service providers, the smartNIC market is still in its early stages,” Fung said, as customers explore viable use cases for them in their data centers.
Most adapter vendors today either offer smartNICs or are sampling them. Besides cloud providers, smartNIC vendors include Marvell, Intel, Xilinx, Nvidia, Napatech, Pensando, Fungible, Ethernity, and Broadcom.
Most of them “are trying to grow their smartNIC share in the tier two cloud, enterprise, and telco segments,” Fung said.
The Dell’Oro Group Ethernet Controller and Adapter Quarterly Report provide complete, in-depth coverage of the market with tables covering manufacturers’ revenue; average selling prices; and unit and port shipments by speed (1 Gbps, 10 Gbps, 25 Gbps, 40 Gbps, 50 Gbps, 100 Gbps, and 200 Gbps) for Ethernet controllers and adapters. The report also covers Smart NIC controllers and adapters. To purchase this report, please contact us at [email protected].
References:
Ethernet Adapter Shipments Stalled by Supply Constraints in 3Q 2021, According to Dell’Oro Group
Synergy Research: Ethernet Switch & Router revenues drop to 7 year low in Q1-2020
Revenues from the Ethernet switch and router market fell 14 percent to a seven-year low of $9.1 billion in Q1 2020, according to Synergy Research Group. Ethernet switches, enterprise routers and service provider routers all saw double digit declines in Q1-2020.
Ethernet switching is the largest of the three segments accounting for 62% of the total Q1 market. While GbE switches remain the largest segment in both fixed and modular Ethernet switches, the most notable feature of the market is the rapid deployment of 100 GbE and 25 GbE fixed switches. Both of these segments actually grew in the first quarter.
In Q1-2020, revenues from enterprise routers were down 15% from 2019, but on a trailing twelve month basis the market still grew thanks to strong numbers in the previous three quarters. Service Provider routers saw the biggest decline in the first quarter, down 19% from 2019. In Q1 North America remained the biggest region accounting for almost 40% of worldwide revenues, followed by APAC, EMEA and Latin America. In aggregate across all switch and router segments, year-on-year revenue declines were broadly similar across the major regions.
Comment from IEEE 802.3 Ethernet standards veteran Geoff Thompson:
Revenue is deceptive in this market which has been characterized by falling prices per function for many years. In a market where the dominant function is trunk switching and higher speeds per lane are coming on line every year that is a very difficult ride.
You have to keep the functional volume growing enough to overcome both the price erosion per transistor and the higher cost efficiency of faster lanes as well as maintain your market share in order to grow your revenue. That is very, very tough.
Cisco’s market share in switches and routers was 51% in Q1, meaning that for eight of the last twelve quarters it has been over the 50% mark. Across the three main markets, Cisco’s Q1 share was 57% for Ethernet switches, 65% for enterprise routers and 35% for service provider routers. Behind Cisco the ranking of vendors was different in each of the three markets, but in aggregate Cisco is followed by Huawei, Nokia, Juniper, Arista Networks and HPE. Beyond this leading group, other active vendors include Ericsson, Extreme, H3C and ZTE.
“In a market that is usually characterized by relative stability and predictability, the first quarter represented a sharp change from the norm, clearly as a result of COVID-19. In a pre-pandemic world we’d have expected total vendor revenues from switches and routers to have been a billion dollars higher that what we actually saw,” said John Dinsdale, a Chief Analyst at Synergy Research Group.
“On balance the Q1 hit was driven more by supply chain issues rather than by soft demand. We’d expect supply chain problems to be resolved reasonably quickly, but demand is a different story. On the service provider side demand remains robust as network traffic continues to grow, but enterprise demand will be a much spottier picture and some sectors will take several months before returning to some form of normality.”
……………………………………………………………………………………………………………………………
About Synergy Research Group
Synergy provides quarterly market sizing and segmentation data on networking, IT and cloud-related markets, including company revenues by segment and by region. Synergy Research Group ( www.srgresearch.com ) helps marketing and strategic decision makers around the world via its unique insights and in-depth analytics.
To speak to an analyst or to find out more about how to access Synergy’s market data, please contact Heather Gallo @ [email protected] or at 775-852-3330 extension 101.
Press Release:
