Quicktake from Ethernet Tech Summit: Ethernet’s Changing Role in the Data Center

As you might expect, the the Ethernet Technology Summit, Feb 21-23, 2012 in San Jose, CA, explored the many ways in that Ethernet is everywhere.  In particular:

• Broadband access: Ethernet over Copper is Ethernet over VSDL or ADSL,  EPONs have been deployed in Japan, Korea, China and elsewhere in Asia

• Transport:  100GE being deployed now in carrier exchanges and long haul networks; Ethernet over OTN (OTU3/4)

•  Cloud Computing:  MEF specified “Connection Oriented Ethernet” for WAN to deliver Private Cloud services to enterprise customer premises

 • Wireless:  Mobile backhaul, Wi-Fi

• LAN:  Ethernet origination in workgroup LANs to campus LANs to metro LANs and Virtual Private LANs

• Data Center:  Enhanced Ethernet with higher speeds (10GE and 40GE) on the horizon

Here are a few Data Center (DC) Trends:
• Server Consolidation and virtualization
• Leading to very large Data Centers
• 600K+ virtual servers in a Data Center
• Scale to 1 million networked devices in a facility
• Efficiency drives many decisions
• Few percent improvements can save $ millions
• Need to scale compute, network, and storage efficiently
• Operations Expenses driving decision
• Power is an important metric
• Need for consolidated management

DC Efficiency Drivers:
• Data center optimized servers
• Eliminate unnecessary components
• Higher efficiency cooling
• Standardized servers for low cost
• Google servers , Facebook Open Compute, Microsoft’s server concept
• Flatter networks for better scaling
• TRILL/Rbridges, Shortest Path Bridging
• Standard interfaces with OpenFlow
• Virtual data centers
• Workload migration between data centers
• Network becomes a simple data backplane
• Separate intelligence

DC’s Focus on Power Efficiency:
• Many now track PUE (Power Usage Efficiency)
• Ratio of power used by IT (servers) to total facility power
• Facility includes cooling and power distribution
• PUE of 2.0 is average
• New facilities targeting PUE of 1.5 or below
• Also need to improve perf per watt
• Power is generally the limiting factor per rack and per data center
• More than a third of total cost of ownership (TCO) is proportional to electrical usage=power consumed
• Leads to Rising Demand for Power Efficient Processors

 

New Opportunity with 10GE:  LAN on Motherboard (LOM) =servers use 10GBaseT component on PC motherboard for access link to DC 10GE Switch.  Expected to happen in 2013-2014.

Source:  Jag Bolaria,  Analyst at Linley Group


Another view of the Data Center:
– 1GE links are saturated and won’t grow much
• 80% of today’s server connections are 1 Gigabit
• 1000BASE-T dominates due to cost, flexibility, low-power
• 10G used primarily for uplinks on TOR switches & aggregation

Today’s Data Center 
– 10G Growing
• ~80% of today’s server connections are 1GbE
• ~20% of today’s server connections are 10GbE
• 10GbE price erosion driving volumes up
• ~200% YOY 2010 (Source Infonetics “Quarterly Worldwide and Regional Market Share, Size, and Forecasts”)

• Data center strategies are diverging
• Blade servers allow higher density, reduce
cabling, higher performance, higher efficiency
• Moving to 10GBASE-KR 
• Rack servers demanding 10G LOM/Mezzanine
• Dominated by SFP+ DAC
• Moving to 10GBASE-T due to cost, flexibility
• Virtualization leading to higher density, higher utilization of servers 
• More focus on energy efficiency
• Blade servers ahead on 10G deployment
• Rack servers trailing but growing faster

Tomorrow’s Data Center – 40GbE and New Architectures:
• Blade Servers moving to 40GbE swiftly
• Virtualization, Convergence, Efficiency Demands will lead to optimization of resources, higher utilization
• Demand for 100GbE aggregation is going to rise accordingly
• Existing 10x10G solutions are bulky, expensive, power hungry


In Thurs AM panel session on Connecting the Next Big Wave of Cloud Computing Infrastructure, Kamal Dalmia of Aquanta Corp said that “10GBASE-T is driving the growth and deployment of 10GE in Data Centers and Cloud Computing.”  The crossover, where 10GE shipments surpass 1GE shipments is expected to be late this year (2012).  10G BaseT is expected to enjoy exponential growth starting in 2013, while 1GE shipments are projected to decline sharply at that time.

For Ethernet Switches
• Cost: 10GBASE-T priced same as emptyoptical socket
• Backward Compatibility
–10G/1000/100BASE-T over RJ45
• Power: High density line cards possible
• Ubiquitous: UTP cable is everywhere (UTP category 6A has been installed in most Data Centers since 2008)

For Servers
• LAN on Motherboard (LOM) economics
–Knife switch transition at 3x price per port of GE
• Backward Compatibility
• Incremental Upgradeability in Data Centers

Summary: 10GBASE-T is driving the 10GE growth
• 10GBASE-T achieves lower cost, lower power, lower latency and higher densitythan Gigabit Ethernet on a per Gbit/s metric
• 10GBASE-T LOM servers start shipping this quarter
• 10GBASE-T switches available from major OEMs