• The Fragmented Edge vs. Centralized Fortresses: Operators are realizing that centralized hyperscale data centers (like AWS or Azure clusters in Virginia) cannot support latency-sensitive “Physical AI” or real-time agentic workflows. To make AI-native networking work, carriers must deploy high-density compute racks directly at the network edge, a highly complex and capital-intensive roll-out.
• Neutral Interconnection Hubs: Multi-cloud setups and distributed training workloads are putting immense pressure on backbones. The expansion rate of neutral interconnect hubs (like Equinix and Digital Realty) is directly gating how fast enterprises and telcos can orchestrate data between fragmented training clusters and edge inference nodes.
• Rack-scale architecture is rapidly emerging as the primary deployment unit as enterprises transition from discrete servers to fully integrated systems capable of supporting the power density, thermal constraints, and interconnect requirements of production-scale AI workloads.

Image Credit:  AMD

……………………………………………………………………………………………………………………………………………………………………………………………

AI data centers supporting telecom networks require fundamentally different power and cooling infrastructure compared to legacy enterprise facilities. The transition to generative AI and real-time edge processing has pushed power density per rack from an average of 5–10 kW up to 40–100+ kW.  Dell Technologies Inc. has been strategically aligning its portfolio to this shift, and at Dell Technologies World 2026, the company introduced an expanded PowerRack portfolio that integrates compute, networking, and storage within a unified rack-scale platform. This evolution underscores a broader transition in system design priorities—from server-centric architectures to tightly coupled, rack-level systems—driven by the escalating demands of AI infrastructure. As Arun Narayanan, senior vice president of compute and networking product management at Dell, indicated, increasing power density and system complexity are making rack-level architectural optimization not just advantageous, but essential.

“Go back two years ago, the largest, most powerful rack was 80 kilowatts,” Narayanan said. “Come to Vera Rubin, you’re going to get racks of 235 kilowatts, and then get to the next generation of Rubin Ultra and Kyber, you’re going to very quickly get to one megawatt racks. You have to fundamentally redesign everything from power distribution to cooling.”

• • Medium-Voltage Power Distribution: Traditional facilities step utility power down to \(480\text{V}\) AC far from the rack. High-density AI data centers run medium-voltage  or  power directly down to the row or container level before stepping down. This minimizes conduction losses through the heavy copper busbars.
  • The Move to 48V DC Busbars: Within the server chassis, power shelf architectures are shifting from traditional \(12\text{V}\) DC distribution to DC busbars. A  delivery architecture reduces the current  required to deliver the same wattage  by a factor of four. Because resistive power loss follows the formula: \(P_{\text{loss}}=I^{2}R\) cutting the current to one-fourth reduces internal rack heat and conduction power losses by 93.75%
  • Grid Interconnection and Substation Constraints: A single rack-scale AI cluster (such as a cluster of 32 or 64 interconnected nodes) can easily pull 2 to 3 Megawatts (MW). Operators are bypassing traditional local distribution grids entirely. They are building dedicated on-site substations tied directly to transmission-level lines to guarantee upstream capacity.

       [ Liquid Cooling Architectures for AI Racks ]
       
 ┌───────────────────────────┐      ┌───────────────────────────┐
 │       Direct-to-Chip      │      │     Immersion Cooling     │
 ├───────────────────────────┤      ├───────────────────────────┤
 │ Closed loop micro-channels│      │ Entire server submerged   │
 │ bolted directly onto GPUs │      │ in dielectric fluid tank  │
 │                           │      │                           │
 │     [ GPU ] ──► [ Liquid] │      │    ┌───┐ ┌───┐ ┌───┐      │
 │   Cold Plate   Coolant    │      │    │GPU│ │CPU│ │RAM│      │
 │    Circuit     Circuit    │      │    └───┴─┴───┴─┴───┘      │
 └───────────────────────────┘      └───────────────────────────┘

• • Direct-to-Chip (Cold Plate) Cooling: This is the primary architecture for 2026 deployments. A closed-loop copper block with micro-channels is bolted directly onto high-thermal-flux components like the GPU or CPU. A specialized dielectric or water-glycol fluid circulates through the block. This absorbs heat directly from the silicon via conduction and pumps it away to a secondary heat exchanger.
  • Immersion Cooling (Single-Phase and Two-Phase):
    • • Single-Phase: The entire server blade is submerged in a bath of non-conductive, hydrocarbon- or synthetic-based dielectric fluid. The fluid circulates through the chassis via natural convection or pumps to remove heat.
      • Two-Phase: The dielectric fluid has a low boiling point (\(50^{\circ }\text{C}\)). The heat from the chips boils the fluid into a vapor. The vapor rises to a condenser coil at the top of the sealed tank, condenses back into liquid, and falls back into the pool. This utilizes the latent heat of vaporization, making it highly efficient.

  • Cooling Distribution Units (CDUs): High-density loops rely on CDUs to act as the barrier between the internal facility water loops (which can be lower quality) and the ultra-pure, treated water circuit flowing directly through the server cold plates.

China vs U.S.: Race to Generate Power for AI Data Centers as Electricity Demand Soars

Big tech spending on AI data centers and infrastructure vs the fiber optic buildout during the dot-com boom (& bust)

Will billions of dollars big tech is spending on Gen AI data centers produce a decent ROI?

AWS to deploy AI inference chips from Cerebras in its data centers; Anapurna Labs/Amazon in-house AI silicon products

Analysis: Cisco, HPE/Juniper, and Nvidia network equipment for AI data centers

Networking chips and modules for AI data centers: Infiniband, Ultra Ethernet, Optical Connections

Lumen Technologies to connect Prometheus Hyperscale’s energy efficient AI data centers