Is the “far edge” a bridge to far to cross for AI inferencing? What about “Distributed AI Grids”?

How Far is the Far Edge?

As major telcos size up distributed edge sites for a possible AI inferencing model, they’re trying to determine how far out the right place is in their networks to invest in AI computing capacity.  According to Light Reading, the “far edge” is a divisive option for inferencing. According to Omdia, owned by Informa, the Far edge includes: radio access network (RAN) cell sites, aggregation hubs, exchange offices, optical line terminal (OLT) nodes, and Tier 2 metro hubs. 

Many telcos are struggling to define how far is the edge from customer premises and how to serve various use cases with compute and intelligence?  It seems that 5G SA core with network slicing would be mandatory to support multiple unique use cases, each with different QoS requirements.

According to Omdia’s Telco Edge Computing Survey last year, just 15% of telcos ranked network far edge as the top location for where most AI inferencing will take place, while even less (11%) said the network near edge would be the main spot (which includes central offices, headend sites and large telco data centers). The results showed AI inferencing is expected to be handled mostly on the end devices themselves and at the enterprise edge (e.g., offices, campus or manufacturing sites).

Kerem Arsal, Omdia senior principal analyst for telco enterprise and whoIe sale, predicted in a research note that this year will see telcos split into camps of “believers” and “doubters” of the far edge. 

Image Credit:  Sphere

…………………………………………………………………………………………………………………………………………………………………………………………………………………..

AT&T VP Yigal Elbaz, speaking at the recent New Street Research and BCG Global Connectivity Leaders Conference, expressed a cautious view on AI compute at the “far edge,” questioning how far the edge truly needs to extend to serve specific use cases effectively.  He said the following (Source: Light Reading)

“The proliferation of compute and high-performing compute across the nation, in all metros is just happening, with a software layer on top of this [and] with the tools that developers need. So, I am not sure that there’s much value in extending that compute all the way to the far edge just to save another millisecond or two milliseconds of latency.”

“AT&T’s fiber and wireless networks can provide the “deterministic experience” needed between any new use cases and help them to “intelligently connect to the right model that they use, the context or the infrastructure that they need because that’s going to be heavily distributed across the US.”

“There’s no doubt that that AI is going to be embedded into wireless networks, and we’re going to call it AI-native and combine the physical space with the intelligence of the network. This is all true,” said Elbaz.

………………………………………………………………………………………………………………………………………………………………………………………………………………………..

Distributed AI Grids:

At this year’s Nvidia GTC event, AT&T was cited as a lead collaborator in the development of distributed AI grids—a geographically dispersed, interconnected fabric designed for high-performance AI infrastructure. In partnership with Cisco and Nvidia, AT&T is architecting an enterprise IoT AI grid focused on localized inference. By moving the compute layer to the network edge—potentially via On-Premises Edge (oPE)—the architecture aims to minimize backhaul latency and process workloads at the data source. Current Proof of Concept (PoC) deployments include a public safety framework and an edge AI-powered video intelligence pilot for site security. Similarly, Comcast is trialing Nvidia GPU-accelerated edge nodes to support deterministic, low-latency AI applications.
For the Cisco AI Grid with Nvidia architecture used by AT&T and Comcast, the interconnect strategy moves beyond standard backhaul to a specialized, deterministic fabric designed for distributed AI inference. AI Grid Interconnect Stack: The architecture leverages a multi-layer protocol approach to ensure low-latency, secure communication between edge nodes and the core:
  • Ethernet with RDMA (RoCE): The foundation is built on Nvidia Spectrum-X Ethernet, which utilizes RDMA over Converged Ethernet (RoCE). This allows for direct memory access between edge GPUs (e.g., Nvidia RTX PRO 6000 Blackwell Server Edition) and the network core, bypassing CPU overhead to achieve near-line-rate performance.
  • Scale-Across Networking: Using Nvidia Spectrum-XGS, the architecture extends standard RoCE to scale across geographically distributed sites. This creates a unified “AI Factory Grid” where remote edge nodes function as a single, programmable compute substrate.
  • Silicon One Routing: Cisco’s Silicon One-based routing is utilized for AI-optimized traffic management, providing the high-speed, high-density throughput required for token-intensive inference workloads.
  • Zero Trust & Secure Pathways: The interconnect includes a Zero Trust security layer embedded directly into the fabric. It utilizes localized traffic breakout and policy-enforced pathways to ensure that sensitive IoT and video data (such as public safety feeds) remain within the customer’s secure domain at the network edge.
  • Orchestration Control Plane: A workload-aware control plane manages these protocols to intelligently route tasks based on real-time KPIs (latency, cost-per-token, and data sovereignty), ensuring that “mission-critical” inference happens at the optimal node.
Focusing specifically on interoperability, the primary concern with a single-vendor AI Grid is the risk of architectural silos that could undermine years of industry progress toward Open RAN and multi-vendor environments.Key interoperability risks for carriers include:
  • Proprietary Software Lock-in: Integrating network functions into a proprietary ecosystem (like Nvidia’s CUDA or AI Aerial) can create a “subscription trap,” where software is inseparable from specific hardware, making it nearly impossible to swap vendors without a total architectural overhaul.
  • Data Fragmentation: Deploying AI across a distributed grid often leads to fragmented data sets across legacy and new multi-vendor platforms, which can result in inaccurate AI models and increased operational complexity.
  • Standardization Lag: While industry bodies like the GSMA are pushing for Open Telco AI standards, the rapid deployment of proprietary AI systems often outpaces these frameworks, leading to entrenched, incompatible systems that require significantly more resources to reconcile later.
  • Integration with Legacy Systems: Modern “agentic AI” and AI-native stacks often struggle to orchestrate processes across siloed legacy infrastructure, creating rigid operational environments that prevent the seamless flow of data needed for automated network troubleshooting.

Bottom Line: While the AI Grid may offer a more viable roadmap than AI-RAN, there is insufficient industry discourse regarding the strategic risks of a global, geographically distributed computing platform—as Nvidia defines it—reliant on a single-vendor hardware stack. Although Nvidia currently maintains undisputed market dominance, historical precedents such as Intel serve as a cautionary tale; long-term dominance is never guaranteed, and even market leaders face potential obsolescence. Furthermore, Nvidia’s practice of providing capital injections to entities that subsequently re-invest those funds back into Nvidia’s own ecosystem raises significant concerns regarding market sustainability and long-term financial health.

……………………………………………………………………………………………………………………………………………………………………………………………………..

References:

https://www.lightreading.com/ai-machine-learning/at-t-cto-casts-doubt-on-ai-compute-at-the-far-edge

https://www.lightreading.com/5g/nvidia-lines-up-ai-grid-as-orange-cto-echoes-the-ai-ran-doubts

Edge AI Computing Explained: Key Concepts and Industry Use Cases

Will “AI at the Edge” transform telecom or be yet another telco monetization failure?

Analysis: Edge AI and Qualcomm’s AI Program for Innovators 2026 – APAC for startups to lead in AI innovation

Private 5G networks move to include automation, autonomous systems, edge computing & AI operations

Nvidia AI-RAN survey results; AI inferencing as a reinvention of edge computing?

Nvidia’s networking solutions give it an edge over competitive AI chip makers