IDC Directions 2019: Autonomous Infrastructure and the Evolution of the Self-Driving Network
Abstract (by Rohit Mehra, IDC Analyst):
Network transformation is well on its way with the evolution of SDN and SD-WAN, leading to flexible network architectures taking hold from the cloud to the enterprise edge, powered by intelligent automation. Increasing use of streaming analytics and pervasive visibility, enhanced by ML and AI, is creating a next-generation, agile network that self-remediates performance issues and proactively responds to security threats. The result will be greater operational efficiencies, improved user experience, and verified SLAs that ensure delivery of meaningful business outcomes. The network is a foundation layer for enabling secure, scalable and efficient use of Cloud, Edge and IoT Applications.
……………………………………………………………………………………………………………………………………………………
Please see comments below this post for Alan’s thoughts on Rohit’s presentation at IDC Directions 2019.
Network Requirements Continue to Expand:
▪Fast and adaptive
▪ Capacity on-demand
▪ Edge-to-Cloud Latency
▪ Network-level security
▪ Analytics capable of yielding new insights and driving digital transformation (DX)
▪ Bridge cloud and telco domains
▪ Global reach
“Self-driving” Networks are now needed to be Automated, Orchestrated and Optimized Network System. Traditional networks break down as they scale (get larger) and increase workloads, making automation essential in future networks, e.g. 5G.
………………………………………………………………………………………………………………………………………………………………………….
Definitions:
Network automation is a methodology in which software automatically configures, provisions, manages and tests network devices. It is used by enterprises and service providers to improve efficiency and reduce human error and operating expenses. Network automation tools support functions ranging from basic network mapping and device discovery, to more complex workflows like network configuration management and the provisioning of virtual network resources. Network automation also plays a key role in software-defined networking, network virtualization and network orchestration, enabling automated provisioning of virtual network tenants and functions, such as virtual load balancing.
Digital Transformation (DX):
IDC defines DX as the continuous process by which enterprises adapt to or drive disruptive changes in their operations, customers, and markets. Today, many businesses are implementing DX without success, and some fail entirely. In part, this is due to pervasive technology shifts that are changing how organizations transact business, address customer expectations, operate and secure products and services, and compete in the marketplace.
………………………………………………………………………………………………………………………………………………………………………….
IDC maintains that virtualization has matured from simple partitioning and encapsulation to leveraging the mobility of virtual machines to improve management and operations of IT environments. Virtualization 2.0 includes a host of new use cases that range from high availability and disaster recovery to hosted clients and true utility computing. Note that this information was not discussed by Rohit, but rather assumed to be known by the session attendees.
………………………………………………………………………………………………………………………………………………………………………….
Emergence of AI-based Network Automation:
✓ Simpler declarative management, enhanced verification and closed-loop processes
✓Network will accurately apply/enforce intent
✓Respond in near real-time to application, network and security events
Limitations of Classical Network Monitoring:
- Lack of pervasive, end-to-end visibility across physical/virtual/cloud
- Minimal application context
- Limited, spotty Analytics
- Frequently complex/costly
- Ping/SNMP/S flow/Trace route
- Unable to capture real-time network events
The Case for Streaming Telemetry In Support for Visibility and Analytics (see Alan’s Comment in box below article):
Infrastructure at Scale Demands Streaming Network Telemetry! Hyperscalers have deployed streaming telemetry extensively because it can provide millions of updates per second: Time stamped at source (for real time and network forensics)
▪ Event-driven, subscription-based
▪ Vendor support for standards based streaming telemetry • e.g., Cisco, Arista, Juniper, Ciena, Nokia
Optimization of apps/user experience (48% 0f respondents), plus security (41%) are top priorities for AI-enabled network automation:
IDC asked: What do you see as the most important aspects of an AI-enabled Network Automation solution? (Pick 3). Here’s the ordered ranking:
- Optimize and enhance application availability/performance and user experience 48%
- Implement security policies, including visibility into encrypted traffic 41%
- Work across multiple networks (on-premises and cloud-based) 38%
- Reduce cost and complexity of network operations 36%
- Simplicity in network deployment, management and operations 36%
- Incorporate streaming telemetry data for real-time visibility and insights 36%
- Leverage existing network infrastructure and/or software defined networking (SDN) deployment 35%
- Anticipate network outages and plan for network changes 30%
Automation at the Network Edge:
▪ Network platforms can leverage aggregate data from 1000s of deployments
▪ Crowd-sourced data is then dynamically applied to similar environments (anonymized)
▪ Benefits include dynamic scaling and mitigation of performance and/or security issues as they arise
Carrier Networks: The Automation Imperative – complexity across carrier networks continues to grow
▪ Multiple-generations of technology
• Ethernet, MPLS, Broadband IP VPNs
• 3G, 4G, 5G cellular
▪ Physical / Virtual
• VMs and Container based VNFs
• Evolution of Telco Cloud
▪ Cloud Aspirations
▪ Monetization Roadmap
The 5G Promise Is Not Achievable Without Significantly Enhanced Automation
▪ Network Slicing is key to delivering on the 5G promise (yet there are no implementable standards for network slicing; they are all proprietary implementations)
▪ Predicated upon automated provisioning, service chaining of cloud-native network components
▪ Automated traffic optimization across fronthaul, mid-haul and back-haul key to efficiency and customer experience
Security Analytics:
▪Traffic Analytics and Behavior Modeling
▪AI-enabled Anomaly Identification
▪Automated, Policy-based Remedial Actions (e.g. Quarantine)
AI-enabled Capacity Planning and Optimization:
▪ AI-powered network automation platforms monitor and assist with network capacity requirements and dynamically optimize flows
▪ Cloud-enabled Day 1 network provisioning and management automation that meets IT and business needs
Automating Enterprise Network Operations:
▪ AI-powered network operations create self-healing networks
• System monitors operations
• Detects performance degradations
• Determines root cause
• Automatically remediates the problem before it impacts users
▪ AI-powered Helpdesk Automation
• User Interfaces leverage Natural Language Processing for queries, e.g. Q: Why is my Wi-Fi coverage weak on the fourth floor? A: Switch to the 2.5Ghz Band
▪ Automated QoS and App Performance Guarantees
• Operator specifies minimum quality of service levels, system automatically maintains those in real time
• Resources are spun up to ensure and maintain service levels
Self-Driving Networks Require Closed-Loop Visibility and Automation:
▪ Self-driving networks will rely on streaming telemetry and closed-loop automation to detect and proactively respond to traffic-management issues and security threats
▪ Feedback loop from AI/ML to policy/intent will provide the ability to Visualize, Correlate and Predict- key ingredients for automation
▪ Requires a robust eco-system of network, visibility/analytics and AI solutions, SI/SPs
Take a Pragmatic Approach to Network Automation:
❖ Pick the right network automation use case(s)
❖ Getting automation right is mission critical
❖ Ensuring Clean, Relevant and Secure Data will be foundational to building AI-enabled network automation
❖ Developing Skills for Network Automation will be key to success
❖ Vendors can do their part by making products simpler to consume, deploy, manage
Final Thoughts on Network Autonomy:
- Journey has begun: We are now at the cusp of major advances thanks to areas such AI, visibility and analytics, streaming telemetry, etc.2
- Broad Applicability: Autonomous Networks will extend from Cloud to the Enterprise/IoT Edge, and will also be foundational to 5G Rollouts
- Augment, not Replace IT: AI-enabled network automation augments human capabilities
- Be Judicious: Move forward judiciously, with caution, leveraging automation lessons from other IT domains
Rohit Mehra: [email protected] +1 (508) 935-4343
2 thoughts on “IDC Directions 2019: Autonomous Infrastructure and the Evolution of the Self-Driving Network”
Comments are closed.
Of all the new paradigm items mentioned by Rohit, streaming telemetry is the one I’m most positive about. Streaming telemetry is a push-based mechanism that removes the inefficiencies associated with polling. The required data is streamed automatically and continuously from network devices to the management system.
Traditional methods of collecting network telemetry data include “pull” based mechanisms such as the SNMP protocol, CLI show commands, and syslog messages. All of these have inefficiencies that inhibit a network engineer’s ability to plan, deploy, and assure services.
With digital transformation initiatives now dominating IT discussions and plans, software-defined networking and virtualized network services and apps are clearly on the rise. SNMP simply can’t keep up with the speed and scale required with today’s new massive scale networks.
Rather than being initiated at the management tool and reaching down into devices via polling, streaming telemetry originates at the device and maintains a mostly northerly track up to one or more systems that subscribe to the telemetry stream. It uses a push-based model to continuously stream the needed performance data from devices to management systems.
The continuous nature of the data flow eliminates the SNMP-style polling of devices altogether. Essentially, it’s a one (device) to many (management tools) publishing model. Gone are the endless repetitive requests from management tools to devices. And since most organizations use more than one management tool, streaming telemetry eliminates the polling requests that were coming from multiple management tools, thereby driving exponential reductions in overhead.
Instead, streaming telemetry uses a policy-based approach, enabling the devices to know what data is required, the frequency of collection, and where the data needs to be sent. Streaming eliminates the inefficiencies of polling and is the mechanism that gives network operators the operational data they need to stay on top of next-gen networks and infrastructures. For IT and NetOps teams, this real-time, analytics-ready data helps them to work more effectively across a range of areas including network automation, traffic optimization, preventative interventions, and faster and more effective incident remediation.
References:
https://www.packetdesign.com/blog/network-basics-by-packet-design-what-is-streaming-telemetry/?cn-reloaded=1
https://www.networkcomputing.com/networking/streaming-telemetry-has-arrived-why-you-should-care
At IDC’s annual Directions conference in Boston on March 12, Mehra said SD-WAN has seen an incredible amount of market growth since 2016, and the natural progression of network evolution inevitably leads to the edge. Both SD-WAN and the edge rely on one another. The edge requires a dynamic WAN, and SD-WAN can benefit edge use cases — the two are interlinked, like a chicken-and-egg scenario, he said.
Enterprise networking will see major changes and advances throughout 2019 due to other technologies, as well. Some of the major networking trends include 5G, 802.11ax — or Wi-Fi 6 — and Gigabit Ethernet (GbE). Overall, automation and its real-time capabilities play significant roles in all these rollouts.
5G and 4G LTE. According to Mehra, 5G is approaching quickly. Select cities worldwide will be lit up with 5G by the end of 2019, while businesses will roll out 5G in experimental phases focused on 5G fixed wireless, he said.
Wireless 5G — with its broadband-like capabilities — and the addition of automation will make 5G a “major, once-in-a-lifetime event,” Mehra said. Automation adds speed and agility to 5G capabilities, which make it a compelling use case for businesses. But widespread 5G rollouts aren’t realistic this year, he added.
“While we wait for full-blown 5G, the use of 4G LTE by the enterprise is becoming more widespread,” he said.
Over the last few years, 4G LTE has evolved, allowing it to mature into a technology that could compete against traditional business-class wireless technologies.
The mobile consumer side likely won’t see the benefits of 5G until 2021.
“For users to benefit from 5G, everyone needs a 5G-enabled handset,” Mehra said. “We’re two or three years from that.”
Network edge. The edge has grown in importance, as increased network complexity requires improved network visibility. End-to-end visibility requires automation — whether it’s used at the enterprise edge, the IoT edge, the cloud edge or the compute edge — and is crucial in ensuring networks operate properly in several different use cases, according to Mehra.
“If you look at healthcare, utilities, mining and manufacturing, all of these IT-related actions happen at the edge,” Mehra said. “For that edge to succeed, you need a more dynamic WAN and connectivity.”
Automation enables dynamic WANs that respond in real time and enhance customer experience at the edge. Automation coupled with 5G for wired or wireless networks can benefit the edge due to 5G’s long-distance communication capabilities, Mehra added.
Another technology that benefits the network edge is SD-WAN, which is a dynamic WAN itself.
SD-WAN. SD-WAN is a building block toward an autonomous network, Mehra said, but automation also applies to SD-WAN. The SD-WAN market is growing, and while it hasn’t yet reached mainstream, its time is fast approaching. The need for automation has also advanced this growth, as automation can be built into the software.
“As the market moves to a more holistic approach to SD-WAN, we feel it’ll be an enabler of newer edge architectures,” Mehra said. “While SD-WAN looks at the conduits for connectivity, wide area networking is the pipe that connects you.”
SD-WAN and automation allow for a more dynamic edge, which closely connects these networking trends.
Wi-Fi 6 and 400 GbE. Also known as 802.11ax, the new Wi-Fi standard’s projected release date is late 2019. Wi-Fi 6 promises real-time responses and aspects, as well as multiuser-MIMO and orthogonal frequency-division multiple access (OFDMA).
“On a client-by-client basis, using MU-MIMO and OFDMA, Wi-Fi 6 provides a best-in-class experience and automatically makes changes in its own architecture to support that,” Mehra said. “Think self-driving networks coming to Wi-Fi.”
Although Wi-Fi 6 touts agility, scalability and real-time responsiveness, Mehra said multi-GbE will suffice for enterprises at this point.
“The 400 GbE use case is clear for the data center, but I think multi-gig to 100 GbE is more than sufficient in the enterprise, even with Wi-Fi 6,” he said.
Most 400 GbE deployments will begin in 2019. The 400 GbE switches will start as spine or core switches in hyperscale data centers and as leaf or spine switches in private and public cloud data centers, Mehra said. Wi-Fi 6 and 400 GbE will not become mainstream until 2023.
https://searchnetworking.techtarget.com/feature/Automation-connects-5G-Wi-Fi-6-and-other-networking-trends