Comcast frequent, intermittent internet outages + long outage in Santa Clara, CA with no auto-recovery!
For over one year now, many U.S. Comcast customers have experienced frequent, but short Xfinity internet and pay TV outages several times per week.
Here is what two customers wrote on the Xfinity Community Forum:
- Outages being reported CONSTANTLY: “I have intermittent outages several times a day for over a month. I work from home and this is inconvenient as I have been kicked out of important meetings and even streaming movies at night is a challenge with these frequent outages.”
For the last month, my Xfinity internet service has REPEATEDLY been dropping, and outages of over 100 people have been getting reported on Xfinity’s website.
It was every day for 10-14 days, then stopped for about a week, and now it’s back to every other day. I contact tech support, and they do the “reset your modem” dance, but inevitably reach the end of their binder of answers. I’ve had two techs out, one told me I needed a new modem, didn’t fix the problem.
But inevitably, a little while after I notice problems, I check Status Center and see there is an outage in my area due to “Network Damage”, affecting 100-500 people. It gets fixed…then in a day or two, it’s dropping me again,and another “Network Damage” outage affecting 100s of people.
Is there a local or regional rep I can speak to who can explain to me why this service I pay for is suddenly the victim of repeated outages every day/every other day due to “network damage”?? I haven’t had issues in years with service, and now I don’t know if this is a coordinated terrorist attacking on the network, the squirrels have declared war on the wires, or what is causing repeated failures. I had more than one person at Xfinity tell me that their techs are just unplugging people in the middle of the day to do network upgrades. My wife and I both use the Internet for work, we are losing money when this happens.
Can anyone recommend next steps for obtaining more information on why Comcast can’t go more than 48 hours without an area outage for the last month?
2. From a Seattle, WA Xfinity internet customer posted on Reddit:
“Looks like there’s a widespread Comcast internet outage. In the image below, Each of the dots represents 500-2000+ reports of internet problems. I know Comcast had some scheduled overnight maintenance to “upgrade the systems,” which has now been updated to “damaged network with no ETA for fixing.”
A much longer Xfinity outage report on Reddit:
“Service out for 36 hours, Xfinity keeps blaming power outage. The only support I can receive is Bots via the chat or a prompt goodbye when I call the phone support. What gives?”
……………………………………………………………………………………………………………..
And there are many, many more social media reports of Xfinity outages, which also takes down the company’s nearby WiFi hot spots, which could be a backup when you lose Xfinity wireline internet service.
………………………………………………………………………………………………………………..
Personal Experience:
On Friday, June 21st, most Xfinity customers in Santa Clara, CA (including this author) experienced a severe and long service outage, which (for me) lasted 17 hours, It was due to a fiber backbone cut by vandals. Here’s one of many text messages I received:
“Hi, it’s Xfinity Assistant. We’re aware you’re experiencing an interruption due to damage to our fiber lines in your area. We’re still working to get you back up and running. We apologize for the inconvenience.”
That outage was a killer for me, as I could not participate in two webinars that morning and had to cancel a 3pm Zoom call with my Doctor (for which I had to pay for since I cancelled <24 hours before the scheduled session). Obviously, my four smart Amazon/Google speakers didn’t work, nor did either of my two Amazon internet TVs. I had to use my cell phone to make voice calls as VoIP was also down.
For three weeks, I’ve been urgently trying to get an explanation from Comcast for why the Santa Clara outage lasted so long. In particular, why wasn’t there protection switching (1:1 or 1:n), self healing rings, router restoration via a standby fiber facility available for auto-switch over of the backbone traffic?
As that did not happen, the fiber cable had to be manually repaired, which took a very long time. Despite endless voicemail tag, I never received an answer to that question.
The only written response was the following:
“As I think you know now, that outage was caused by vandalism to our network. Vandals inflicted severe, significant damage to our system that required extensive, complex and time consuming repairs to our fiber. On behalf of our entire team I want you to know how sorry we are that this caused your services to be down for a lengthy duration.”
Here is what the Fiber Optic Association recommends:
In the case of fiber optic network restoration, nothing is more important than having complete, up-to-date documentation on the network. If possible, design a network with backup options. Many users run dual links, one transmitting data and one “hot back-up” ready to switch over in milliseconds. Electronics must be installed with duplicate links and all power must be backed up with batteries, generators or fuel cells.
Critical systems should add in geographic diversity; two links available running paths that are as widely separated as possible to ensure that if one suffers a failure due to damage to the fiber optic cable plant itself, the other can be switched in immediately. Rings provide a logical way to have route diversity, but simply being able to patch fibers manually to switch over to another fiber/cable is still quicker than repair. Even with backup, a failure requires immediate restoration, as one should never depend on a single link any longer than necessary.
All cables should have spare fibers, especially since fiber is extremely inexpensive compared to installation or restoration costs. Fibers tend to get broken at the ends where terminated or inside splice closures during splicing or re-entry. Having spare fibers makes it easy to simply switch fibers to restore operation. Whenever possible, store extra cable in service loops that can be pulled together for splicing. This can save immense amounts of restoration time for cables installed indoors or pulled in conduit outdoors.
OSP underground cables should be buried sufficiently deep (~1m/3 feet) that it is protected from casual digging and marker tapes that show up on cable locators buried above them. (See OSP Underground Construction in the FOA Guide) Bright colored conduits also help visibility. Cables should be listed in the “Dial 811, Call Before You Dig” database and markers installed where possible.
………………………………………………………………………………………………….
This post will be updated if and when I get an answer from Comcast as to why it took so long to restore service after the fiber cut.
References:
https://www.thefoa.org/tech/ref/restoration/rest.html
https://en.wikipedia.org/wiki/Self-healing_ring
https://www.advsyscon.com/blog/self-healing-it-operations/
Great article!
I think that there is no doubt that operating an access network is challenging! Power outages and vandalism are big problems, although these can be mitigated with methods such as those described in the article.
I have never been a Comcast customer, but what I am hearing most often is that their customer support is abysmal. I understand that they may experience technical problems. It is their lack of transparency and their poor customer communications that seem to frustrate their customers the most!
Indeed, Comcast/Xfinity customer support is abysmal. When you call or text during an outage you never get an ETA for service restoration. The text messages take you on a circular route to nowhere. Examples:
1] Hi, it’s Xfinity Assistant. We’re aware you’re experiencing an interruption due to damage to our fiber lines in your area. We’re still working to get you back up and running. We apologize for the inconvenience.
2] Hi, it’s Xfinity Assistant.
Your service interruption should be resolved as soon as possible. We’re working quickly to get you up and running again and will text you once your service is restored.
When Xfinity customers report outages by calling 1-800-Xfinity (1-800-934-6489) and reach a live person, they are told “we don’t know when the outage will end” or they get a recorded announcement urging them NOT TO SPEAK TO AN AGENT because that won’t result in anything.
If you report the outage online at xfinity.com/xfinityassistant you get a run around of circular text messages where you don’t get any useful information.
Why is Xfinity customer service so incredibly POOR and horrible?
Problems started June 1st and after 9 tech support visits, supervisor visits, 9 cable cards, advanced repair team tickets, poor overseas support and script reading and the problem is still not resolved. I can’t believe Xfinity can get away with this. Yes you read that right. 3 months of craziness and the problem is still not fixed. No one to contact to express our concerns about the poor service. Ironically the billing works like clockwork and if you threaten to cancel your service you get a live person immediately. How convenient???
I’m beginning to get the feeling that Xfinity is only interested in the dollar and not the customer. Anything to avoid any commitment or ownership of any problem. It’s really very very pathetic!
And Xfinity wonders why they are losing internet and payTV customers? This is why! POOR CUSTOMER SERVICE!!! Should be a law against this. Come on Xfinity. Get your act together!
https://forums.xfinity.com/conversations/customer-service/why-is-xfinity-customer-service-so-incredibly-poor-and-horrible/64ef004ede75d267de8316dd
The Fiber Broadband Association’s tips are good ones that will help. Of course, if the outside plant is aerial, that makes the infrastructure that much more vulnerable. It is understandable that Comcast can’t have redundancy to the individual home (unless there was a counter-rotating ring fiber architecture to the dwelling, which doesn’t exist).
Still, one would think there would be fiber redundancy to their optical RF nodes. If the fiber cut was intentional, the perpetrators, assuming they are found, should be prosecuted to the full extent of the law. It might even rise to the level of domestic terrorism.
Note that there are wireless back-up options which Comcast/Xfinity evidently does not offer. They do have Storm-Ready WiFi provides which provides unlimited cellular data and a rechargeable battery backup to keep your WiFi connected during power outages, but not for fiber backbone cuts. All the local Xfinity WiFi hotspots are down during such a fiber cut
The introduction of 5G-Residential Gateways with fail-over capability will offer the potential to improve the customer experience when fiber cuts occur. Will Comcast/Xfinity deploy it?
Here are Xfinity’s Gateways which are all based on home WiFi with no cellular backup when wireline internet service fails:
https://www.xfinity.com/support/articles/broadband-gateways-userguides
Even the XFI FIBER GATEWAY fails to have a backup route to the internet if any part of the fiber (access or backbone network) is cut.
“This WiFi gateway/MTA solution provides complete home connectivity and coverage at high speeds at our Fiber-to-the-Unit properties.”
Terrible service and terrible customer service:
We have experienced more outages with xfinity than ant other internet provider we’ve ever had. The customer service bot is awful and the fact that I can’t even get a polite full sentence response is gobsmacking/utterly astonishing!
I spoke with a Customer Service agent about the constant outages and the no warning or even notification of the outages and all I got was ” there’s a outage detected in your area” “team working on it”. Like of course there’s a outage in our area, why else would I be complaining about constant outages and asking for compensation?!
When I asked for a time frame I got a response of “4am”. At this point I was so annoyed with the customer service that the constant outages wasn’t even on the top of my list.
The whole company is ridiculous. Paying top dollar for unlimited highest internet package and we can’t even have reliable service or even decent customer service.
https://forums.xfinity.com/conversations/customer-service/terrible-service-and-terrible-customer-service/6559a4ef77f8594310301200
……………………………………………………………………………………………
Outages/billing/terrible customer service:
I have had nothing but trouble trying to get an update on at least an eta on when my internet service will be restored. When I finally got connected to a chat representative he completely stopped responding when I asked about the outages and billing. I’m sure you can understand how irritating that is!
So I would like to ask again…. Is there anyway I can get an eta when my service will be restored and is my bill getting prorated for the amount of time I am without service?
https://forums.xfinity.com/conversations/customer-service/outagesbillingterrible-customer-service/64977947ce76231eba55173c
Despite several emails to Comcast/Xfinity execs, there was no reply or any explanation for why the June 21st Santa Clara, CA internet outage lasted so long. One must conclude that Comcast/Xfinity had no auto recovery/standby facility backup when their fiber optic backbone was cut by vandals.
Failover within a communications network is the process of transferring tasks from a failed component to a similar redundant component to avoid disruption and maintain operations. Automated failover is the ability to automatically reroute data from a failed component such as a server or network connection, to a functioning component and is essential for mission-critical systems.
From Excesa – EVERYTHING YOU NEED TO KNOW ABOUT NETWORK FAILOVER:
Entire network architecture, or individual network components, may be configured as a cold spare (manual failover, requiring human intervention), hot spare or High Availability (HA) pair (automatic failover, with a short delay) or fully redundant (synchronized disaster recovery site). The three critical elements requiring failover configuration are power, network connectivity and server capacity.
The capability to switch automatically to a redundant or standby system or network upon failure happens without human intervention (see Failover Hierarchy for other types of failover). Automated failover is essential in servers, systems or networks requiring continuous availability and a high degree of reliability — those responsible for mission-critical processes and data
https://www.ecessa.com/technology-briefs/everything-you-need-to-know-about-network-failover/
Multiple Outages Every Day:
We have been having outages every day, multiple times a day for the past month or so. Starting around 9AM til 9PM. Loss of signal on the modem. Usually lasts anywhere from 2 minutes to 3 hours. I have called multiple times, once I was told there was some work and they have constantly “closed my tech visit” for me because of this. Some have tried to blame the modem (which is mine) but this only happens during the day time hours. Its completely stable after 9PM or so. What work is being done, and when can we expect to have a reasonably stable Internet connection again? Central Florida; 34715 User_85960
https://forums.xfinity.com/conversations/your-home-network/multiple-outages-every-day/649b6fc7ce76231eba55921e
TEXT MESSAGE TODAY: Hi, it’s Xfinity Assistant.
We’ve identified a network performance issue that may be affecting your Xfinity service and needs to be repaired right away. We’re actively working to fix it by 08/16/2024 10:10 AM (Pacific). We’ll do our best to minimize disruption in the area. During this time, you could experience a service interruption in Santa Clara, CA. We apologize for the inconvenience and will text you when the issue is resolved.
Hi, it’s Xfinity Assistant.
Your service interruption should be resolved by 08/16/2024 10:10 AM (Pacific). We’re working quickly to get you up and running again and will text you once your service is restored.
At 10:20am another text:
We’re sorry your service isn’t back up. We recommend restarting your TV box, Gateway or both. To restart, unplug your device, wait 12 seconds, and plug it back in. Wait 10 minutes for your device to come back online. If it still doesn’t work, reply “Internet issues”.
Starlink subs report seeing fewer outages than cable – A Recon Analytics study shows that Starlink customers experience fewer outages than cable subs. The majority of Starlink subs are in rural areas and come from DSL providers, but 11% are completely new to home Internet.
Starlink subscribers tend to see fewer outages when compared to those with cable broadband, but see more outages than fiber broadband subs, according to a new study from Recon Analytics. Specifically, 30% of Starlink subs said they experienced an Internet outage over a period of 90 days, versus major fiber providers (24%), large FWA operators (25%), major DSL service providers (33%) and large cable operators (39%).
“Starlink customers also tell us that they experience near industry-leading speed consistency with the most reliable router,” Roger Entner, founder and lead analyst of Recon Analytics, explained in the emailed report.
Recon Analytics based its findings on data from 1,300 Starlink customers that were part of a larger pool of 153,000 respondents to a customer satisfaction study between May 12, 2023, and July 5, 2024.
Given Starlink’s focus on areas that have no or inadequate access to wireline broadband, it was unsurprising that the study found that some 85% of Starlink respondents said they are in rural areas, versus 5% in suburbs and 10% in zip codes classified as urban areas.
Additionally, the study found that 11% of Starlink customers are new to home Internet, but the largest contributors of subscribers among incumbent wireline operators are CenturyLink/Lumen, Charter Communications and Frontier Communications. The bulk of Starlink customers originate from DSL service providers or other satellite service operators that aren’t as competitive with speed and latency.
In May, Starlink said via X (formerly Twitter) that it has surpassed 3 million subs worldwide.
Starlink also scored well against other broadband operators/platforms in Recon Analytics’ tracking of component net promotor scores over a year-long period.
The analyst was also impressed with Starlink’s customer satisfaction metrics considering Starlink relies on direct line-of-sight to a passing low-Earth orbit (LEO) satellite.
“Starlink has been able to get 6,146 working satellites into orbit, providing significant capacity and reliability to its subscribers. It has also been able to manage bandwidth, even during peak hours. It is also clear that Starlink’s router is among the most stable in the market,” Entner explained.
https://www.lightreading.com/satellite/starlink-subs-report-seeing-fewer-outages-than-cable-study