CenturyLink and 911 National Downtime [was: ATT Crash]
https://www.reddit.com/r/networking/search?q=centurylink&sort=relevance&t=week https://www.reddit.com/r/networking/comments/a9z6tb/centurylink_outage_west_... https://www.reddit.com/r/technology/comments/aa030y/centurylink_customers_re... https://www.reddit.com/r/centurylink/comments/a9zwkx/centurylink_down_all_ov... https://downdetector.com/status/centurylink https://status.ctl.io/ https://i.redd.it/dsd0r48tnw621.png " Spoke to CenturyLink this morning about a downed 10G LAN PHY in WA, the guy I talked to said they had a major DWDM outage causing problems all over the west side of the country. Their own internal ticketing system is currently down, ControlCenter is unreachable from various locations on the west side, all kinds of problems. They were working with their DWDM vendor, weren't able to give me an ETR. 6:30pm Eastern, I've repeatedly tried to reach 3rd level escalation lines xxxxxxxxx or xxxxxxxx however both numbers were flooded and indicated all services are unavailable upon each attempt and ultimately was unable to get through. Called the 4th level escalation lines xxxxxxxx or xxxxxxxx and spoke with Director of operations - who indicated that they still do not have an ETA, however assured me that they have engaged onsite hardware vendor support on all fronts to solve the problem and restore service. Efforts to regain visibility to sites in Atlanta, GA and Chicago, IL remain ongoing. Once visibility has been restored the filter will be applied to limit communication traffic between sites which was causing CPU spikes that in turn prevented the devices from functioning properly. ​From the call I'm on, I've been told they had lost complete management access due to these issues and had to wait for someone to be on site to resolve it. That's pretty extreme. "
participants (1)
-
grarpamp