Global internet health check and network outage report
The reliability of services delivered by ISPs, cloud providers and conferencing services (such as unified communications-as-a-service) is critical for enterprise organizations. ThousandEyes monitors how providers are handling any performance challenges and provides Network World with a weekly roundup of interesting events that impact service delivery. Read on to see the latest analysis, and stop back next week for another update. Additional details available here.
Internet report for August 7-13
ThousandEyes reported 171 global network outage events across ISPs, cloud service provider networks, collaboration app networks and edge networks (including DNS, content delivery networks, and security as a service) during the week of August 7-13. That’s the same as the week prior. Specific to the U.S., outages decreased from 63 to 59, down 5%. Here’s a breakdown by category:
ISP outages: Globally, the number of ISP outages increased from 94 to 105, up 12%, and in the U.S. they decreased from 31 to 29, down 12%.
Public cloud network outages: Globally, cloud-provider network outages increased from 16 to 17, and in the U.S., they doubled from seven to 14.
Collaboration app network outages: Globally, collaboration app network outages tripled from two to six. In the U.S., there was one collaboration-app network outage, up from zero the week prior.
Two notable network outages
On August 10, NTT America, a global Tier 1 ISP and subsidiary of NTT Global, experienced an outage that impacted some of its customers and downstream partners across multiple regions including, the U.S., Singapore, Japan, Thailand, India, the Netherlands, South Korea, Malaysia, Brazil, Denmark and Germany. The outage, lasting 10 minutes, was first observed around 8:00 AM EDT and appeared to initially center on NTT nodes located in Los Angeles, CA, Singapore, Tokyo, Japan, and Vienna, Austria. Five minutes into the outage, nodes located in Los Angeles, CA, and Vienna, Austria, appeared to clear, leaving just nodes located in Tokyo, Japan, and Singapore in exhibiting outage conditions. The outage was cleared around 8:15 AM EDT. Click here for an interactive view.
On August 9, Google experienced an outage on its network that impacted access to services running on Google environments in multiple regions including the U.S. and Canada. The outage, lasting a total of 13 minutes, was divided into two occurrences spread over a thirty-minute period. The first occurrence, lasting 9 minutes, was observed around 6:15 AM EDT and appeared to be centered on Google nodes located in Los Angeles, CA, San Jose, CA and San Francisco, CA. Fifteen minutes after the first occurrence cleared, the second occurrence was observed, this time appearing to center on nodes located in Dallas, TX. The outage was cleared around 6:45 AM EDT. Click here for an interactive view.
Internet report for July 31- August 6
ThousandEyes reported 171 global network outage events across ISPs, cloud service provider networks, collaboration app networks and edge networks (including DNS, content delivery networks, and security as a service) during the week of July 31- August 6. That’s up from 156 the week prior, a increase of 10%. Specific to the U.S., outages increased from 60 to 63, up 5%. Here’s a breakdown by category:
ISP outages: Globally, the number of ISP outages decreased from 108 to 94, down 13%, and in the U.S. they decreased from 39 to 31, down 21%.
Public cloud network outages: Globally, cloud-provider network outages jumped from nine to 16, and in the U.S. decreased from eight to seven.
Collaboration app network outages: Globally, collaboration-app network outages dropped from three to two, and in the U.S. they dropped from one to zero.
Two notable network outages
On August 1, Zayo Group, a U.S. based Tier 1 carrier headquartered in Boulder, Colorado, experienced an outage that impacted some of its partners and customers across the U.S. and Canada. The outage lasted around 14 minutes, was first observed around 1:40 AM EDT, and appeared to initially center on Zayo Group nodes located in Paris, France. Five minutes into the outage, the nodes located in Paris, France, appeared to clear, replaced by nodes located in Minneapolis, MN, in exhibiting outage conditions. Around 1:50 AM EDT, the nodes located in Minneapolis, MN, appeared to clear, replaced by nodes located in Seattle, WA, exhibiting outage conditions for the remainder of the outage. The outage was cleared around 1:55 AM EDT. Click here for an interactive view.
On August 3, NTT America, a global Tier 1 ISP and subsidiary of NTT Global, experienced an outage that impacted some of its customers and downstream partners across multiple regions including, the U.S., South Korea, Japan, and the U.K. The outage, lasting 19 minutes, was first observed around 7:45 PM EDT, and appeared to center on NTT nodes located in Los Angeles, CA. The outage was cleared around 8:05 PM EDT. Click here for an interactive view.
Internet report for July 24-30
ThousandEyes reported 156 global outage events across ISPs, cloud service provider networks, collaboration app networks and edge networks (including DNS, content delivery networks, and security as a service) during the week of July 24-30. That’s down from 186 the week prior, a decline of 16%. Specific to the U.S., outages decreased from 74 to 60, down 19%. Here’s a breakdown by category:
ISP outages: Globally, the number of ISP outages decreased from 118 to 108, down 8%, and in the U.S. they increased slightly from 37 to 39, up 5%.
Public cloud network outages: Globally, cloud provider-network outages decreased from 10 to nine, and in the U.S. they decreased from nine to eight.
Collaboration app network outages: Globally, collaboration-app network outages dropped from four to three, and in the U.S. they decreased from three to one.
Two notable network outages
On July 24, Microsoft experienced an issue that impacted connectivity to SharePoint Online and OneDrive for Business services. First observed around 3:05 PM EDT, it appeared to impact connectivity for users globally. Users encountered a certificate error when attempting to access SharePoint Online and OneDrive due to an erroneous change in the SSL certificate that prevented the establishment of a secure connection to the services. Approximately ten minutes later, at around 3:15 PM EDT, it appeared to be replaced with a valid certificate, and SharePoint and OneDrive service reachability was restored for most users by around 3:20 PM EDT. Around 5:34 PM EDT, Microsoft announced that the outage was the result of a configuration issue and had been resolved. Click here for an interactive view.
On July 26, GTT Communications, a Tier 1 ISP headquartered in Tysons, VA, experienced an outage that impacted some of its partners and customers across multiple regions, including the U.S., Canada, the U.K. and the Republic of Korea. The outage lasted 8 minutes in total and was divided into two episodes over a 15-minute period, first observed around 8:30 AM EDT. The first period of the outage, lasting around 4 minutes, appeared to be centered on GTT nodes located in Seattle, WA, and New York, NY. Five minutes after appearing to clear, the second occurrence was observed, this time with just nodes located in New York, NY, exhibiting outage conditions. This reduction in nodes exhibiting outage conditions appeared to coincide with a decrease in the number of regions and partners impacted. The outage was cleared around 8:45 AM EDT. Click here for an interactive view.
Internet report for July 17-23
ThousandEyes reported 186 global outage events across ISPs, cloud service provider networks, collaboration app networks and edge networks (including DNS, content delivery networks, and security as a service) during the week of July 17-23. That’s down from 192 the week prior, a decrease of 3%. Specific to the U.S., outages decreased from 96 to 74, a decline of 23%. Here’s a breakdown by category:
ISP outages: Globally, the number of ISP outages increased from 115 to 118, up 3%, and in the U.S. they decreased from 60 to 37, a decline of 38%.
Public cloud network outages: Globally, cloud provider-network outages decreased from 17 to ten. In the U.S., however, cloud provider network outages increased from four to nine.
Collaboration app network outages: Globally, collaboration-app network outages increased from two to four, and in the U.S. they increased from one to three.
Two notable network outages
On July 21, Cogent Communications, a U.S. based multinational transit provider, experienced an outage that impacted multiple downstream providers as well as Cogent customers across multiple regions including the U.S., Brazil, Singapore, India, Spain, the U.K., Hong Kong, Portugal, Germany, Switzerland, France, Mexico, Australia, Canada, China, Japan, Columbia, Costa Rica, Greece, Malaysia, and Luxembourg. The outage, lasting a total of 33 minutes, was divided into two occurrences over a fifty-minute period. The first occurrence was observed around 6:55 AM EDT and appeared to initially be centered on Cogent nodes located in El Paso, TX, Atlanta, GA, and Denver, CO. Five minutes after first being observed, nodes exhibiting outage conditions expanded to include nodes located in Cleveland, OH, Baltimore, MD, Miami, FL, and Phoenix, AZ. This rise of the number of nodes exhibiting outage conditions appeared to coincide with an increase in the number of impacted regions, downstream partners, and customers. Around fifteen minutes after appearing to clear, the second occurrence was observed. In the second occurrence, the nodes located in Atlanta, GA, El Paso, TX, Denver, CO, Miami, FL, and Phoenix, AZ, were joined by Cogent nodes located in Houston, TX, and Mexico City, Mexico, in exhibiting outage conditions. Fifteen minutes into the second occurrence, all the nodes, with the exception of the nodes located in El Paso, TX, appeared to recover. The outage was cleared around 7:45 AM EDT. Click here for an interactive view.
On July 21, Zayo Group, a U.S. based Tier 1 carrier headquartered in Boulder, Colorado, experienced an outage that impacted some of its partners and customers in multiple countries including the U.S., Germany, and Malaysia. The outage lasted around 8 minutes in total and was divided into two occurrences over a fifteen-minute period. First observed around 7:20 AM EDT, the outage appeared to center on Zayo Group nodes located in Phoenix, AZ. The outage was cleared around 7:35 AM EDT. Click here for an interactive view.
Internet report for July 10-16
ThousandEyes reported 192 global outage events across ISPs, cloud service provider networks, collaboration app networks and edge networks (including DNS, content delivery networks, and security as a service) during the week of July 10-16. That’s a jump from 117 the week prior, an increase of 64%. Specific to the U.S., outages spiked from 61 to 96, an increase of 57%. Here’s a breakdown by category:
ISP outages: Globally, the number of ISP outages increased from 68 to 115, up 69%, and in the U.S. they spiked from 32 to 60, up 88%.
Public cloud network outages: Globally, cloud-provider network outages increased from three to 17, and in the U.S. they increased from three to four.
Collaboration app network outages: Globally, collaboration-app network outages dropped from five to two, and in the U.S. there was one collaboration app network outage, same as the week prior.
Two notable network outages
On July 12, Hurricane Electric, a network transit provider headquartered in Fremont, CA, experienced an outage that impacted customers and downstream partners across the U.S. The outage, first observed at around 1:20 PM EDT, lasted a total of 32 minutes and was divided into three occurrences over a one-hour and 5-minute period. The first occurrence, lasting 24 minutes, appeared to center on Hurricane Electric nodes located in Dallas, TX, and Kansas City, MO. Around 1:25 PM EDT, the nodes located in Kansas City, MO, appeared to temporarily clear before once again exhibiting outage conditions again around 1:30 PM EDT. Around 1:40 PM EDT, the nodes located in Dallas, TX appeared to clear, leaving the nodes located in Kansas City, MO, exhibiting outages for the remainder of the first occurrence. Around 2:00 PM EDT, fifteen minutes after the nodes located in Kansas City, MO appeared to clear, the second occurrence was observed and appeared to center on nodes located in Dallas, TX. Five minutes after the Dallas, TX nodes cleared, the third occurrence was observed and appeared to center on nodes located in Kansas City, MO. The outage was cleared at around 2:15 PM EDT. Click here for an interactive view.
On July 14, Zayo Group, a U.S. based Tier 1 carrier headquartered in Boulder, Colorado, experienced an outage that impacted some of its partners and customers across the U.S. The outage lasted around 14 minutes, was first observed around 3:50 AM EDT, and appeared to center on Zayo Group nodes located in Seattle, WA. Five minutes after first being observed, the number of nodes located in Seattle, WA, exhibiting outage conditions appeared to increase. This rise of the number of nodes exhibiting outage conditions appeared to coincide with an increase in the number of impacted downstream partners and customers. The outage was cleared around 4:05 AM EDT. Click here for an interactive view.
Updated June 27