On the morning of June 8, some of the biggest websites on the internet simultaneously went offline.
Online shoppers couldn’t access Amazon. Breaking news stories weren’t able to be published on The New York Times, CNN, or The Verge. Reddit went down, causing meme stock investors to congregate in the YouTube comments section of the Gangnam Style music video where they speculated the whole situation was a plot against them by hedge funds.
Well, we now know what caused some of the most popular websites to go down this week. It turns out there wasn’t some big conspiracy. However, the truth is still pretty bizarre.
A single CDN service customer changed their network settings, activating a glitch that took down large swaths of the World Wide Web.
On Tuesday evening, CDN provider Fastly published a post on its website explaining the issue with its service that caused the website outages earlier that day.
According to Fastly’s senior vice president of engineering and infrastructure Nick Rockwell, a software update from last month unknowingly “introduced” a bug to the platform. Fast forward to early Tuesday morning when a single customer – as Gizmodo put it – “reconfigured his internet connection.” That change then set off a domino effect that took down some of the biggest sites on the web.
“A customer pushed a valid configuration change that included the specific circumstances that triggered the bug, which caused 85% of our network to return errors,” Rockwell said.
That’s all it took.
According to Fastly, the company detected the outage within a minute and immediately went to work to solve the problem. After figuring out the issue, 95 percent of the websites affected were back online with 49 minutes of downtime, wrote Rockwell.
Many of your favorite high-trafficked websites use CDN providers like Fastly or Cloudflare. When they work, CDNs, or content delivery networks, help improve website performance and deliver content to users faster and more efficiently.
But, when CDNs don’t work, it’s extremely noticeable because it affects some of the internet’s most popular platforms. Just last year, for example, Cloudflare issues caused services like Discord to temporarily go down. And that’s not the first time it’s happened either.
Fastly says it is deploying a bug fix and investigating further to ensure these issues don’t happen again.
While technical glitches and human error will always occur on any platform or service, it’s certainly not a good sign that said glitches and errors — from a single source — can cause such widespread problems.