Facebook and some of its most widely used apps, including Instagram, WhatsApp, and Messenger, experienced a major outage affecting users globally on Monday.
Services began coming back online about 6 p.m. ET.
Some of Facebook’s internal-communications tools were also knocked offline, adding another obstacle to the company’s efforts to get its services working again.
The outage even hit Facebook employees not directly involved in troubleshooting, which led Instagram’s chief, Adam Mosseri, to tweet that “it does feel like a snow day” for workers.
Here’s what we know about the outage so far.
About 11:45 a.m. ET on Monday, users around the world reported issues accessing Facebook apps like Instagram, WhatsApp, Messenger, and Facebook itself on the outage-tracking website Downdetector.
Facebook employees also had issues accessing internal tools and even physical areas within the company’s offices.
“Devs can’t access their apps online to push new builds, no documentation, nothing. All Oculus platform services are down. No avatars as well,” a developer whose work includes Facebook platforms told Insider.
Facebook security employees also were slowed while assessing the incident because their digital badges stopped working, which kept them from accessing Facebook’s servers, The New York Times reported.
Other Facebook employees were locked out of Internet of Things-connected conference rooms, the NBC News reporter Kevin Collier tweeted.
What might have caused the outage?
Facebook published a brief blog post Monday evening attributing the massive global outage that took its services and internal communications tools offline for several hours to a “faulty configuration change” to its routers.
“Configuration changes on the backbone routers that coordinate network traffic between our data centers caused issues that interrupted this communication. This disruption to network traffic had a cascading effect on the way our data centers communicate, bringing our services to a halt,” Facebook vice president of infrastructure Santosh Janardhan wrote in the post.
“The underlying cause of this outage also impacted many of the internal tools and systems we use in our day-to-day operations, complicating our attempts to quickly diagnose and resolve the problem,” Janardhan added.
Facebook said it had “no evidence that user data was compromised as a result of this downtime.”
Facebook employees and internet experts outside the company had hinted at a similar cause early during the outage.
A cyberattack likely wasn’t to blame, two Facebook security-team members told The New York Times, because it would have been difficult for a single attack to target the wide variety of underlying technologies affected during the outage.
One possibility is an issue with the Facebook-run servers responsible for telling internet users where to find content.
CloudFlare Chief Technology Officer John Graham-Cumming tweeted: “Facebook and related properties disappeared from the Internet in a flurry of BGP updates.”
CloudFlare describes BGP, or border gateway protocol, as the “postal service of the Internet … responsible for looking at all of the available paths that data could travel and picking the best route.”
Just before Facebook’s outage, Graham-Cunning said CloudFlare detected “a large number of BGP changes (mostly route withdrawals)” to Facebook’s autonomous systems – basically, the local post-office branches.
CloudFlare explained its take in further detail in a blog post.
Brian Krebs, a security researcher, also said the likely culprit was a misstep on Facebook’s end that caused its services to drop off the internet’s map.
“The DNS records that tell systems how to find http://facebook.com or http://Instagram.com got withdrawn this morning from the global routing table,” Krebs tweeted.
“We don’t know why this change was made. It could well have been the result of an internal, system wide change or update that went awry,” he added. “It’s all speculation at this point why. FB alone is in control over its DNS records.”
What are the effects of the outage?
For one, the chain reaction the outage created was a major headache for Facebook’s employees.
Alex Hern, an editor at The Guardian, said in a series of tweets that much of Facebook’s IT system depends on Facebook.
So, Hern said, if the company accidentally cut its servers off from the internet, that would have also impaired its ability to “update” the internet with correct information, log in to Facebook’s systems to send that update, use a digital badge to access the server room where Facebook’s systems are housed, and send messages to the security team asking them to unlock the doors with a physical key.
Facebook’s services also power a substantial portion of the digital economy, meaning businesses that run Facebook and Instagram ads and communicate with customers on the platforms were also caught up in the outage.
In many parts of the world, especially Latin America and India, WhatsApp is the most popular messaging app and used widely for personal, professional, and political communication, which means Facebook’s outage disrupted the ability of users in those regions to communicate with one another, José Caparroso, a Forbes editor, said.
Responding to Caparroso, Rep. Alexandria Ocasio-Cortez said Facebook’s acquisition and subsequent integration of WhatsApp into its infrastructure meant that vastly more of the global digital-communications system depended on Facebook – and was affected when it failed.
Facebook CEO Mark Zuckerberg and other shareholders in the company also had a rough day: Zuckerberg’s net worth dropped by $7 billion on Monday as Facebook’s stock slid by about 5%.
The internet had a field day, flocking to Twitter and other social-media platforms to post memes making fun of the situation.
What is the company saying?
Read Facebook’s blog post addressing the outage.
Facebook: “To the huge community of people and businesses around the world who depend on us: we’re sorry. We’ve been working hard to restore access to our apps and services and are happy to report they are coming back online now. Thank you for bearing with us.”
Facebook Chief Technology Officer Mike Schroepfer also chimed in, tweeting: “*Sincere* apologies to everyone impacted by outages of Facebook powered services right now. We are experiencing networking issues and teams are working as fast as possible to debug and restore as fast as possible.”
Instagram: “Instagram and friends are having a little bit of a hard time right now, and you may be having issues using them. Bear with us, we’re on it! #instagramdown.”
WhatsApp: “We’re aware that some people are experiencing issues with WhatsApp at the moment. We’re working to get things back to normal and will send an update here as soon as possible. Thanks for your patience!”
The article originally published on Business Insider.