In March 2025, Meta avoided an major outage that would have resulted in prolonged sitewide unavailability. Instead, we were able to quickly contain the issue in a matter of minutes and avoid the worst case scenario. This is a story of how we re-bootstrapped the infrastructure control plane in an entire region, on the fly, using emergency tooling. It’s a showcase of how to put DR-preparedness and calm incident response into practice. And it’s a pretty cool story.
