NOVEMBER 14, 2022

Developing, Deploying, Operating In-House Network Switches at a Massive Scale

FBOSS is Meta’s software stack for controlling/managing network switches deployed in Meta’s datacenters. FBOSS is one of the largest services in Meta (in terms of the number of instances deployed) and powers Meta’s Network. The talk presents our experience designing, developing and operating FBOSS: an in-house software built to manage and support a set of features required for data center switches of a large scale Internet content provider. We present the key ideas underpinning the FBOSS model viz.: switch-as-a-server and deploy-early-and-iterate. These ideas have enabled us to build a stable and scalable network. We demonstrate these ideas with concrete examples: Switch Abstraction Interface (SAI) defines a vendor-independent API for programming the forwarding ASIC. We developed a new SAI based implementation for FBOSS. The new FBOSS implementation was deployed at a massive scale to a brownfield deployment, and was also leveraged to onboard a new switch vendor into the Meta infrastructure. We present this effort as a case study to further illustrate the FBOSS design, development and operations model. In particular, the case study shares our experience and lessons learned while: -Developing the new FBOSS implementation to be functionally equivalent to the existing implementation for all production use cases, -Methodology for Third party vendor ASIC/SDK integration and functionality validation, -FBOSS push infrastructure and qualification pipeline, Maintaining and upgrading a mix of old and new implementation during gradual and phased brownfield deployment, -Tooling enhancements to detect/mitigate/triage/repair issues discovered while rolling out the new implementation to the brownfield deployment as well as onboarding new switch vendor to production, -Rolling out patches to mitigate/fix production issues without causing traffic disruption.

To help personalize content, tailor and measure ads, and provide a safer experience, we use cookies. By clicking or navigating the site, you agree to allow our collection of information on and off Facebook through cookies. Learn more, including about available controls: Cookies Policy