Dev Tools @Scale 2017

As people across the world connect on Facebook, we want to make sure our apps and services work well in a myriad of scenarios. At Facebook's scale, this means testing important interactions across numerous different types of devices and operating systems for both correctness and speed. In order to improve reliability on services like end to end testing, performance testing and others, we've built a brand new resource management system. Learn the challenges we encountered and how other services are using this new system to be faster and more reliable.

Software release cycles are now measured in days instead of months. Cutting edge companies are continuously delivering high-quality software at a fast pace. In this session, we will cover how you can begin your DevOps journey by sharing best practices and tools used by the "two pizza" engineering teams at Amazon. We will showcase how you can accelerate developer productivity by implementing continuous Integration and delivery workflows. We will also cover an introduction to AWS CodeStar, AWS CodeCommit, AWS CodeBuild, AWS CodePipeline and AWS CodeDeploy, the services inspired by Amazon's internal developer tools and DevOps practice.

At Facebook, new code is being produced and shipped fast in a big code base shared by hundreds of engineers. Thus, we use a number of techniques to ensure good quality of code in this environment, such as unit and end-to-end tests and static analysis. Static analysis is a technique for detecting software errors statically, before a product is actually shipped, and without running the code. In this talk we describe the Infer Static Analyzer, an open source tool developed at Facebook that can find complex bugs such as Null Dereferences and Memory Leaks. It can also run fast in the CI, so that we can catch and fix bugs before code gets committed. We will also describe a domain specific language (DSL) that we are building for writing simple static analysis checks, such that developers can write their own checks easily.

At GitHub we host a large number of wildly different projects and are the de-facto destination for many people. This means we see a lot of different use cases come through our services. Sometimes it's the shape of the data, sometimes its size, sometimes it's an unexpected combination of both for a novel use case that makes it harder to host. In this talk we'll see how GitHub has developed projects like Spokes and Lariat to make sure we can host these Git repositories reliably. From replication and heuristics to caching and application of quotas, they help make sure a project with over-eager CI systems or expensive Git processing requirements does not endanger the overall reliability of the system.

Facebook's continuous integration infrastructure is powered by thousands of servers that have different configurations and installed dependencies; on the other side, jobs have different requirements. How do we schedule jobs to servers or vice-versa? In this talk, we will learn about Jupiter, our matching service that helped this infrastructure scale.

Earlier this year we announced that we are migrating the Windows team to Git, enabled in large part by the Git Virtual File System (GVFS). Since then, we have successfully migrated most of the Windows team, and we now have over 3000 engineers working in a single Git repo that is over 300GB in size and has more than 3 million files in it. In this talk we’ll discuss the challenges of working in a repo of that size, and the changes we’ve made in Visual Studio Team Services and in GVFS to make this possible.

As the Spotify Android codebase and number of engineers grows, our existing testing tools and infrastructure have struggled to keep up with the rapid pace of development. In this talk, learn about how we are keeping up the pace by creating new testing tools that allow teams to easily create fast and stable tests for the features they create and maintain.

Supercomputers for scientific applications are rapidly approaching one million cores, designed for use by tightly-coupled latency-sensitive applications. We talk about the architectural choices made to allow interactive debugging and lightweight profiling of such applications and the challenges in presenting information to users who are quite firmly domain experts first and programmers second. We show two practical use cases – one enabling research for predicting strokes in individuals and another deploying deep learning in the cloud.

Code search tools are an important part of an efficient developer workflow. At Facebook, we’ve built custom tooling to keep code search efficient for large repositories. In this talk, we’ll go over some of the work we’ve done at Facebook to allow our engineers to quickly find files and commits that match certain patterns.

Google's recently open-sourced build system, Bazel, enables reproducible, performant builds at Google's codebase scale. Moreover, it is extensible and allows for seamless multilingual builds. This talk will cover Bazel's architecture and extensibility model. We will talk about how Bazel scales for both large (pure line count) and diverse (multi-lingual) code bases.