Addressing continuous deployment’s resource creep at scale
Resource creep is one disadvantage of continuous deployment. Small changes, on average, have smaller incremental resource demands (CPU, memory, network, disk IOPS, etc.). The additional resource demands may seem reasonable or inconsequential when viewed in isolation. However, over time they add up. This causes increased demands on computing infrastructure that may not be dealt with until it becomes a problem, which typically means significantly after software has been released. Some of the additional resources needed are legitimate — in support of the product enhancements — but others are inefficiencies that need be optimized. This talk will describe Facebook’s approach to dealing with resource creep at scale. Facebook employs two philosophies: 1) in order to keep product developers as productive as possible, resource creep is handled by a separate team, entirely focused on the problem; and 2) Facebook scales its engineering through tools and automation rather than by adding people. Facebook will describe the tools and approaches that are used to identify and correct resource creep for the company’s PHP developers with regressions on the online data storage and access infrastructure.