Avoiding Overload in Distributed Systems

At AWS, we build systems using a variety of complementary strategies for maintaining predictable, consistent performance in the face of overload. In this talk, we describe techniques such as implementing layers of protection, preventing retry storms, prioritization, and fairness in multi-tenant systems. We also cover architectural approaches that isolate uncorrelated workloads, such as shuffle-sharding, and the per-request isolation model in AWS Lambda.