Managing data center networks with low loss requires understanding traffic patterns, especially burstiness of the traffic, at fine time granularity. Yet, monitoring traffic with millisecond granularity fleet wide is challenging. To gain more visibility into our production network, we design millisampler, a BPF based lightweight traffic measurement tool deployed in every server in the entire fleet for continual monitoring that operates at high granularity timescale. Millisampler data allows us to characterize microbursts at millisecond or even microsecond granularity, and simultaneous data collection enables analysis of how synchronized bursts interact in rack buffers. We’ll discuss the design, implementation, and production experience with millisampler, and share some interesting observations we got with the millisampler data.

At Meta, we need to be able to readily determine if network conditions are responsible for instances of poor quality of experience (QoE) such as images loading slowly or video stalling during playback. In response, we’ve developed Network SLOs, which can be thought of as a product’s "minimum network requirements’ for good QoE. If the network between Meta and a user does not meet the product’s SLO requirements, QoE will be degraded. In this talk, we describe our work over the past three years on deriving and operationalizing Network SLOs for Meta’s user facing products. First, we discuss how we measure quality of experience for a handful of products and our approach to quantifying the relevant network conditions. We then discuss how we use a combination of statistical tools to derive Network SLOs, and how we process trillions of measurements each day to evaluate Network SLO compliance. We present case-studies of how Network SLOs have been used to triage regressions in QoE, identify gaps in Meta’s edge network capacity, and surface inefficiencies in how product utilizes the network.

Since the early days of the Internet, capacity has been the prime metric to quantify the quality of the Internet access. While capacity was the primary challenge back in those days, we have successfully reached a point where the vast majority of users can easily access sufficient capacity for the majority of the use-cases. But still, Internet experience is often lacking the smoothness that we would expect. Video-conferencing still has frequent issues, video gaming is rarely a smooth experience and web-browsing still suffers from bad page-load times. We present a new metric, called ""Responsiveness under working conditions"", which significantly broadens the scope far beyond traditional capacity and latency measurements. This metric aims at quantifying the network's ability to provide low latency while at the same time providing high capacity. The measurement methodology not only measures the network, but also the end-host networking stack. We will describe how measuring responsiveness will allow to detect deep buffers in the server's networking stack and how it affects the end-user experience. Further, we describe steps that can be taken to reduce those buffers. We will conclude this talk by providing resources and open-source tools to allow everyone to reproduce the same measurement on their infrastructure and tune their networking stack for the benefit of their end-users.

Featuring Yimeng Zhao, Brandon Schlinker, Sharad Jaiswal. Moderated by Neil Spring

We work on a layer 4 load balancer called Shiv. Shiv routes packets to backends using a consistent hash of the 5-tuple of the packet (namely, the source ip, destination ip, source port, destination port, and protocol). Shiv’s objective is to route packets for a connection (which all have the same 5-tuple) to the same backend for the duration of the connection. If it is unable to do so, this leads to broken connections and user impact (for example, stalled videos). While consistent hashing is quite resilient to changes, when a large number of backends are added or removed, remappings occur, resulting in broken connections. To protect from such changes, Shiv maintains a cache that contains a mapping from 5-tuple to backend. The logic used by Shiv to route packets can be summarized as follows: If the 5-tuple of the packet is in its cache, route it to the backend indicated by the cache. Otherwise, calculate the hash function on the 5-tuple to obtain the destination backend, route the packet to that backend, and place the (5-tuple, backend) entry in the cache. Shiv works well under the following conditions: - In steady state, when the arrangement of Shivs and backends is the same. - When the arrangement of Shivs changes. In this case, packets for a connection may land on a different Shiv host than earlier packets, but both Shiv hosts use the same consistent hash function, and therefore, pick the same backend. - When the arrangement of backends changes. In this case, packets for a connection continue to land on the same Shiv host, which utilizes its cache to route the packet to the same backend as it used to. However, during changes to the arrangement of both Shivs and backends, a nontrivial number of misroutings occur, because the following sequence of events could happen: - Packets for a connection C arrive at a Shiv host A, which picks a backend X - A large topology change occurs on the Shivs and backends. - Packets for connection C now land at Shiv host B != A, which picks a backend Y != X because the hash ring has changed. We have implemented two solutions to this problem, that we will talk about: - Embedding “Server ID” hints into packets, that enable Shiv to route the packets to a specific server without having to perform a consistent hash. - Sharing the 5-tuple to backend cache among all Shivs in a cluster, thereby facilitating consistent decision making among them in the face of hash ring changes.

AWS Global Accelerator (AGA) leverages Amazon’s global network, anycast routing, and integrated load-balancing techniques to reduce latency and improve availability. In this session, we discuss the architecture, capabilities, and challenges of operating AGA at scale. We will provide insight into how AGA drives path optimization through the integration of customer-defined traffic control and real-time internet performance. We end the session describing future work in this space.

Featuring: Aman Sharma, Andrii Vasylevskyi, Alan Halachmi, Akshat Aranya Moderated by Bharet Parekh