News & Ideas | At Scale Conferences

Building a continuous product maintenance loop with AI agents A product ships. The maintenance clock starts. Each product’s life cycle spans design, development, testing, launch, and ongoing operation. Once it is live, teams monitor its health, respond to user feedback, fix failures, and improve performance, reliability, and usability. Each improvement goes through implementation, review, testing, […]

By: Sumeet Arora

Mobile, Video and Web

From On-Call to Always-On

Building a continuous product maintenance loop with AI agents A product ships. The maintenance clock starts. Each product’s life cycle spans design, development, testing, launch, and ongoing operation. Once it is live, teams monitor its health, respond to user feedback, fix failures, and improve performance, reliability, and usability. Each improvement goes through implementation, review, testing, […]

Bringing a new codec, AV1, to RTC at Meta has been a long journey. This talk covers the breakthroughs of the past three years. We’ll share how we expanded AV1 coverage, including how we used large-scale product data to identify the best codec settings. We’ll also explain how we improved AV1 call quality through end-to-end […]

By: Yu-Chen (Eric) Sun

Mobile, Video and Web

Adopting AV1 for Real-Time Communication (RTC) at Scale

Bringing a new codec, AV1, to RTC at Meta has been a long journey. This talk covers the breakthroughs of the past three years. We’ll share how we expanded AV1 coverage, including how we used large-scale product data to identify the best codec settings. We’ll also explain how we improved AV1 call quality through end-to-end […]

In my talk, I shared how AI voice translations make content on Facebook and Instagram accessible across languages, so creators can reach new audiences in their own voice. Over half a billion people on each app now watch AI-translated videos every week. Delivering that depends on high-quality speech translation, and my colleagues on Meta’s speech […]

By: Esra Cansizoglu

Mobile, Video and Web

Learn more: the research behind AI voice translations at Meta

In my talk, I shared how AI voice translations make content on Facebook and Instagram accessible across languages, so creators can reach new audiences in their own voice. Over half a billion people on each app now watch AI-translated videos every week. Delivering that depends on high-quality speech translation, and my colleagues on Meta’s speech […]

A data center network begins life as a logical diagram: this many spine switches, connected to this many fabric switches, in this fabric topology. Building it means answering two much harder questions. Which of the thousands of rack positions should each device occupy, given the power, space, and failure domain of each? And which of […]

By: Richard Cziva ...

Systems and Networking

Physical Network Design at Scale

A data center network begins life as a logical diagram: this many spine switches, connected to this many fabric switches, in this fabric topology. Building it means answering two much harder questions. Which of the thousands of rack positions should each device occupy, given the power, space, and failure domain of each? And which of […]

When we talk about the future of artificial intelligence, the conversation often centers on models, algorithms, and GPUs. But beneath every breakthrough in AI lies something far more tangible—the physical infrastructure that makes it all possible. In New Albany, Ohio, a first-of-its-kind engineering project is quietly redefining what’s possible in data center connectivity: the NAO […]

By: Fabrice Ouandji ...

Systems and Networking

Building the Backbone of AI: Inside the NAO Mega Cluster’s Fiber Revolution

When we talk about the future of artificial intelligence, the conversation often centers on models, algorithms, and GPUs. But beneath every breakthrough in AI lies something far more tangible—the physical infrastructure that makes it all possible. In New Albany, Ohio, a first-of-its-kind engineering project is quietly redefining what’s possible in data center connectivity: the NAO […]

Systems and Networking

Keynote from Meta

Peter will share the latest progress from Meta’s mission to build the future of human connection and the technology that makes it possible, including infrastructure for AI by leveraging agentic innovation across the stack.

WATCH VIDEO

Systems and Networking

LIVE PANEL WITH AURORA, GOOGLE, META AND MICROSOFT

Saad Syed (Google) Mark Jen (Meta) Viral Mehta (Aurora) Jenny Lawrance (Microsoft)

WATCH VIDEO

Systems and Networking

AI Storage Blueprint

The rapid, exponential growth in model capabilities and training dataset sizes over the last few years has accelerated AI innovation. New frontier models are now being released in a matter of weeks, down from months just a year ago. This pace makes reliable, consistent fast access to storage essential for managing both the speed and […]

WATCH VIDEO

Systems and Networking

Taming AI Infrastructure Failures with Agentic Debugging

NCCL watchdog timeouts are a common failure mode in distributed AI model training. They impact not only Meta, but broadly affect anyone running PyTorch distributed training—and they’re notoriously hard to debug: even experts can spend hours triaging a single incident, and non-experts may be unable to root-cause them at all. Over the past year, we […]

WATCH VIDEO

Systems and Networking

Securing Production Debugging at Hyperscale

This presentation will go over how Microsoft uses SSH to debug across our global fleet of K8s clusters at hyperscale. We’ll cover how we integrate that with JIT elevations and Task-Based Execution in a secure and auditable way. We will alsogo over how we use these building blocks to engage AI and the Semantic Kernel […]

WATCH VIDEO

Systems and Networking

[Almost] Fail & Tell: Stop The World

In March 2025, Meta avoided an major outage that would have resulted in prolonged sitewide unavailability. Instead, we were able to quickly contain the issue in a matter of minutes and avoid the worst case scenario. This is a story of how we re-bootstrapped the infrastructure control plane in an entire region, on the fly, […]

WATCH VIDEO

Systems and Networking

Building Privacy Aware Infrastructure in the AI-Native Era

Building privacy-aware infrastructure in the AI-native era requires systems that can learn, evaluate, and operationalize improvements continuously—at the scale of millions of data assets. This talk presents an AI-native approach through a case study of AI-native asset classification: a hybrid deterministic rules engine with LLM fallback that transforms messy context (metadata, lineage, code references, scan […]

WATCH VIDEO

LATEST ON @SCALE

From On-Call to Always-On

Adopting AV1 for Real-Time Communication (RTC) at Scale

Learn more: the research behind AI voice translations at Meta

Physical Network Design at Scale

Building the Backbone of AI: Inside the NAO Mega Cluster’s Fiber Revolution

Keynote from Meta

LIVE PANEL WITH AURORA, GOOGLE, META AND MICROSOFT

AI Storage Blueprint

Taming AI Infrastructure Failures with Agentic Debugging

Securing Production Debugging at Hyperscale

[Almost] Fail & Tell: Stop The World

Building Privacy Aware Infrastructure in the AI-Native Era