LogDevice At Scale | Miroslav Crnic & Nick Sukhanov

Meta uses a strongly consistent distributed log storage system to broadcast updates in graphs, deliver signals to ML training pipelines, and collect data for analytics. All of these cases require the underlying log system to be highly available, especially on the write side since we don’t have any other place to store generated data. This talk will cover some optimizations in the consensus algorithm we are using that are required at Meta’s scale to make its systems even more reliable in the presence of hardware maintenance and organic failures.