JUNE 02, 2016

Scale out resource management: YARN++

We then changed gears to a topic that is relevant to many: how to manage resources within a large-scale compute cluster in a way that optimizes system resources, all while balancing the competing needs of the various processing jobs. Microsoft’s Big Data team has embraced Apache YARN and is building around YARN as the resource manager in its clusters. In this talk, Sriram Rao described the team’s work in building a YARN-based, scale-out architecture. The system is self-configuring and tolerates failures of subcomponents while providing continued availability. This work has been contributed back to Apache YARN and ships with various Hadoop distributions.

To help personalize content, tailor and measure ads, and provide a safer experience, we use cookies. By clicking or navigating the site, you agree to allow our collection of information on and off Facebook through cookies. Learn more, including about available controls: Cookies Policy