AI @Scale 2020: Azure Cognitive Services @Scale

Azure Cognitive Services sits at the core of many essential products and services at Microsoft for internal and external workloads. Anand’s talk describes the hardware and software infrastructure that supports Ai services at global scale. Azure Cognitive Services workloads are extremely diverse: services require many different types of models in practice. This diversity has implications at all layers in the system stack. In addition, the computational requirements are also intense, leveraging both GPUs and CPUs for real-time inference. Addressing these and other emerging challenges continues to require diverse efforts that span algorithms, software, and hardware design. In this talk, Anand also walks through some of the challenges, including data privacy, deep customization, and bias correction, and discusses solutions they have built to tackle these challenges.