SEPTEMBER 28, 2022

OPT-175B: LLM Development Lifecycle & Challenges

This talk will be walking through the development life-cycle of OPT-175B, the first 175B-parameter language model that has been made publicly available for research-use in May 2022. Topics covered will include limitations of transferring results from small-scale experiments, working around hardware failures compounded at scale, numerical instabilities that may occur mid-way through training, amongst others.

To help personalize content, tailor and measure ads, and provide a safer experience, we use cookies. By clicking or navigating the site, you agree to allow our collection of information on and off Facebook through cookies. Learn more, including about available controls: Cookies Policy