This talk will be walking through the development life-cycle of OPT-175B, the first 175B-parameter language model that has been made publicly available for research-use in May 2022. Topics covered will include limitations of transferring results from small-scale experiments, working around hardware failures compounded at scale, numerical instabilities that may occur mid-way through training, amongst others.