The existing congestion control/bandwidth estimation algorithms have several hand-tuned parameters and actions depending on network conditions, leading to challenges in engineering, experimentation, maintenance, and operational monitoring. A machine learning based approach can offer a cleaner alternative with improved performance by addressing the problem holistically across layers (BWE, Network Resiliency, and Transport). This talk outlines our method for addressing BWE and CC issues in RTC using machine learning, discussing obstacles encountered, recent findings, and upcoming plans. Specifically, we explore network characterization and prediction problems, showcasing a few representative examples.