Loss landscape of deep neural networks

Speaker(s)

Bartłomiej Polaczyk

Date

March 25, 2021, 12:15 p.m.

Information about the event

meet.google.com/yew-oubf-ngi

Seminar

Seminarium "Machine Learning"

Finding the global minimum of a general non-convex function is in general an NP-complete problem. The optimization objective of deep neural networks (DNN) is not only non-convex but even non-smooth in case of ReLU activation function. Yet, practice suggests that even simple first-order methods such as gradient descent are able (in many cases) to find the global optimum of DNN at random initialization. Understanding this phenomenon is one of the core problems in theoretical machine learning.