Despite the huge empirical success of deep learning, theoretical understanding of neural networks learning process is still lacking. This is the reason, why some of its features seem ‘mysterious’. We emphasize two mysteries of deep learning: generalization mystery, and optimization mystery. In this essay we review and draw connections between several selected works concerning the latter. An Essay on Optimization Mystery of Deep Learning
Document worth reading: “An Essay on Optimization Mystery of Deep Learning”
31 Tuesday Mar 2020
Posted Documents
in