Course details

8.3122

Reading Club: The Principles of Deep Learning Theory

 Independent Study Course
WS 2023 Ulf Krumnack, Nohayr Muhammad Abdelmoneim OFFLINE
2h/wk
4 ECTS 		B.Sc modules:
CS-BWP-AI - Artificial Intelligence
CS-BWP-NI - Neuroinformatics
KOGW-WPM-KI - Artificial Intelligence
KOGW-WPM-NI - Neuroinformatics
M.Sc modules:
CC-MWP-AI - Artificial Intelligence
CC-MWP-NI - Neuroinformatics
CS-MWP-AI - Artificial Intelligence
CS-MWP-NI - Neuroinformatics

CS-BW - Bachelor elective course
CS-MW - Master elective course 		Fri: 10-12

Deep learning is a field that recently emerged at the intersection of neuroinformatics, machine learning, big data, and black arts. Trial and error has been the predominant method for setting up highly successful systems outperforming many classical approaches. However, lacking a solid theoretical foundation, basic questions like the relation of width and depth of networks, the role of the initialization scheme, the influence of different activation functions, or the dynamics of the learning process, can still be considered open problems. Daniel A. Roberts, Sho Yaida, and Boris Hanin address such questions in a principled way. Borrowing ideas from statistical physics, they derive rules governing the behavior of deep networks from the properties of their constituents, summarized in their recent book (https://deeplearningtheory.com/). The goal of this reading club is to gather curious and brave people to jointly approach this inaccessible piece of text, discuss central concepts and experimentally verify claims. Our club is open to everyone interested in deep learning. Some affinity to math, specifically calculus and probability theory, will probably increase the joy of participating in this endeavor. People with a background in theoretical physics will gain premium membership.