The Principles of Deep Learning Theory
An Effective Theory Approach to Understanding Neural Networks

This volume develops an effective theory approach to understanding deep neural networks of practical relevance.

Daniel A. Roberts (Author), Sho Yaida (Author), Boris Hanin (Contributions by)

9781316519332, Cambridge University Press

Hardback, published 26 May 2022

472 pages
26.1 x 18.4 x 2.6 cm, 1.06 kg

'An excellent resource for graduate students focusing on neural networks and machine learning … Highly recommended.' J. Brzezinski, Choice

This textbook establishes a theoretical framework for understanding deep learning models of practical relevance. With an approach that borrows from theoretical physics, Roberts and Yaida provide clear and pedagogical explanations of how realistic deep neural networks actually work. To make results from the theoretical forefront accessible, the authors eschew the subject's traditional emphasis on intimidating formality without sacrificing accuracy. Straightforward and approachable, this volume balances detailed first-principle derivations of novel results with insight and intuition for theorists and practitioners alike. This self-contained textbook is ideal for students and researchers interested in artificial intelligence with minimal prerequisites of linear algebra, calculus, and informal probability theory, and it can easily fill a semester-long course on deep learning theory. For the first time, the exciting practical advances in modern artificial intelligence capabilities can be matched with a set of effective principles, providing a timeless blueprint for theoretical research in deep learning.

Preface
0. Initialization
1. Pretraining
2. Neural networks
3. Effective theory of deep linear networks at initialization
4. RG flow of preactivations
5. Effective theory of preactivations at initializations
6. Bayesian learning
7. Gradient-based learning
8. RG flow of the neural tangent kernel
9. Effective theory of the NTK at initialization
10. Kernel learning
11. Representation learning
?. The end of training
?. Epilogue
A. Information in deep learning
B. Residual learning
References
Index.

Subject Areas: Machine learning [UYQM], Artificial intelligence [UYQ], Mathematical physics [PHU], Statistical physics [PHS]