Mathematical Analysis of Machine Learning Algorithms

Introduction to the mathematical foundation for understanding and analyzing machine learning algorithms for AI students and researchers.

Tong Zhang (Author)

9781009098380, Cambridge University Press

Hardback, published 10 August 2023

479 pages
25.4 x 17.8 x 2.5 cm, 1.113 kg

'This book gives a systematic treatment of the modern mathematical techniques that are commonly used in the design and analysis of machine learning algorithms. Written by a key contributor to the field, it is a unique resource for graduate students and researchers seeking to gain a deep understanding of the theory of machine learning.' Shai Shalev-Shwartz, Hebrew University of Jerusalem

The mathematical theory of machine learning not only explains the current algorithms but can also motivate principled approaches for the future. This self-contained textbook introduces students and researchers of AI to the main mathematical techniques used to analyze machine learning algorithms, with motivations and applications. Topics covered include the analysis of supervised learning algorithms in the iid setting, the analysis of neural networks (e.g. neural tangent kernel and mean-field analysis), and the analysis of machine learning algorithms in the sequential decision setting (e.g. online learning, bandit problems, and reinforcement learning). Students will learn the basic mathematical tools used in the theoretical analysis of these machine learning problems and how to apply them to the analysis of various concrete algorithms. This textbook is perfect for readers who have some background knowledge of basic machine learning methods, but want to gain sufficient technical knowledge to understand research papers in theoretical machine learning.

1. Introduction
2. Basic probability inequalities for sums of independent random variables
3. Uniform convergence and generalization analysis
4. Empirical covering number analysis and symmetrization
5. Covering number estimates
6. Rademacher complexity and concentration inequalities
7. Algorithmic stability analysis
8. Model selection
9. Analysis of kernel methods
10. Additive and sparse models
11. Analysis of neural networks
12. Lower bounds and minimax analysis
13. Probability inequalities for sequential random variables
14. Basic concepts of online learning
15. Online aggregation and second order algorithms
16. Multi-armed bandits
17. Contextual bandits
18. Reinforcement learning
A. Basics of convex analysis
B. f-Divergence of probability measures
References
Author index
Subject index.

Subject Areas: Pattern recognition [UYQP]