Statistical Mechanics of Learning

Artificial neural networks, learning, statistical mechanics; background material in mathematics and physics; examples and exercises; textbook/reference.

A. Engel (Author), C. Van den Broeck (Author)

9780521774796, Cambridge University Press

Paperback, published 29 March 2001

344 pages, 1 table 136 exercises
24.4 x 17 x 1.8 cm, 0.55 kg

'… recommended both to students of the subjects artificial intelligence, statistics, of interdisciplinary subjects in psychology and philosophy, and to scientists and applied researchers interested in concepts of intelligent learning processes.' Zentralblatt für Mathematik und ihre Grenzgebiete Mathematics Abstracts

Learning is one of the things that humans do naturally, and it has always been a challenge for us to understand the process. Nowadays this challenge has another dimension as we try to build machines that are able to learn and to undertake tasks such as datamining, image processing and pattern recognition. We can formulate a simple framework, artificial neural networks, in which learning from examples may be described and understood. The contribution to this subject made over the last decade by researchers applying the techniques of statistical mechanics is the subject of this book. The authors provide a coherent account of various important concepts and techniques that are currently only found scattered in papers, supplement this with background material in mathematics and physics and include many examples and exercises to make a book that can be used with courses, or for self-teaching, or as a handy reference.

1. Getting started
2. Perceptron learning - basics
3. A choice of learning rules
4. Augmented statistical mechanics formulation
5. Noisy teachers
6. The storage problem
7. Discontinuous learning
8. Unsupervised learning
9. On-line learning
10. Making contact with statistics
11. A bird's eye view: multifractals
12. Multilayer networks
13. On-line learning in multilayer networks
14. What else?
Appendix A. Basic mathematics
Appendix B. The Gardner analysis
Appendix C. Convergence of the perceptron rule
Appendix D. Stability of the replica symmetric saddle point
Appendix E. 1-step replica symmetry breaking
Appendix F. The cavity approach
Appendix G. The VC-theorem.

Subject Areas: Machine learning [UYQM], Mathematical theory of computation [UYA]