Evaluating Learning Algorithms
A Classification Perspective

Gives a solid basis for conducting performance evaluations of learning algorithms in practical settings with an emphasis on classification algorithms.

Nathalie Japkowicz (Author), Mohak Shah (Author)

9780521196000, Cambridge University Press

Hardback, published 17 January 2011

424 pages, 40 b/w illus. 45 tables
23.4 x 15.6 x 2.4 cm, 0.77 kg

"This book has the merit of organizing most of the material about the evaluation of learning algorithms into a homogeneous description, covering both theoretical aspects and pragmatic issues. It is a useful resource for researchers in machine learning, and provides adequate material for graduate courses in machine learning and related fields."
Corrado Mencar, Computing Reviews

The field of machine learning has matured to the point where many sophisticated learning approaches can be applied to practical applications. Thus it is of critical importance that researchers have the proper tools to evaluate learning approaches and understand the underlying issues. This book examines various aspects of the evaluation process with an emphasis on classification algorithms. The authors describe several techniques for classifier performance assessment, error estimation and resampling, obtaining statistical significance as well as selecting appropriate domains for evaluation. They also present a unified evaluation framework and highlight how different components of evaluation are both significantly interrelated and interdependent. The techniques presented in the book are illustrated using R and WEKA, facilitating better practical insight as well as implementation. Aimed at researchers in the theory and applications of machine learning, this book offers a solid basis for conducting performance evaluations of algorithms in practical settings.

1. Introduction
2. Machine learning and statistics overview
3. Performance measures I
4. Performance measures II
5. Error estimation
6. Statistical significance testing
7. Data sets and experimental framework
8. Recent developments
9. Conclusion
Appendix A: statistical tables
Appendix B: additional information on the data
Appendix C: two case studies.

Subject Areas: Pattern recognition [UYQP], Machine learning [UYQM]