Density Ratio Estimation in Machine Learning

This book introduces theories, methods and applications of density ratio estimation, a newly emerging paradigm in the machine learning community.

Masashi Sugiyama (Author), Taiji Suzuki (Author), Takafumi Kanamori (Author)

9781108461733, Cambridge University Press

Paperback / softback, published 29 March 2018

341 pages, 79 b/w illus. 18 tables
23 x 15.3 x 1.8 cm, 0.55 kg

'There is no doubt that this book will change the way people think about machine learning and stimulate many new directions for research.' Thomas G. Dietterich, from the Foreword

Machine learning is an interdisciplinary field of science and engineering that studies mathematical theories and practical applications of systems that learn. This book introduces theories, methods and applications of density ratio estimation, which is a newly emerging paradigm in the machine learning community. Various machine learning problems such as non-stationarity adaptation, outlier detection, dimensionality reduction, independent component analysis, clustering, classification and conditional density estimation can be systematically solved via the estimation of probability density ratios. The authors offer a comprehensive introduction of various density ratio estimators including methods via density estimation, moment matching, probabilistic classification, density fitting and density ratio fitting, as well as describing how these can be applied to machine learning. The book provides mathematical theories for density ratio estimation including parametric and non-parametric convergence analysis and numerical stability analysis to complete the first and definitive treatment of the entire framework of density ratio estimation in machine learning.

Part I. Density Ratio Approach to Machine Learning: 1. Introduction
Part II. Methods of Density Ratio Estimation: 2. Density estimation
3. Moment matching
4. Probabilistic classification
5. Density fitting
6. Density-ratio fitting
7. Unified framework
8. Direct density-ratio estimation with dimensionality reduction
Part III. Applications of Density Ratios in Machine Learning: 9. Importance sampling
10. Distribution comparison
11. Mutual information estimation
12. Conditional probability estimation
Part IV. Theoretical Analysis of Density Ratio Estimation: 13. Parametric convergence analysis
14. Non-parametric convergence analysis
15. Parametric two-sample test
16. Non-parametric numerical stability analysis
Part V. Conclusions: 17. Conclusions and future directions.

Subject Areas: Pattern recognition [UYQP], Machine learning [UYQM], Computer science [UY]