{"product_id":"partially-observed-markov-decision-processes-filtering-learning-and-controlled-sensing-hardback-9781009449434","title":"Partially Observed Markov Decision Processes; Filtering, Learning and Controlled Sensing (Hardback) 9781009449434","description":"\u003cfont face=\"Georgia\"\u003e\r\n\u003cp\u003e\u003cfont size=\"6\"\u003ePartially Observed Markov Decision Processes\u003c\/font\u003e\u003cbr\u003e\r\n\u003cfont size=\"5\"\u003eFiltering, Learning and Controlled Sensing\u003c\/font\u003e\u003c\/p\u003e\r\n\r\n\u003cp\u003e\u003cem\u003eThis new edition includes inverse reinforcement learning, non-parametric Bayesian inference, variational Bayes and conformal prediction.\u003c\/em\u003e\u003c\/p\u003e\r\n\r\n\r\n\u003cp\u003e\u003cfont size=\"4\"\u003eVikram Krishnamurthy (Author)\u003c\/font\u003e\u003c\/p\u003e\r\n\r\n\u003cp\u003e\u003cfont size=\"3\"\u003e9781009449434, Cambridge University Press\u003c\/font\u003e\u003c\/p\u003e\r\n\r\n\u003cp\u003e\u003cfont size=\"3\"\u003eHardback, published 5 June 2025\u003c\/font\u003e\u003c\/p\u003e\r\n\r\n\u003cp\u003e\u003cfont size=\"3\"\u003e652 pages\u003cbr\u003e25.9 x 18.5 x 4 cm, 1.4 kg\u003c\/font\u003e\u003c\/p\u003e\r\n\r\n\r\n\r\n\u003cp align=\"justify\"\u003e\u003cem\u003e\u003cfont size=\"3\"\u003e'An outstanding advanced graduate-level introduction to the increasingly important topic of partially observed Markov decision processes. The book is a delight to read - comprehensive, clear, up-to-date and insightful while preserving rigor. An essential resource for both researchers seeking to further advance the field and practitioners wishing to implement stochastic control in real engineering systems.' Rob Evans, University of Melbourne\u003c\/font\u003e\u003c\/em\u003e\u003c\/p\u003e\r\n\r\n\u003cp align=\"justify\"\u003e\u003cstrong\u003e\u003cfont size=\"3\"\u003eCovering formulation, algorithms and structural results and linking theory to real-world applications in controlled sensing (including social learning, adaptive radars and sequential detection), this book focuses on the conceptual foundations of partially observed Markov decision processes (POMDPs). It emphasizes structural results in stochastic dynamic programming, enabling graduate students and researchers in engineering, operations research, and economics to understand the underlying unifying themes without getting weighed down by mathematical technicalities. In light of major advances in machine learning over the past decade, this edition includes a new Part V on inverse reinforcement learning as well as a new chapter on non-parametric Bayesian inference (for Dirichlet processes and Gaussian processes), variational Bayes and conformal prediction.\u003c\/font\u003e\u003c\/strong\u003e\u003c\/p\u003e\r\n\r\n\u003cp\u003e\u003cfont size=\"3\"\u003ePreface to revised edition\u003cbr\u003e Notation\u003cbr\u003e 1. Introduction\u003cbr\u003e I. Stochastic Models and Bayesian Filtering: 2. Stochastic state space model\u003cbr\u003e 3. Optimal filtering\u003cbr\u003e 4. Algorithms for maximum likelihood parameter estimation\u003cbr\u003e 5. Multi-agent sensing: social learning and data incest\u003cbr\u003e 6. Nonparametric Bayesian inference\u003cbr\u003e II. POMDPs: Models and Applications: 7. Fully observed Markov decision processes\u003cbr\u003e 8. Partially observed Markov decision processes\u003cbr\u003e 9. POMDPs in controlled sensing and sensor scheduling\u003cbr\u003e III. POMDP Structural Results: 10. Structural results for Markov decision processes\u003cbr\u003e 11. Structural results for optimal filters\u003cbr\u003e 12. Monotonicity of value function for POMDPs\u003cbr\u003e 13. Structural results for stopping-time POMDPs\u003cbr\u003e 14. Stopping-Time POMDPs for quickest detection\u003cbr\u003e 15. Myopic policy bounds for POMDPs and sensitivity to model parameters\u003cbr\u003e IV. Stochastic Gradient Algorithms and Reinforcement Learning: 16. Stochastic optimization and gradient estimation\u003cbr\u003e 17. Reinforcement learning\u003cbr\u003e 18. Stochastic gradient algorithms: convergence analysis\u003cbr\u003e 19. Discrete stochastic optimization\u003cbr\u003e V. Inverse Reinforcement Learning: 20. Revealed preferences for inverse reinforcement learning\u003cbr\u003e 21. Bayesian inverse reinforcement learning\u003cbr\u003e Appendix A. Short primer on stochastic stimulation\u003cbr\u003e Appendix B. Continuous-time HMM filters\u003cbr\u003e Appendix C. Discrete-time Martingales\u003cbr\u003e Appendix D. Markov processes\u003cbr\u003e Appendix E. Some limit theorems in statistics\u003cbr\u003e Appendix F. Summary of POMDP algorithms\u003cbr\u003e Bibliography\u003cbr\u003e Index.\u003c\/font\u003e\u003c\/p\u003e\r\n\r\n\u003cp\u003e\u003cfont size=\"3\"\u003eSubject Areas: Probability \u0026amp; statistics [\u003ca title=\"See our other books on Probability \u0026amp; statistics\" href=\"https:\/\/freshlyprintedbooks.co.uk\/search?q=%22Probability%20\u0026amp;%20statistics%20%5BPBT%5D%22\"\u003ePBT\u003c\/a\u003e]\u003c\/font\u003e\u003c\/p\u003e\r\n\r\n\r\n\u003c\/font\u003e","brand":"Cambridge University Press","offers":[{"title":"Brand New","offer_id":52472097964312,"sku":"9781009449434","price":76.58,"currency_code":"GBP","in_stock":true}],"thumbnail_url":"\/\/cdn.shopify.com\/s\/files\/1\/0730\/2037\/5320\/files\/9781009449434i.jpg?v=1785716905","url":"https:\/\/freshlyprintedbooks.co.uk\/products\/partially-observed-markov-decision-processes-filtering-learning-and-controlled-sensing-hardback-9781009449434","provider":"Freshly Printed Books","version":"1.0","type":"link"}