Reinforcement Learning: An Introduction

The significantly expanded and updated new edition of a widely used text on Reinforcement Learning, one of the most active research areas in artificial intelligence.

Reinforcement Learning is a computational approach to learning where an agent attempts to maximize the total amount of reward it receives while interacting with a complex, uncertain environment. In "Reinforcement Learning," Richard Sutton and Andrew Barto provide a clear and straightforward presentation of the central ideas and algorithms of the field. This second edition has been significantly expanded and updated, with new topics covered and other topics revised.

Like the first edition, this second edition focuses on the fundamental online learning algorithms, with the mathematical material highlighted in shaded boxes. Part I covers as much as possible about Reinforcement Learning without going beyond the tabular case, for which exact solutions can be found. Many of the algorithms presented in this part are new to the second edition, including UCB, Expected Sarsa, and Double Learning. Part II extends these ideas to function approximation, with new sections on topics such as artificial neural networks and the Fourier basis, and provides an expanded treatment of off-policy learning and policy gradient methods. Part III includes new chapters on the relationships of Reinforcement Learning to psychology and neuroscience, as well as an updated chapter on case studies that includes AlphaGo and AlphaGo Zero, playing Atari games, and the betting strategy of IBM Watson. The final chapter discusses the future societal impacts of Reinforcement Learning.