Mark H. Liu is an Associate Professor of Finance, the (Founding) Director of the MS Finance Program at the University of Kentucky. He obtained his Ph.D. in finance from Boston College in 2004 and his M.A. in economics from Western University in Canada in 1998. Dr. Liu has more than 20 years of coding experience and is the author of two books: Make Python Talk (No Starch Press, 2021) and Machine Learning, Animated (CRC Press, 2023).

List of Figures

Preface

Acknowledgments

Section I Rule-Based A.I.

Chapter 1 Rule-Based AI in the Coin Game

Chapter 2 Look-Ahead Search in Tic Tac Toe

Chapter 3 Planning Three Steps Ahead in Connect Four

Chapter 4 Recursion and MiniMax Tree Search

Chapter 5 Depth Pruning in MiniMax

Chapter 6 Alpha-Beta Pruning

Chapter 7 Position Evaluation in MiniMax

Chapter 8 Monte Carlo Tree Search

Section II Deep Learning

Chapter 9 Deep Learning in the Coin Game

Chapter 10 Policy Networks in Tic Tac Toe

Chapter 11 A Policy Network in Connect Four

Section III Reinforcement Learning

Chapter 12 Tabular Q-Learning in the Coin Game

Chapter 13 Self-Play Deep Reinforcement Learning

Chapter 14 Vectorization to Speed Up Deep Reinforcement Learning

Chapter 15 A Value Network in Connect Four

Section IV AlphaGo Algorithms

Chapter 16 Implement AlphaGo in the Coin Game

Chapter 17 AlphaGo in Tic Tac Toe and Connect Four

Chapter 18 Hyperparameter Tuning in AlphaGo

Chapter 19 The Actor-Critic Method and AlphaZero

Chapter 20 Iterative Self-Play and AlphaZero in Tic Tac Toe

Chapter 21 AlphaZero in Unsolved Games

Bibliography