Understanding Markov Chains, 2nd Edition

This book offers a comprehensive undergraduate-level introduction to discrete and continuous-time Markov chains, emphasizing first step analysis and its applications to average hitting times and ruin probabilities. It explores core topics such as recurrence, transience, stationary distributions, and branching processes, starting with detailed examples like gambling processes and random walks. Additional chapters cover discrete-time martingales, their connection to exit times, and spatial Poisson processes. With clear explanations, practical examples, 138 exercises, and 9 fully solved problems, the book is ideal for students seeking a solid foundation in stochastic processes.