Markov processes / James R. Kirkwood.

Includes bibliographical and index.

1: Review of Probability;
2: Discrete-Time, Finite-State Markov Chains;
3: Discrete-Time, Infinite-State Markov Chains;
4: Exponential Distribution and Poisson Process;
5: Continuous-Time Markov Chains;
6: Reversible Markov Chains;
Bibliography.

The book begins with a review of basic probability, then covers the case of finite state, discrete time Markov processes. Building on this, the text deals with the discrete time, infinite state case and provides background for continuous Markov processes with exponential random variables and Poisson processes. It presents continuous Markov processes which include the basic material of Kolmogorov’s equations, infinitesimal generators, and explosions. The book concludes with coverage of both discrete and continuous reversible Markov chains.