From the preface: This is a text on stochastic processes for graduate students in science and engineering, including mathematics and statistics. It has become somewhat commonplace to find growing numbers of students from outside of mathematics enrolled along with mathematics students in our graduate courses on stochastic processes. In this book we seek to address such a mixed audience. For this purpose, in the main body of the text the theory is developed at a relatively simple technical level with some emphasis on computation and examples. Sometimes to make a mathematical argument complete, certain of the more technical explanations are relegated to the end of the chapter under the label theoretical complements. This approach also allows some flexibility in instruction. A few sample course outlines have been provided to illustrate the possibilities for designing various types of courses based on this book. The theoretical complements also contain some supplementary results and references to the literature.
Measure theory is used sparingly and with explanation. The instructor may exercise control over its emphasis and use depending on the background of the majority of the students in the class. Chapter 0 at the end of the book may be used as a short course in measure theoretical probability for self study.
Chapter applications, appearing at the end of the chapters, are largely drawn from physics, computer science, economics, and engineering. There are many additional examples and applications illustrating the theory; they appear in the text and among the exercises.
Some of the more advanced or difficult exercises are marked by asterisks. Many appear with hints. Some exercises are provided to complete an argument or statement in the text. Occasionally certain well-known results are only a few steps away from the theory developed in the text. Such results are often cited in the exercises, along with an outline of steps, which can be used to complete their derivation.
Chapter headings: I. Random walk and Brownian motion. II. Discrete-parameter Markov chains. III. Birth-death Markov chains. IV. Continuous-parameter Markov chains. V. Brownian motion and diffusions. VI. Dynamic programming and stochastic optimization. VII. An introduction to stochastic differential equations. O. A probability and measure theory overview.