Lecture Notes on Stochastic Modeling I

Professor Karl Sigman
Department of Industrial Engineering and Operations Research
karl.sigman@columbia.edu
http://www.ieor.columbia.edu/~sigman

Stochastic Modeling I Lecture Notes WEB SITE:
http://www.ieor.columbia.edu/~sigman/stochastic-I.html

Lecture Notes

  1. Introduction to discrete-time Markov chains I
  2. Gambler's ruin problem
  3. Stopping times
  4. Markov chains II: recurrence and limiting (stationary) distributions
  5. Time-reversible Markov chains
  6. Uniform integrability
  7. Poisson processes, elementary renewal theorem with proof
  8. Borel-Cantelli Lemmas
  9. Continuous-time Markov chains
  10. Little's law
  11. Renewal Theory I; inspection paradox, renewal reward theorem
  12. Renewal theory II; central limit theorem for counting processes, stationary renewal processes, key renewal theorem, weak convergence
  13. Regenerative processes
  14. Discrete-time renewal processes
  15. Introduction to Discrete-time martingales; the optional stopping theorem with applications
  16. Using conditional expectations