Lecture Notes on Stochastic Modeling I
Professor Karl Sigman
Department of Industrial Engineering and Operations Research
karl.sigman@columbia.edu
http://www.ieor.columbia.edu/~sigman
Stochastic Modeling I Lecture Notes WEB SITE:
http://www.ieor.columbia.edu/~sigman/stochastic-I.html
Lecture Notes
- Introduction to discrete-time Markov chains I
- Gambler's ruin problem
- Stopping times
- Markov chains II:
recurrence and limiting (stationary) distributions
- Time-reversible Markov chains
- Uniform integrability
- Poisson processes, elementary renewal theorem with proof
- Borel-Cantelli Lemmas
- Continuous-time Markov chains
- Little's law
- Renewal Theory I; inspection paradox, renewal reward theorem
- Renewal theory II; central limit theorem for
counting processes, stationary renewal processes, key renewal theorem, weak convergence
- Regenerative processes
- Discrete-time renewal processes
- Introduction to Discrete-time martingales; the optional stopping theorem with applications
- Using conditional expectations