IEOR 6711. Stochastic Modeling I

Fall, 2009

Instructor: Professor Karl Sigman
Department of Industrial Engineering and Operations Research

Office hours: Wednesdays 3-4PM, 310 S.W. Mudd Building
Phone: (212) 854-3556
FAX: (212) 854-8103

Office hours: Fridays 10:30AM --12:30 PM

Email: yl2342@columbia.edu

Homework Assignments

1. HMWK1.
2. HMWK2.
3. HMWK3.
4. HMWK4.
5. HMWK5.
6. HMWK6.
7. HMWK7.

Homework Solutions

1. HMWK1.
2. HMWK2.
3. HMWK3.
4. HMWK4.
5. HMWK5.
6. HMWK6.
7. HMWK7.

Lecture Notes

1. Introduction to discrete-time Markov chains I
2. Gambler's ruin problem
3. Stopping times
4. Markov chains II: recurrence and limiting (stationary) distributions
5. Time-reversible Markov chains
6. Uniform integrability
7. Poisson processes, elementary renewal theorem with proof
8. Borel-Cantelli Lemmas
9. Continuous-time Markov chains
10. Little's law
11. Renewal Theory I; inspection paradox, renewal reward theorem
12. Renewal theory II; central limit theorem for counting processes, stationary renewal processes, key renewal theorem, weak convergence
13. Regenerative processes
14. Discrete-time renewal processes

Tablet PDF Lectures

1. Thursday, October 1 (Poisson process)
2. Tuesday, October 6 (Poisson process continued, begining CTMC)

Midterm Exam Information

• Exam I: Thursday, October 22, in-class; 1 hour and 15 minutes.
• Solutions to Midterm I.
• Exam II: Tuesday, November 24, in-class; 1 hour and 15 minutes.

Final Exam Information

• DATE: Thu 12/17/09 7:10pm-10:00pm. 333 Uris (Date Determined by Columbia University (not Professor Sigman).)