Theory of Probability

MATH-UA 233

Date, time, location

Monday, Tuesday, Wednesday and Thursday, 2:10PM-4:15PM, 25W4, Room C-20

Exams

Midterm: Monday, 7/25/2016, 2:10PM-4:15PM, 25W4, Room C-20
Will cover all material through Thursday 7/21/2016.

Final: Thursday, 8/11/2016, 2:10PM-4:15PM, 25W4, Room C-20
Will cover all material through Tuesday 8/9/2016.

Both are closed book, closed notes exams. No devices or documents are allowed while taking an exam. No collaboration is permitted.

Textbook

Sheldon Ross's A First Course in Probability (9th edition). The 8th edition of the book should suffice as well.

Harvard Video Lectures

Prof. Joseph Blitzstein has a great collection of video lectures online at Harvard Stat 110 that cover much of the same material we will cover.

Acknowledgements and Helpful Links

The lecture notes and other materials in this course are drawn from the materials from the following two courses. All errors are mine.

  1. NYU Theory of Probability (Tim Austin)
  2. NYU Theory of Probability (Brett Bernstein)

Office hours

Monday and Wednesday, 4:25PM-5:25PM, 25W4, Room C-20 or contact Vlad to make an appointment.

Grading

  • Homework/Quizzes: 30%
  • Midterm: 35%
  • Final Exam: 35%

Tentative Syllabus & Notes

  1. [Jul 5 , 6] Counting, combinations and permutations [Ch.1]
    2. Lecture 1 Lecture 2
  2. [July 7, 11, 12] Axioms of probability [Ch. 2]
    3. Lecture 3 Lecture 4 Lecture 5
  3. [Jul 13,14, 18] Conditional probability [Ch. 3]
    4. Lecture 6 Lecture 7 Lecture 8
  4. [Jul 19, 20, 21] Discrete random variables and expectations [Ch.4]
    5. Lecture 9: Ross, Prop 3.4.1, Examples 3.4e, 3.4g, 3.4f, 3.4j (the problem of the points), 4.1a, 4.1b, 4.1c, Prop Ross p. 117.
    Lecture 10
    Lecture 11: Ross, Section 4.9 (Expectation as a sum of several RVs, Prop 4.9.1 Corollary 4.9.2 (linearity of expectation), Example 4.9d, equation (7.3.1), Examples 7.2h, 7.2j, Section 4.6 (Bernoulli & binomial RVs), Examples 4.6b, 4.6d, Prop p 132 and Example 7.2e, p 284, Example 4.6c, Example 7.3a (variance of binom RV); Prop 6.1 (maximum of binom RV); Example 4.6g.
  5. [Jul 25] Midterm Solutions
  6. [Jul 26, 27, 28, Aug 1] Continuous random variables [Ch. 5]
    Lecture 12: Ross, Prop 4.8.1 (geometric RVs), Section 4.8.3 (hypergeometric RVs), Example Ross 4.8h (maximum likelihood estimate), Example 4.8j or 7.2g (E[X] and Var (X) for a hypergeometric RV; approximating one RV with another (p 153); hypergeometric RV converges to a binomial distribution (Ross p 153-4); Section 4.7 (Poisson RV and Poisson approximation; Proposition Ross p 136 (binom converges to Poisson; expectation and variation of Poission RV; Examples 4.7a and 4.7b.
    Lecture 13: Ross, p 179 (CDF and PDF related by FTC), identities involving PDF, p 176, 177, Section 5.3 (uniform RVs); Lecture 13 Review Problems, adapted Example 5.7b. Examples 5.1a, 5.1b, 5.3d, 5.3c, Section 5.2, (Expectation and variance of continuous RVs), Example 5.2a, Ross Prop 5.2.1, Lemma 5.2.1, Example 5.2c.
    Lecture 14: Ross, Prop p 183 (Var aX+B = a^2 Var (X)), Example 5.1d, Theorem 5.7.1, Example 5.7c; Section 5.4 (Normal RVs), Proposition Ross p 188; E[X] and Var (X) of normal RV (Example 5.4a), Lecture 14 Review Problems
    Lecture 15: Ross, identity (5.4.1) p 190, Example 5.4f, Exponential RVs: Propositions Ross p 198; Exponential approximation to the geometric (covered in class but not from Ross); Prop p 199 (the memoryless property), Example 5.5d, Example (cf. Problem 5.13 in Ross)
  7. [Aug 2, 3] Jointly distributed random variables [Ch. 6]
    8. Lecture 16: Ross, identity (5.1.2), Prop p 221, Example 6.1f (Multinomial distribution), Proposition Ross p 224 Lecture 16 Review Problems
    Lecture 17: Ross, Prop 7.2.1, Independence of RVs Section 6.2, Prop, pp. 228-9, Ex 6.2b (the splitting property of the Poisson distribution); Ex 6.2d (Buffon's needle); Section 6.7 (Transformations of jointly continuous RVs) Lecture 17 Review Problems
  8. [Aug 4,8] More properties of expectation and generating functions [Ch. 7]
    9. Lecture 18 Lecture 19
  9. [Aug 9] Limit theorems [Ch. 8]
    10. Lecture 20
  10. [Aug 10] Markov chains and other additional topics [Ch. 9]
  11. [Aug 11] Final

Homework

Collaboration is allowed (and encouraged) on all homework (but not the quizzes, midterm and final). Nevertheless, you must write solutions on your own. Homework will be assigned several times per week, and collected at the beginning of the class on the due date. There are no extensions available as solutions will be posted promptly. The assigned problems are the same in the 8th edition of the book except as otherwise noted.

  1. Homework 1
  2. Homework 2
  3. Homework 3
  4. Homework 4: Ross, Chapter 2, Problems 45, 52, 54 and Theoretical Exercise 18 due Tuesday, July 12.
  5. Homework 5: Ross, Chapter 2, Theoretical Exercises 11 and 12 due Wednesday, July 13.
  6. Homework 6: Ross, Chapter 3, Problems 2, 5, 6 and 29.
  7. Homework 7: Ross, Chapter 3, Problems 13, 30, 49, 56, 70 and Theoretical Exercises 1 and 8.
  8. Homework 8 Ross, Chapter 3, Problems 66, 78 (hint for part (b): what are all the possible situations after the first two games?), 86, Theoretical Exercise 22 (note typo: it should say "Show that if the weather...").
  9. Homework 9
  10. Homework 10: Ross, Chapter 4, Problems 25, 27, 28, 32 due Thursday, July 21
  11. Homework 11 due Tuesday, July 22: Ross, Chapter 4, Problems 38(a), 44, 50, 54 due Tuesday, July 26.
  12. Homework 12: Extra Credit due Thursday, August 4.
  13. Homework 13: Ross, Chapter 4, Problems 57, 60 due Wednesday, July 27.
  14. Homework 14: Ross, Chapter 5, Problems 2, 4, 6 (a and b), 10 due Thursday, July 28.
  15. Homework 15
  16. Homework 16: Ross, Chapter 5, Problems 39, 41 and Theoretical Exercises 13 and 17 due Tuesday, August 2.
  17. Homework 17: Ross, Chapter 6, Problems 2, 4, 8, 19 due Wednesday, August 3.
  18. Homework 18: Ross, Chapter 6, Problems 14, 18, 20, 44, 56 parts (a and b), and Theoretical Exercise 15 due Thursday, August 4.
  19. Homework 19: Ross, Chapter 5, Problems 20, 24, 26, Chapter 6, Problem 33 and Theoretical Exercise 17, Chapter 7, Problems 30, 38, 42 due Monday, August 8. These problems are the same in the 8th edition of the book EXCEPT for Problem 6.33, which is number 6.32 in the 8th edition.
  20. Homework 20: Ross, Chapter 6, Problems 38, 42, 43 (assume s=1, so Gamma(s, alpha) just means Exp(alpha)), Theoretical Exercises 20 (read 'conditional PDF' for 'conditional distribution), 21 (this one tells you what you need to know about Gamma RVs) due Tuesday, August 9.
  21. Homework 21: Ross, Chapter 7, Problems 49, 56 (hint: Law of Total Expectation), Theoretical Exercise 26 (you may assume X and Y are both discrete), Chapter 8, Problems 1, 6 due Wednesday, August 10.

Homework Solutions

  1. Homework 1 Solutions
  2. Homework 2 Solutions
  3. Homework 3 Solutions
  4. Homework 4 Solutions
  5. Homework 5 Solutions
  6. Homework 6 Solutions
  7. Homework 7 Solutions
  8. Homework 8 Solutions
  9. Homework 9 Solutions
  10. Homework 10 Solutions
  11. Homework 11 Solutions
  12. Homework 12 Solutions: Please see me after the submission deadline if you would like to discuss this extra credit problem.
  13. Homework 13 Solutions
  14. Homework 18 Solutions
  15. Homework 20 Solutions
  16. Homework 21 Solutions

Quizzes

Quizzes will be given throughout the course. They will typically be based directly on the lectures and homework, differing only by some minor changes. The key theorems and proofs to remember are listed in the following document.

  1. List of Key Theorems
  2. Quiz 1 Solutions
  3. Quiz 2 Solutions

Review Materials

  1. Prerequisites Review

Vladimir Kobzar
vk283@nyu.edu