IEOR E4525: Machine Learning for OR & FE (Columbia University)
      

I last taught this advanced-level MS course in spring 2017 in the IE&OR Department at Columbia University. It‘s an elective course for the MS in Financial Engineering and MS in Operations Research programs at Columbia. Because the selection of topics varied over the years there is considerably more material here than could be covered in a single course. Rather than identifying what topics (or subsets of topics) were covered each year, I have simply provided a list of topics that were covered in some version of the course. I have also provided some additional slides / topics that never made it into the course but that I nonetheless used / developed at some point for other purposes. If a link isn’t provided then that simply means I do not wish to post the slides (probably because I am in the “process” of editing them – a process that could take a very long time indeed). I will not be posting solutions to the assignments or code / software so please don’t send me an email asking me to do so!  Finally, please note that I do not have time to answer emails asking me to clarify or explain issues arising in these notes and assignments. A syllabus and description of the course logistics from spring 2017 (when I co-taught the course with Garud Iyengar) can be found here. I’m also grateful to the excellent textbooks of (1) James, Witten, Hastie & Tibshirani (2) David Barber and (3) Christopher Bishop. Many of the figures in the slides below were taken from these sources.

 

Lecture Slides (and Occasional Notes)

  1. Very Brief Introduction to Machine Learning
  2. Regression I (Linear regression, bias-variance decomposition)
  3. Classification I (k-NN, Naïve Bayes, LDA & QDA, logistic regression, optimal Bayes classifier, reduced-rank LDA)
  4. Resampling Methods (Bootstrap; cross-validation)
  5. Regression II (Subset selection, ridge regression, Lasso etc.)
  6. Classification II (Classification & Regression Trees, Bagging, Random Forests & Boosting)
  7. Clustering
  8. An Introduction to Causality
  9. Support Vector Machines
  10. Kernels & the Kernel Trick (including reproducing kernel Hilbert spaces (RKHS))
  11. The EM Algorithm (Notes and slides)
  12. Dimension Reduction Methods (PCA, kernel PCA, recommender systems and matrix factorization, PageRank)
  13. Hidden Markov Models (HMMs)
  14. Bayesian Models and MCMC (Notes and slides from my Monte-Carlo Simulation course)
  15. Introduction to Graphical Models (Directed acyclic graphs (DAGs) and Markov random fields)
  16. Variational Inference (KL divergence, variational Bayes, expectation propagation)

 

 

Assignments (from 2017 version of course)

I’ll get around to posting them soon.