EECS E6892 Topics in Information Processing
Bayesian Models for Machine Learning


Columbia University, Fall 2015


Instructor: John Paisley
Location: 750 Schapiro CEPSR
Time: Thursday 4:10-6:40pm

Instructor office hours: Monday 11am-1pm, Seeley Mudd 422

TA office hours: Dawen Liang,
dliang@ee.columbia.edu,   hrs: Tuesday 2-4pm @ Shapiro CEPSR 7LE4 (7th floor)
                          Aonan Zhang, az2385@columbia.edu,     hrs: Wednesday 7-9pm @ CS TA-Room in Mudd (1st floor)


Synopsis: This intermediate-level machine learning course will focus on Bayesian approaches to machine learning. Topics will include mixed-membership models, latent factor models and Bayesian nonparametric methods. We will also focus on mean-field variational Bayesian inference, an optimization-based approach to approximate posterior learning. Applications of these methods include image processing, topic modeling, collaborative filtering and recommendation systems. We will discuss a selection of these in class.

Text: There is no required text. Much of the material can be found in Pattern Recognition and Machine Learning by Christopher Bishop. Other suggested papers will be posted here.

Grading: 4 homeworks (70% total, 15-15-20-20% split), midterm exam (30%)

Date
Topics covered
Suggested readings
Additional information
9/10/2015
Probability review, Bayes rule, conjugate priors
Bishop Ch. 1.2, 2.1-2.4

Homework 1 (Due 10/2 by 11:59pm)

9/17/2015
Bayesian linear regression, Bayes classifiers, predictive distributions
Bishop Ch. 1.5, 3.1-3.4, 4.5

9/24/2015
Laplace approximation, Gibbs sampling, logistic regression, matrix factorization
Bishop Ch. 4.3-4.4, 11.2-11.3


10/1/2015
EM algorithm, probit regression
Bishop Ch. 9, 12.1-12.2, 4.3.5
Homework 2 (Due 10/16 by 11:59pm)

10/8/2015
EM to variational inference
Bishop Ch. 10


10/15/2015
Variational inference, finding optimal distributions
Bishop Ch. 10


10/22/2015
Midterm exam



10/29/2015
Latent Dirichlet allocation, exponential families
LDA, Bishop Ch. 2.4

Homework 3 (Due 11/20 by 11:59pm)

11/5/2015
conjugate exponential family models, scalable inference
SVI


11/12/2015
Gaussian mixture models
Bishop Ch. 9.2, 10.2


11/19/2015
Bayesian nonparametric clustering
overview, CRP

Homework 4 (Due 12/14 by 11:59pm)

11/26/2015
No class (Thanksgiving)



12/3/2015
Hidden Markov models
Bishop Ch. 13.2, tutorial


12/10/2015
Poisson matrix factorization

ML paper, VI chapter, Tech report