EECS E6892 Topics in Information Processing
Bayesian Models for Machine Learning
Columbia University, Spring 2014
Instructor: John Paisley
Location: 403 International Affairs Building
Time: Thursday 4:106:40
Office hours: Monday 1112, Shapiro CEPSR 712
TA: Dawen Liang, dl2771@columbia.edu
Synopsis:
This course provides an introduction to Bayesian approaches to machine
learning. Topics will include mixedmembership models, latent factor
models and Bayesian nonparametric methods. We will also focus on
meanfield variational Bayesian inference, an optimizationbased
approach to approximate posterior learning. Applications of these
methods include image processing, topic modeling, collaborative
filtering and recommendation systems. We will discuss a selection of
these in class.
Text:
There is no required text. There will be suggested readings from the
literature, and much of the material will come from the following two
textbooks:
Christopher Bishop, Pattern Recognition and Machine Learning
David MacKay, Information Theory, Inference, and Learning Algorithms (can be found online for free)
Grading: 4 homeworks (15% each), final project (40%, writeup and short presentation)
Date

Topics covered

Suggested readings

Additional information

1/23/2014

Probability review, Bayes rule, conjugate priors, exponential family

Bishop Ch. 1.2, 2.12.4
MacKay Ch. 2, 3, 23


1/30/2014

Bayesian approaches to regression and classification

Bishop Ch. 1.5, 3.13.4, 4
MacKay Ch. 27

Homework 1 (Due Feb. 13)

2/6/2014

Hierarchical models, matrix factorization, sparse regression models, sampling

Bishop Ch. 3.5, 7.2, 11.211.3
paper 1 (for more details), paper 2


2/13/2014

EM algorithm, mixture models, factor models

Bishop Ch. 9, 12.112.2
MacKay Ch. 22

Homework 2 (Due Feb. 27)

2/20/2014

Variational inference I, mixture of Gaussians

Bishop Ch. 10.110.3
MacKay Ch. 33 

2/27/2014

Variational inference II, conjugate exponential models, latent Dirichlet allocation 
Bishop Ch. 10.4
paper 1, chapter (D. Blei)

Homework 3 (Due March 13)

3/6/2014

Variational inference III, approximations for nonconjugate models 
Bishop 10.510.6
paper 1 (for more details), notes


3/13/2014

Variational inference IV, inference for big data sets 
paper 1 (through Sec. 3.2)
chapter (example application)


3/20/2014

No class (spring break)



3/27/2014

Bayesian nonparameterics I, Dirichlet process and Chinese restaurant process

slides, notes
paper1, paper2

Homework 4 (Due April 17)

4/3/2014

Bayesian nonparameterics II, Dirichlet process and stickbreaking

slides, paper, notes


4/10/2014

Bayesian nonparameterics III, Gaussian process

book Chapters 1, 2, 4
slides


4/17/2014

Bayesian nonparameterics IV, beta process and sparse latent factor models

slides, paper1, paper2


4/24/2014

No class



5/1/2014

Project presentations I & II (Shapiro CEPSR 414)


Project writeup due May 5

