ELEN E4903 Machine
Learning
Columbia University, Spring 2018
Instructor:
John Paisley
Location: 501 Northwest Corner Building
Time: Wednesday 7pm  9:30pm
Office
hours: Monday 1112, Mudd 422
TA's:

Aonan Zhang

az2385@columbia.edu 
hrs: Friday 7  9pm @ CS TA room, Mudd 122A (1st floor)

Ghazal Fazelnia

gf2293@columbia.edu

hrs: Wednesday 9:30  11:30am @ DSI Space, Mudd (4th floor)


Yiyang Li

yl3789@columbia.edu 
hrs: Wednesday 2  4pm @ CS TA room, Mudd 122A (1st floor)


Kejia Shi

ks3403@columbia.edu

hrs: Friday 9:30  11:30am @ CS TA room, Mudd 122A (1st floor)


Sidharth Prasad

sp3591@columbia.edu 
hrs: Monday 5:30  7:30pm @ CS TA room, Mudd 122A (1st floor)


Yiran Shi

ys3077@columbia.edu

hrs: Thursday 4  6pm @ CS TA room, Mudd 122A (1st floor)


Di Lu

dl3152@columbia.edu

hrs: Tuesday 2:40  4:40pm @ CS TA room, Mudd 122A (1st floor)





Synopsis: This course provides an introduction to supervised and unsupervised
techniques for machine learning. We will cover both probabilistic and
nonprobabilistic approaches to machine learning. Focus will be on
classification and regression models, clustering methods, matrix
factorization and sequential
models. Methods covered in class include linear and
logistic regression, support vector machines, boosting, Kmeans
clustering, mixture models, expectationmaximization
algorithm, hidden Markov models, among others. We will cover
algorithmic techniques for optimization, such as gradient and
coordinate descent methods, as the need arises. This class is part of the Topics in Electrical & Computer
Engineering series.
Prerequisites:
Basic linear algebra and calculus, introductorylevel courses in
probability and/or statistics strongly encouraged. Comfort with a programming language (e.g.,
Matlab) will be essential for completing the homework assignments. Not
open to students who have taken COMS 4721, COMS 4771, STATS 4240, STATS 4400 or
IEOR 4525.
Text:
There is no required text for the course. Suggested readings for each
class will be given from the textbooks below. These readings are
meant to be general pointers and may contain more material than we
cover in class.
T. Hastie, R. Tibshirani and J. Friedman, The Elements of Statistical Learning,
Second Edition, Springer.
[link]
C. Bishop, Pattern
Recognition and Machine Learning, Springer. [link]
H. Daume, A
Course in Machine Learning, Draft. [link]
Grading:
5 homework assignments (50%), midterm exam (25%), final inclass exam
(25%). Each homework assignment will have a programming component that
will count significantly toward the final homework grade. The final
inclass exam will focus on material from the second half
of the course.

Date


Topics
covered

Suggested
readings

Additional
Information

Week 1

1/17/18


Introduction, maximum likelihood estimation

ESL Ch. 12; PRML Ch. 2.12.3

Homework 1 out (see Courseworks)




linear regression, least squares, geometric view

ESL Ch. 3.13.2; PRML Ch. 1.1, 3.1

Due February 4 by 11:59pm

Week
2

1/24/18


ridge regression, probabilistic views of linear regression

ESL Ch. 3.33.4; PRML Ch. 3.13.2





biasvariance, Bayes rule, maximum a posteriori

ESL Ch. 7.17.3, 7.10; PRML Ch 2.3


Week 3

1/31/18


Bayesian linear regression

PRML 3.33.5





sparsity, subset selection for linear regression

ESL Ch. 3.33.8


Week
4

2/7/18


nearest neighbor classification, Bayes classifiers

ESL Ch. 13.313.5; CML Ch. 2, 7

Homework 2 out (see Courseworks)




linear classifiers, perceptron

ESL Ch. 4.5; CML 3

Due February 25 by 11:59pm

Week 5

2/14/18


logistic regression, Laplace approximation

ESL Ch. 4.4; PRML Ch. 4.34.5





kernel methods, Gaussian processes

ESL Ch. 6; PRML Ch. 6; CML Ch. 9


Week
6

2/21/18


maximum margin, support vector machines 
ESL Ch. 12.112.3; PRML Ch. 7.1 




trees, random forests 
ESL Ch. 9.2, 15; CML Ch. 1 

Week 7

2/28/18


boosting 
ESL Ch. 10; CML Ch. 11 







Week
8

3/7/18


Midterm exam


Homework 3 out (see Courseworks)






Due March 18 by 11:59pm

Week 9

3/14/18


No class (Spring break)









Week
10

3/21/18


clustering, kmeans

ESL Ch. 14.3; PRML Ch. 9.1; CML Ch. 13





EM algorithm, missing data

ESL Ch. 8.5; PRML Ch. 9.39.4


Week 11

3/28/18


mixtures of Gaussians

PRML Ch. 9.2; CML Ch. 14

Homework 4 out (see Courseworks)




matrix factorization

Review article

Due April 10 by 11:59pm

Week
12

4/4/18


nonnegative matrix factorization

ESL Ch. 14.6; Review article





latent factor models, PCA and variations

ESL Ch. 14.5; PRML Ch. 12.112.3


Week 13

4/11/18


Markov models

PRML Ch. 13.1





hidden Markov models

PRML Ch. 13.2


Week
14

4/18/18


continuous statespace models

PRML Ch. 13.3





association analysis

ESL Ch. 14.2; Book chapter


Week 15

4/25/18


Final inclass exam 








