ELEN E4835, Spring 2013
Intro. to Adaptive Signal Representations:
    Learning and Optimization for Signal Processing
Mondays and Wednesdays 10:10-11:25 AM
Mudd 834

Instructor:
Prof. John Wright
Email: johnwright@ee.columbia.edu
Office: 716 CEPSR
Office hours: Thursdays 10-11 AM
                      (or by appointment)

Teaching assistant:
Xiao-Ming Wu
Email:  
Office: 
Office hours: TBA
             
                   
The goal of this course is to introduce students to tools from numerical optimization and statistical learning that are useful for intelligently processing signals such as images, videos, audio and more. The course will be taught at at the level of an advanced undergraduate or master's student, with an emphasis on problem formulation.

Readings and Lecture Notes:

        Lecture 1 - January 23 -- What is it all about?
                Course introduction, examples, review of linear algebra
                HW0 released, due January 30 (just a survey!)

                Lecture 1 notes are available on CourseWorks.

         Lecture 2 - January 28 -- More linear algebra review, eigenvectors of symmetric matrices
                We reviewed linear transformations, matrices, systems of equations
                We showed that symmetric matrices have real eigenvalues and a corresponding orthonormal basis of eigenvectors.

                Some fun reading on PageRank and related topics: The $25,000,000,000 Eigenvector: The Linear Algebra Behind Google

         Lecture 3 - January 30 -- The singular value decomposition
                The material from Lectures 2-3 can also be found in the Lecture 1 notes on CourseWorks.

        Lecture 4 - February 4 -- Optimality and Computation of SVD, examples

                Turk and Pentland, Eigenfaces for Recognition
                Basri and Jacobs, Lambertian Reflectance and Linear Subspaces

                Deerwester, Dumais, Furnas, Landauer, Harshman, Indexing by Latent Semantic Analysis
                Hofmann, Probabilistic Latent Semantic Analysis

        Lecture 5 - February 6 -- More SVD examples, nonlinear extensions
                
       


Homework:

        Homework 0 - due January 30 in class [pdf] [tex]
                A quick survey to help us learn more about your background and goals for the course.
                If you had trouble downloading the file, you can turn this in February 4, or email it to me.

        Homework 1 - due February 25, available on CourseWorks


Administrative information:

Prerequisites: Past courses on linear algebra (essential) and probability (desirable). Having already taken signals and systems would be preferable. The course is aimed at undergraduates and MS students. PhD students working in other areas are also welcome. If you're uncertain whether to take the class, please talk to me. 

Texts: There are no required texts. The instructor's lecture notes will be provided on CourseWorks.
    Students will be encouraged to augment their reading with material from:
        Boyd + Vandenberghe, Convex Optimization. (Cambridge)
        Hastie + Tibshirani + Freedman, The Elements of Statistical Learning. (Springer)

Required work: there will be homework, a midterm exam (date TBA) and a final project. 

Homework: a mix of analytical work and Matlab experiments.

Midterm: will be mostly analytical. I will provide a list of topics and some example questions closer to the time.

Course project: students will complete independent course projects. Have fun and be creative in formulating your project! Virtually any idea is ok, as long as it is relevant to the course material and well-executed. A great course project might involve an application of the tools learned in the course to a new real-life problem, or a deeper study of some aspect of the course material. If you have questions or potential topic ideas, please come to office hours. I will be happy to discuss with you. Most of the assigned homework will be given during the first 3/4 of the semester to allow ample time for the final project.

Grade breakdown (subject to minor changes):
 Homework 35%
 Midterm exam 25%
 Final project 30%
 Class participation 10%

Do your own work! CU School of Engineering FAQ on Academic Integrity. Discussing assignments with other students is permissible, but work out the answers on your own. Copying work is impermissible, and will be dealt with harshly if detected.