The past few years have seen exciting developments in theory and algorithms for estimation in high-dimensional spaces. Beautiful theoretical results show that structured signals, such as sparse vectors and low-rank matrices, can be recovered from relatively small sets of linear observations. These results raise intriguing possibilities for addressing engineering problems in signal and image processing, and beyond.

The goal of this course is to provide students with the theoretical understanding, algorithmic tools, and implementation experience needed to use these tools to solve problems in their own area of interest, or even to begin doing novel work in this area.

Pablo Martinez-Nuevo, Sampling Sparse Bandlimited Signals at the Rate of Innovation 
Tugce Yazicigil, Compressed Sensing Spectrum Scanners
Tanbir Haque, A power efficient front-end employing a deterministic measurement matrix for compressed sensing spectrum scanners
Scott Newton, Effects of Quantization on Signal Recovery in Compressed Sensing Receivers
Alden Goldstein, Compressed Sensing Algorithms for Channel Estimation
Carlos Abad, L1 Edge and Trend Filtering with FISTA
Cun Mu, Solving max norm related optimization problems via ADMM
Wen-Hsiang Shaw, Coreset of Dictionary Learning

Abdulkadir Elmas, Reconstruction of novel target genes of the regulatory proteins via sparse biclustering on microarray expression data
Cheng-Heng Yeh, Exploring Sparse Structure in Neural Connectivity of Fruit Fly Olfactory System
Dawen Liang, Nonparametric Bayesian Dictionary Learning for Machine Listening
Colin Raffel, Towards a perceptually-informed sparse coding of audio signals
Zhuo Chen, Exploration of the low rank and sparse structure in audio with the application of phoneme detection
Yaqing Mao, Texture Classification with Sparse Recovery
Mingyang Sun, Structured Sparsity and Occlusion
Jiawei Chen, Photometric Reconstruction with RPCA
Yan Wang, Propagating labels from ImageNet to 3D point clouds
Juan Liu, Low-Rank Estimation in 3D Urban Dataset

        Introductory material:
            Donoho, Elad and Bruckstein - From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images, SIAM Review 2009
            Davenport, Duarte, Eldar and Kityniok - Introduction to Compressed Sensing, 2011
            Wright, Ma, Saipro, Mairal, Huang, Yan - Sparse Representation for Computer Vision and Pattern Recognition, Signal Processing Magazine 2010

        Hardness results for sparse recovery:
            Natarajan - Sparse Approximate Solutions to Linear Systems, SIAM Journal on Computing 1995
                                    (You can access this through the Columbia Library -- log in using your uni and password).
            Amaldi and Kann - On the Approximability of Minimizing Nonzero Variables or Unsatisfied Relations in Linear Systems
                                    Theoretical Computer Science, 1997

        For a review of convexity, see the Chapters 1 and 2 of Boyd and Vandenberghe's book.

        The material on spark and uniqueness of sparse solutions comes from 
            Donoho and Elad - Optimally Sparse Representation in General (nonorthogonal) Dictionaries via L₁ Minimization, PNAS 2003
            See also, Gorodnitsky and Rao - Sparse Signal Reconstruction from Limited Data using FOCUSS - A Reweighted Minimum Norm Algorithm, IEEE TSP 1997

        Lecture notes! Are available on "New Courseworks". Log in using your uni, go to the ELEN 6886 tab, and go to "Files and Resources".

        Lecture 3 - September 18

                  For some general discussion on the noisy case, two very influential papers -- one from statistics and one from signal processing -- are
                        Tibshirani - Regression shrinkage and selection via the Lasso, JRSS B 1996
                        Chen, Saunders and Donoho - Atomic Decomposition by Basis Pursuit, SIAM Rev. 1998

                  The restricted isometry property is discussed in more detail in
                        Candes and Tao - Near-optimal signal recovery from random projections: Universal encoding strategies? IEEE IT 2006
                        Candes and Tao - Decoding by Linear Programming, IEEE IT 2005

                  For somewhat simpler proofs of the results we saw in class, see
                        Candes - The Restricted Isometry Property and Its Implications for Compressed Sensing, 2008

                  Lecture notes! Are available on "New Courseworks". Log in using your uni, go to the ELEN 6886 tab, and go to "Files and Resources".

Texts: There are no required texts. Students may find the following useful and enjoyable reading:
    Miki Elad: Sparse and Redundant Representations: From Theory to Applications in Image Processing  
                      Elad's book is available digitally for Columbia Students:  http://clio.cul.columbia.edu:7018/vwebv/holdingsInfo?bibId=8579104.
                        (Thanks, Chung-Heng for the link.)
    Jiri Matousek: Lectures on Discrete Geometry (especially the last 4 chapters)   
    Stephen Boyd and Lieven Vandenberghe: Convex Optimization

Grades: Grades will be given based on course participation and a final project. The project should explore in more depth some area not covered in class, and ideally should involve some novel work. The topic could be theory, application, or a mix. The project is open-ended: be creative and show you are thinking about the material! If you have any questions about the suitability of a potential project topic, please contact the instructor.  

We will set aside class time in October for project proposals. At the end of the semester, all students will be required to give a 20 minute presentation and submit a final project report.