Statistical and Computational Inverse Problems, Fall 2008

Statistical and Computational Inverse Problems, APAM 6901, Fall 2009

IMPORTANT: The course will be canceled unless more students sign up. Please let your friends know about it. The following syllabus is from the Fall 2008 course. For 2009 we will follow roughly the same format. I will tailor the course content depending upon the students needs.

This course will present an introduction to inverse problems. We adopt a Bayesian viewpoint, which allows us to deal with ill-posedness. Essentially, an inverse problem is ill-posed if small changes in the measured data result in large changes to the reconstructed function.

Days/Times: Tuesday, Thursday, 11- 12:15

Instructor: Ian Langmore

Instructor's website: www.columbia.edu/~il2176

Textbook: Statistical and Computational Inverse Problems by Kaipio and Somersalo. Software: MATLAB

Prerequisites: Strong linear algebra skills. Ability to write basic scientific code.

Further Reading (in no particular order)

Monte Carlo Statistical Methods by Robert and Casella. A comprehensive introduction and advanced topic text. Well-written, with theorems, proofs, and examples. Assumes graduate level knowledge of probability/statistics.
Draper's online books. In particular, Bayesian Modeling, Inference and Prediction provide an introduction with many examples (in my opinion, the examples are too complex and the theory is way too limited).
Numerical Optimization by Nocedal and Wright. A comprehensive introduction to optimization. Recommended to me by Kui-Ren (an optimization guru).
IP Course Notes By Guillaume Bal. An introduction to deterministic inverse problems, with an emphasis on imaging.
IP Course Notes by Fox and Nicholls. A Bayesian approach to inverse problems. We could have used these for the entire course.
Bayesian Data Analysis by Gelman. A standard statistics introduction. Covers quite a bit.

(Last year's) Weekly Outline

Week	Lecture/Reading	HW	Supplimentary Reading
	Part 1: Basics of Ill-Posed Problems
1	Ill-posed problems and noise. Ch. 1 Truncated SVD Regularization. Ch. 2.1, 2.2	HW1 solutions	Chapter 8 in [4]. SVD supplement
2	Tikhonov Regularization. Ch. 2.3 Truncated Iterative Methods. Ch. 2.4	HW2.
3	Implementing these methods with MATLAB HW2 explained Conjugate gradient methods explained		Nocedal/ Wright, [3] Advanced Matlab matrix functions
	Part 2: Intro to Statistical Inversion Theory
4	Review of probability, Appendix B, and on line notes.	HW3 solutions	Short Review of Probability
5 Sept 30, Oct 2	Introduction to Bayesian Modeling, Ch. 3.1
6 Oct 7, 9	Estimators, Ch. 3.1.1 The likelihood function, basic noise models. Ch. 3.2	HW4 solutions m-file
7 Oct 14, 16	Prior Models. Ch. 3.3	HW5 solutions
8 Oct 21, 23	Gaussian Densities, Ch. 3.4	HW6 Solutions	Tensors in Matlab
	Part 3: MCMC, noise modeling, estimation theory (no more h.w.). Project presentations.
9 Oct 28, 30	Introduction to Markov Chains. Ch. 3.6
10 Nov 4,6 No class Tues	Markov Chain Monte Carlo (MCMC) methods. (continue introduction, introduce algorithms) Ch. 3.6
11 Nov 11,13	Metropolis-Hastings algorithm Ch. 3.6.1, 3.6.2 Rate of convergence, [1]		Casella/Robert [1], and Draper [2] Chapters 7-9 from Fox/Nicholls [5]
12 Nov 18, 20	Diagnosing convergence (ch. 9 in [5]) Importance Sampling, [1]		Gelman's "Bugs" package (in R), see also [6]. MCMC diagnoistic (MATLAB) Another MCMC diagnoistic (MATLAB)
13 Nov 25, No class Thursday	Noise Modeling Ch. 5.8, 7.6
14 Dec 2, 4	Project Presentations: 10 minute presentation, 5 minute setup, Q&A A projector will be provided, but chalkboard presentations are just fine. Make your presentation interesting and use the same notation we have been using in clas.
	---------No class finals week--------------