Columbia University Fu Foundation School of Engineering and Applied Science
APMA E4300: Introduction to Numerical Methods
Spring 2009
Lectures: Tuesdays and Thursdays, 1:10 - 2:25 pm
Location: 833 S. W. Mudd
Instructor:
Edmond Chow
E-mail:
<>
(the best way to reach me)
Office Hours:
No Office Hours on Sat. May 2.
Remaining Office Hours: May 7, 1:00-2:30 pm (833 Mudd)
and May 9, 2-5 pm (Engineering Library).
Call me if you can't find me.
Appointments can also be made for other times, and you can also
catch me after each class.
Teaching Assistants:
Francois Monard <>,
Office Hours: Mondays 3-4 pm
Weiwei Shen <>,
Office Hours: Thursdays 3-4 pm
Timur Dykhne <>,
Office Hours: Wednesdays 2-3 pm
TA office hours are held in 287 or 292 Engineering Terrace.
Announcements (in reverse chronological order)
- [Apr 28] No office hours on Sat May 2 (work on the practice exam!).
Instead, I will be available on Thurs May 7 (when you're likely to
have more questions) at 1:00-2:30 in 833 Mudd. I will also be
available on Sat May 9 at the regular time.
- [Apr 28] Complete readings have been posted below.
- [Apr 26] Recitation session with Francois Monard on May 5 in our
classroom at our regular class time (study week).
- [Apr 26] Sample Final Exam posted on CourseWorks.
- [Apr 16] Assignment 7 has been posted.
- [Apr 14] Regions of Absolute Stability,
by Tony Humphries (McGill)
- [Apr 7] Assignment 6 has been posted.
- [Mar 24] Midterm solutions and grade histogram are posted on CourseWorks.
If you score is 10 or higher, then you are doing well. If your score is 6 or lower,
please see me or the TAs for extra help.
- [Mar 24] The Matlab Help Room has a Matlab tutorial taught by a MathWorks representative
on Friday, March 27 at 11 am in 252 Engineering Terrace.
- [Mar 24] Assignment 5 has been posted. The readings should be very helpful.
- [Mar 9] An Introduction
to the Conjugate Gradient Method Without the Agonizing Pain. This is
a long but easy to follow introduction. It also contains all the elementary linear
algebra you need to understand the CG method.
- [Mar 9] Matlab m-file for generating a 7-point 3-D Laplacian matrix
- [Mar 4] List of topics we've covered so far.
- [Mar 1] "Solutions" for assignments 1 and 2 are
here. It's actually more (and less)
than the solutions, and you are responsible for knowing this material.
- [Mar 1] Additional chapters from the Ascher and Greif supplementary text
have been posted on CourseWorks.
- [Mar 1] The Midterm Exam will be on Tue March 10, in class. The exam will be closed book. The only allowed aids are a) one handwritten sheet of notes (8.5 x 11 in, both sides), and b) a non-graphing, non-programmable scientific calculator.
- [Feb 13.] There is an assignment drop box in the APAM office (2nd floor of Mudd).
If you will not be in class on a day an assignment is due, have someone deposit
your assignment in this box no later than 12:40 pm on the due date.
- [Feb 4.] Secant method convergence rate derivation.
- Advice for Assignment 1, Question 4. Here is
regula_falsi.m which we developed in class.
The hard part of this question is estimating the convergence rate. Modified
regula falsi has the very realistic property of non-smooth convergence.
For example, it may seem to not make progress to the root for a few steps,
and then suddenly make progress. This behavior is related to when the
stagnating side is halved. Use your creativity to think of a good way
to estimate the convergence rate.
- One student had a question about my comment that subtractive cancellation
is not always a bad thing. (Unfortunately, I did not have time to answer it -
my apologies.) Cancellation occurs when two nearly equal numbers are subtracted.
If x1 and x2 are computed results with some uncertainty, then subtraction
magnifies this uncertainty (the relative error of this uncertainty is very large).
If x1 and x2 are error free, then the cancellation is harmless (assuming the
result is computed accurately). For more information, the best reference for
these ideas is Chapter 1 of Nick Higham's book,
Accuracy and Stability
of Numerical Algorithms (in the first edition, Section 1.7 is on cancellation).
- If you want more examples of looking at floating-point numbers using Matlab and
hexadecimal format, see Moler, Section 1.7.
- There are three copies of the textbook on reserve at the Engineering Library.
- The textbook is on the shelves at the bookstore (I checked on Jan 27).
Readings
An asterisk (*) denotes helpful but optional readings.
- Introduction and Error Analysis I
- Heath, Sections 1.1, 1.2.1 to 1.2.4, 1.3 (we will cover Sections 1.2.5 to 1.2.7 later)
- *Ascher and Greif, Chapters 1 and 2 (more detailed coverage than Heath)
- *Moler, Section 1.7 (Floating Point Arithmetic). This has good examples of looking
at floating point numbers (using hexadecimal "digits") in Matlab.
- *Probably the most widely read and useful document on IEEE floating-point is this one:
What Every Computer Scientist
Should Know About Floating-Point Arithmetic.
It is certainly easier than reading the IEEE standard itself!
- Nonlinear Equations
- Heath, Sections 5.1 to 5.5
- *Ascher and Greif, Sections 3.1 to 3.3
- *Moler, Chapter 4
- Linear Systems
- Heath, Sections 2.1 to 2.5 (Direct methods)
- Heath, Sections 11.5.1 to 11.5.4 (Basic iterative methods)
- *Moler, Sections 2.1 to 2.10
- Error Analysis II
- Heath, Sections 1.2.5 to 1.2.7
- Large-scale linear and nonlinear systems
- Heath, Section 11.4 (Direct methods for sparse systems)
- Heath, Sections 11.5.5 and 11.5.6 (Conjugate gradient method)
- Heath, Sections 5.6.1 and 5.6.2 (Methods for nonlinear systems)
- Polynomial and Spline Interpolation
- Required: Ascher and Greif, Chapters 10 and 11 (not 11.3 and 11.4)
- Required: Heath, 7.1 to 7.4.2
- Numerical Differentiation
- Required: Ascher and Greif, Chapter 14 (not 14.3)
- Required: Heath, 8.6
- Numerical Integration
- Required: Ascher and Greif, Chapter 15, pages 401-419
- Required: Heath, 8.3 (not 8.3.2 and 8.3.4)
- Numerical Solution of Ordinary Differential Equations
- Required: Heath, 9.1, 9.3 (not 9.3.7 and 9.3.9)
- Required: Ascher and Greif, 16.2, 16.4, 16.5 (up to page 478)
- Boundary Value Problems
- Required: Heath, 10.3, 10.4, 11.3.1. Heath's finite difference
example in Section 10.4 is very poor, since it only involves a single
point. Look at our example in class, or Heath 11.3.1 for
a 2-D example.
- Least Squares
- Required: Heath, 3.1, 3.2.1, 3.4
- Eigenvalue Problems
- Required: Heath, 4.5.1 to 4.5.3
Assignments
Assignments are due at the beginning of class on the date indicated.
- Assignment 1, due Thu Feb 5
- Assignment 2, due Tue Feb 17
- Assignment 3, due Tue Mar 3
- MIDTERM, Tue Mar 10, in class
- SPRING BREAK, Mar 16-20
- Assignment 4, due Tue Mar 24. You also
need these two files: rb11.mat and a2.mat
- Assignment 5, due Tue Apr 7
- Assignment 6, due Thu Apr 16
- Assignment 7, due Thu Apr 30
Course Description
Introduction to fundamental algorithms and analysis of numerical methods
commonly used by scientists, mathematicians and engineers. This course
is designed to give a fundamental understanding of the building blocks
of scientific computing that will be used in more advanced courses in
scientific computing and numerical methods for PDE's. Topics include
numerical solutions of algebraic systems, linear least-squares, eigenvalue
problems, solution of non-linear systems, interpolation,
numerical integration and differentiation, initial value problems and
boundary value problems for systems of ODE's. All programming exercises
will be in Matlab.
Prerequisites
Vector calculus (MATH V1201), Ordinary differential equations
(MATH E1210), and Linear algebra (APMA E3101) or their equivalents.
A working knowledge of Matlab will also be helpful.
Topics
Topics will be covered in approximately this order:
- Introduction
(1 lecture)
- Error Analysis I:
Truncation error, roundoff error, and floating point computation
(1 lecture)
- Solution of nonlinear equations
(4 lectures)
- Solution of linear systems
(4 lectures)
- Error analysis II: Stability, conditioning, and backward error
(1 lecture)
- Large-scale linear systems and systems of nonlinear equations
(2 lectures)
- MIDTERM EXAM
- Polynomial interpolation and splines
(3 lectures)
- Numerical differentiation
(1 lecture)
- Numerical integration
(2 lectures)
- Numerical solution of ordinary differential equations
(4 lectures)
- Solution of two-point boundary value problems
(1 lecture)
- Least squares problems
(1 lecture)
- Matrix eigenvalue problems
(1 lecture)
Grading
50% Assignments (7 during the semester)
20% Midterm (Tue Mar 10, in class)
30% Final (Projected date: Tue May 12, 1:10 pm, 3 hours)
The Final Exam will cover material from the entire semester.
Late assignments will not be graded except
in exceptional circumstances (with documentation).
Requests for regrading of assignments or exams
must be accompanied by a written justification. Be careful that regrading
in many cases can result in your grade going down!
Textbook and References
Required Text
Our in-class lectures will approach the material differently than the
presentation in the textbook
and thus you will have at least two perspectives on our topics.
I will make a lot of references to the textbook, such as what sections
to read to complement the lectures and what problems to try in the textbook.
This is admittedly an expensive textbook, but it is also very clearly
written.
As the subtitle of the textbook says, this book is both an introduction
and a survey. As a survey, it covers about twice as many topics as
we can cover in class. After this course is over, however, the text
will also serve as a great reference and starting point for additional
techiques for your work or research (if you decide to hold on to the
book!). As an introduction, the text generally does not take topics
to the same depth as many other textbooks, particularly in analysis.
Thus we will generally cover methods and techniques not in the text
(especially when we study the numerical solution of ODE's) and I will
provide additional resources in these cases.
Supplementary Text
-
Uri M. Ascher and Chen Greif, A First Course on Numerical Methods
Some of the material in our course is covered in more detail in this
supplementary text than in the required text. The authors are letting
us use this text for free! (I'm sure you did not want to buy a second
textbook!) Selected chapters are available on the CourseWorks web
site for this course, and I will add more if necessary.
Please do not distribute the files for this text
outside of class. The text is not yet published and, in return for the
authors' generosity, it would be nice if we could give them our comments
and suggestions on the text (I can collect your comments or you can write to
the authors directly at the end of the course).
Additional References
- Course website
for APMA E4300 taught last spring by Prof. Marc Spiegelman. There is
a wealth of resources here!
- Cleve Moler's
Numerical Computing with Matlab. Chapters of this book are available
for free. This is not really a textbook, but it shows you how to use
Matlab in Scientific Computing, and it might help you on your assignments.
- Other textbooks.
The curriculum for this course is very well established, and there are
several good textbooks (including older editions and reprinted classics
at affordable prices) that cover our material.
If you have access to
other texts, they would help you learn better by showing you the same
material in different ways. Let me know if you have questions about
these other texts. If you wish to pursue graduate work in an area
of numerical analysis, I can also suggest more advanced texts which
you may want to start reading.
Matlab Resources
- Matlab Help Room, operated by Columbia's SIAM Student Chapter.
Hours are Tuesday 4:10-6 pm and Friday 11-1 pm in 252 Engineering Terrace.
- Getting Started with Matlab, available
here.
- Matlab Resources: Tutorials and Beyond, listed
here.
- Kermit Sigmon's Matlab tutorial, available
here.
This is a bit outdated, but has the basic stuff. I used this to learn
Matlab when I was an undergrad.