E4650 Convex Optimization for Engineering

Department of Electrical Engineering, Columbia University

This is the Summer A, 2021 course homepage for EEOR E4650: Convex Optimization for Electrical Engineering.

Note: The official name for the course is “Convex optimization for electrical engineering”. However, the course is suitable for any student within SEAS (or beyond) provided you satisfy the math prerequisite.

Basic information

  • Format: Hybrid

  • Location: TBA

  • Lectures: Tuesday & Thursday, 1:10-3:40 pm.

  • Recitations: As and when needed

  • Instructor: James Anderson (james.anderson@columbia.edu)

  • TA: TBA

  • Office hours: James: TBA

Course announcements

  • 29 March: Welcome to E4650, Summer A’21

Overview

Course description

You will learn to recognize and solve convex optimization problems that arise in applications spanning engineering, mathematics, and computer science. The course can be divided into three parts: theory, applications, and algorithms; A tentative list of topics can be seen below in the schedule. Application areas we will consider include signal processing, statistics and machine learning, finance, energy systems, and optimal control.

Intended audience

This course should benefit anyone who uses or will use scientific computing, data science, or optimization in engineering or related work (e.g., machine learning, finance). Within SEAS, people from the following departments and fields: Electrical Engineering (especially areas like signal processing, smart energy, communications, control); Applied Physics & Applied Math (numerical analysis, high performance computing), Civil Engineering & Engineering Mechanics (structural analysis, optimization, design); Computer Science (especially machine learning, computer graphics, algorithms & complexity, computational geometry); Mechanical Engineering (robotics, control, fluid mechanics); Industrial Engineering & Operations Research. The course may be useful to students and researchers in several other fields as well: Mathematics, Statistics, Finance, Economics.

Textbook

This course closely follows Convex Optimization by Boyd and Vandenberghe, freely available here.

The following books are useful as additional reference texts.

  • Laurent El Ghaoui's lecture notes

  • Convex Optimization Theory, D. Bertsekas (Athena Scientific)

  • Lectures on Convex Optimization, Y. Nesterov (Springer)

Schedule

The tentative syllabus is set out below. Note that the current timetable is only a rough guide, and that there will be an in-class midterm which is not shown.

Chapter numbers refer to the textbook Convex Optimization by Boyd and Vandenberghe.

Week Topic Chapter Additional notes Homework (due)
0 Preparation: Software installation
1 Introduction & logistics
Convex sets
1
2
2 Convex functions 3
3 Convex optimization problems 4
4 Duality 5
5 Applications: Approximation & fitting 6
6 Applications: Statistical estimation 7
7 Numerical linear algebra review
Algorithms: Unconstrained optimization
Appendix C
9
8 Algorithms: Constrained optimization 10
9 Algorithms: Interior point methods 11
10 tba reading tba

Course organization

Prerequisites

Students should have taken APMA E3101 or be comfortable with basic linear algebra at the level of Chapter 2 of Laurent El Ghaoui's notes as well as basic probability.

In addition, familiarity with basic programming will be necessary to complete some homework questions. One of CVX (Matlab), CVXPY (Python), CVXR (R), or Convex.jl (Julia) will be used to write simple scripts. No prior knowledge of CVX is assumed, a recitation on this topic will be provided.

Course requirements

  • Homework assignments: Approximately 1 homework per week, due in 10 days after release.

  • Literature review: A list of topics will be provided, students will be expected to read around the topic, recreate results, and write a short report.

  • Recitations: As and when needed.

  • Final exam: The format is a 48-hour take home exam scheduled for the last week of class.

Grading

  • Homework 40%, Literature review 20%, Final Exam 40%.

Homework

All homework must be submitted via Courseworks. Late homework is not accepted However, you have 5 ‘‘late days’’ to use at your discretion. After these days have been used up, no late homework will be accepted. Each question will be graded on a scale of {0,1,2,3}. We will drop your lowest scoring homework when calculating your average.

You are allowed to work on homework in small groups, but the work you submit (including code) must be your own.