As described in the SEAS Bulletin, operations research is concerned with quantitative decision problems, generally involving the allocation and control of limited resources, often in the presence of significant uncertainty. This course is part of a two-course introduction to operations research. This course can be regarded as a sequel to IEOR 3608 - Introduction to Mathematical Programming - or IEOR 4004 - Introduction to Operations Research: Deterministic Models, both of which are introductions to deterministic optimization models in operations research.

This course also can serve as part of a year-long introduction to probability theory and its applications, being a sequel to IEOR 3600 - Introduction to Probability and Statistics (or, alternatively, IEOR 4105, SIEO 3658 or SIEO 4150). Building on the first course in probability, which is a prerequisite, this course is an introduction to stochastic processes and their applications in engineering.

In a first course on probability theory (such as IEOR 3600) we learn about random variables and their probability distributions. In that first course, attention is usually focused on only one or two random variables. In this subsequent course we will extend the focus to stochastic processes, which are collections of random variables, usually indexed by time. (In stochastic process models, time can be regarded as either discrete or continuous.) For example, we might use stochastic processes to model the evolution of a stock price over time, the damage claims received by an insurance company over time, the work-in-process inventory in a factory over time or the number of calls waiting in a telephone call center over time, all of which evolve with considerable uncertainty.

Among the stochastic processes to be considered are discrete-time Markov chains, random walks, continuous-time Markov chains, Poisson processes, birth-and-death processes, renewal processes, renewal-reward processes, Brownian motion and geometric Brownian motion. Among the engineering applications to be considered are queueing, inventory and finance.