Math 632

Notes on Math 632: Introduction to Stochastic Processes @ University of Wisconsin-Madison Your comments and criticism are greatly welcomed.

Resources

Past Exams

Textbook: Essentials of stochastic processes by Rick Durrett

Homeworks: HW01 HW02 HW03 HW04 HW05 HW06 HW07 HW08

Lecture Notes (PDF version)

Week 1
9/6 Review, Introduction to Stochastic Processes
Week 2
9/11 Introduction to Markov Chain
9/13 Simple Random Walk, P^n, Gambler's Ruin
Week 3
9/18 T, Strong Markov Property, T_y, ρ_yy, Recurrence
9/20 Recurrence, Closed, Irreducible, Communication
Week 4
9/24 Theorems Related to Recurrence
9/27 Stationary Distribution/Measure, Renewal Chain
Week 5
10/2 Positive/Null Recurrent, Limit Behavior
10/4 Periodicity, Limiting Behavior
Week 6
10/9 Convergence Theorem
10/11 Midterm 1
Week 7
10/16 Doubly Stochastic, Detailed Balance
10/18 Exit Distribution
Week 8
10/23 Exit Time
10/25 Probability Review for Poisson Process
Week 9
10/30 Introduction to Poisson Process
11/1 Compound Poisson Process
Week 10
11/6 Thinning, Superposition, and Conditioning
11/8 Poisson Process Comprehensive Problems
Week 11
11/13 More Exercises on Poisson Process
11/15 Midterm 2
Week 12
11/20 Introduction to Renewal Process
11/22 Thanksgiving
Week 13
11/27 Renewal Process, Age and Residual Life
11/29 Continuous Time Markov Processes
Week 14
12/4 M/M/s Queue, Kolmogorov Equations
12/6 Properties of CTMC
Week 15
12/11 CTMC Exercises

Note: You can download a PDF version of the notes here