Notes on Math 632: Introduction to Stochastic Processes @ University of Wisconsin-Madison
Your comments and criticism are greatly welcomed.
Resources
Past Exams
Textbook: Essentials of stochastic processes by Rick Durrett
Homeworks: HW01 HW02 HW03 HW04 HW05 HW06 HW07 HW08
Lecture Notes (PDF version)
| Week 1 |
| 9/6 |
Review, Introduction to Stochastic Processes |
| Week 2 |
| 9/11 |
Introduction to Markov Chain |
| 9/13 |
Simple Random Walk, P^n, Gambler's Ruin |
| Week 3 |
| 9/18 |
T, Strong Markov Property, T_y, ρ_yy, Recurrence |
| 9/20 |
Recurrence, Closed, Irreducible, Communication |
| Week 4 |
| 9/24 |
Theorems Related to Recurrence |
| 9/27 |
Stationary Distribution/Measure, Renewal Chain |
| Week 5 |
| 10/2 |
Positive/Null Recurrent, Limit Behavior |
| 10/4 |
Periodicity, Limiting Behavior |
| Week 6 |
| 10/9 |
Convergence Theorem |
| 10/11 |
Midterm 1 |
| Week 7 |
| 10/16 |
Doubly Stochastic, Detailed Balance |
| 10/18 |
Exit Distribution |
| Week 8 |
| 10/23 |
Exit Time |
| 10/25 |
Probability Review for Poisson Process |
| Week 9 |
| 10/30 |
Introduction to Poisson Process |
| 11/1 |
Compound Poisson Process |
| Week 10 |
| 11/6 |
Thinning, Superposition, and Conditioning |
| 11/8 |
Poisson Process Comprehensive Problems |
| Week 11 |
| 11/13 |
More Exercises on Poisson Process |
| 11/15 |
Midterm 2 |
| Week 12 |
| 11/20 |
Introduction to Renewal Process |
| 11/22 |
Thanksgiving |
| Week 13 |
| 11/27 |
Renewal Process, Age and Residual Life |
| 11/29 |
Continuous Time Markov Processes |
| Week 14 |
| 12/4 |
M/M/s Queue, Kolmogorov Equations |
| 12/6 |
Properties of CTMC |
| Week 15 |
| 12/11 |
CTMC Exercises |
Note: You can download a PDF version of the notes here
