2010 IntroToStochasticProcesses
- (Riddles, 2010) ⇒ John Riddles. (2010). “Introduction to Stochastic Processes.” Unpublished Glossary. Department of Statistics, Iowa State University.
Subject Headings: Stochastic Process, Discrete-Time Markov Chain
Notes
Cited By
Quotes
Introduction to Stochastic Processes
Stochastic Process: A Random Variable indexed by time (for an infinite number of time points). Equivalently, it is a Random Vector of infinite dimension.
State Space: X(t) has a support for each value of t. The state space is the union of all of these supports.
Discrete-Time Stochastic Process: A stochastic process indexed by a countable number of time points.
Continuous-Time Stochastic Process: A stochastic process indexed by an uncountable number of [[time points.
Bernoulli Process: A sequence of independent Bernoulli random variables.
Homogenous Poisson Process: Any stochastic process X(t) such that: 1) The state space is f0; 1; 2; 3; :::g, 2) for 0 � t1 < t2, X(t2)X(t1) � Poisson([math]\displaystyle{ \lambda }[/math](t2-t1)), and 3) for any [math]\displaystyle{ 0 \le t_1 \le t_2 \le t_3 \le t_4 }[/math], [math]\displaystyle{ X(t_2)-X(t_1) }[/math] is independent of [math]\displaystyle{ X(t_4)-X(t_3) }[/math].
For a Poisson([math]\displaystyle{ \lambda }[/math]) process, the time between two subsequent successes is Exponential([math]\displaystyle{ \lambda }[/math]). The time until the [math]\displaystyle{ k }[/math]th success is Erlang(k,[math]\displaystyle{ \lambda }[/math]).
Birth and Death Process: A stochastic process is called a Birth and Death Process if 1.) its state space is f0; 1; 2; 3; :::g or f0; 1; 2; 3; :::; ng, and 2.) conditional on X(t1) = [math]\displaystyle{ k }[/math], X(t2) = k + 1 or X(t2) = k 1 for the smallest t2 such that t1 < t2 and X(t1) 6= X(t2).
For a Birth and Death Process, the long term probability of being in state 0 is given by p0 = S1 where S = 1 + �0 �1 + �0�1 �1�2 :::. �i represents the birth rate i ! i + 1 conditional on being in state i. Similarly, �i represents the death rate i ! i 1 conditional on being in state i. If S is not finite, then p0 = 0. That is, the process is unstable and doesn't converge in distribution.
For a Birth and Death Process, the long term probability of being in state i is given by pi = �0�1:::�i1 �1�2:::�i
Discrete-Time Markov Chains
Discrete-Time Markov Chain: A Discrete-Time Markov Chain is a discrete-time stochastic process such that for each tn, X(tn)jX(tn1);X(tn2); :::X(t0) is equal to X(tn)jX(tn1). Initial Distribution p0 is the initial distribution of a Markov chain. p0 = [P(X(t0) = 0); P(X(t0) = 1); :::; P(X(t0) = n)] when the state space of the chain is f0; 1; :::; ng.
Probability Transition Matrix: The (i; j) entry of the probability transition matrix P, represents the probability of moving from state i to state j in one step, conditional on being in state i.
n-Step Probabilities: The n-step probability, pij(n), is the probability of moving from state i to state j in n steps, conditional on being in state i. The matrix of these probabilities is given by Pn where P is the probability transition matrix. Note that multiplication of P with itself is done through matrix multiplication, not element-wise multiplication.
Marginal n-Step Probabilities: The marginal probability of being in state i after n steps is given by the ith entry of p0Pn.
Accessible: A state j is accessible from state i if there is a path from state i to state j. There is a path if pij(n) > 0 for some n > 0.
Recurrent States: State i is recurrent if for every state j that is accessible from state [math]\displaystyle{ i }[/math], state i is also accessible from state j.
Transient States: If a state is not recurrent, it is called transient.
Do not worry about: The negative binomial distribution, brownian motion, steady state probabilities, absorption probabilities, expected time until absorption, or any of the statistics material (other than CLT, since it was covered earlier in the semester).
,