Stochastic Processes : Poisson Process and Markov Chains

1. Suppose that shocks occur according to a Poisson process with rate A> 0. Also suppose that each shock independently causes the system to fail with probability 0 < p < 1. Let N denote the number of shocks that it takes for the system to fail and let T denote the time of the failure.
(a) FindP{T>tNrrn}.
(b) FindP{NrrnT=t}.
(c) Describe how the results from part (b) could have alternatively been determined by considering an appropriate decomposition of the original Poisson process.

2. Let {N(t)}>0 be a nonhomogeneous Poisson process with intensity function A(t) > 0. Then the mean value function 4u(t) is given by ji(t) j A(x) dx. Recall that, for 0 <s <t, N(t)
(a) Find E[N(s)N(t)] for 0 < s < t.
(b) Find the covariance of N(s) and N(t) for 0 <s <t.
(c) Find the correlation between N(s) and N(t) for 0 <s <t.

5. Consider two urns, each of which contains ri-i balls. Initially, the first urn contains w white balls and ri-i ? w black balls, where 0 <w < in, and the second urn contains in black balls. At each step, one ball is chosen from each of the urns and the two chosen balls are switched. Define X7, to be the number of white balls in the first urn at time n. Then {X0,X1,X2,.. .} is a Markov chain.
(a) Find the transition probabilities of the Markov chain.
(b) Find the stationary distribution of the Markov chain.
(c) Describe how the results from part (b) could have been determined by considering how the balls would be distributed between the two urns after the Markov chain has reached stationarity.

Hello. These are practice problems for an upcoming exam. I would like solutions to compare against my own work.

In problems 3 and 4, F is presumably any cdf (no particular distribution was specified).

PS If you are not well versed in stochastic processes then please do not sign this problem out.

PPS This is remuneration for problems 1, 2, and 5