Probability Models
Instructions
• Write your full name, “Midterm 2”, and the date at the top of the first page.
• Show all work, including each step of your solution, to earn maximal partial credit.
• Each question has multiple parts. Write legibly and neatly. Box your final answers.
• Use Genius Scan or a similar application to convert your solutions to .pdf format.
• Submit a single .pdf file to Gradescope under the assignment “Midterm 2”.
Assignment (2 Problems: 50 + 50 = 100 points total.)
Q Problem 1 Consider the Markov chain {Xn}∞n=0 with infinite state space X = {0, 1, 2, 3, 4, . . .}and 1-step transition probabilities if j = i Pij = 0.1 if j = i + 1
• 1.1 [10 points] Assuming X0 = 0, calculate the time n marginal distribution of this Markov chain. Precisely, for each i in X, calculate
P (Xn = i | X0 = 0).
In this problem, both n and i are arbitrary.
• 1.2 [10 points] Assuming X0 = 0, let T be the random time this Markov chain spends in the state 0 before jumping to a different state. Find the probability mass function of the discrete random variable T . Precisely, for each n = 1, 2, 3, . . ., calculate
P (T = n).
In this problem, n is arbitrary.
• 1.3 [10 points] Find the communicating classes of the Markov chain.
• 1.4 [10 points] Determine if each communicating class is recurrent or transient.
• 1.5 [10 points] Is this Markov chain reversible or irreversible? Explain your reasoning.
Hint: use Kolmogorov’s criterion
Q Problem 2 Consider the simple graph Γ = (V, E) with 4 vertices and 3 edges above and let {Xn}∞n=0 be the simple random walk on Γ as defined in HW5.
• 2.1 [10 points] Find the 1-step transition matrix Q for the simple random walk on Γ.
• 2.2 [10 points] Suppose X0 = a. Let T be the first time at which the Markov chain returns to the state a. This first return time T is a discrete random variable
T = min{n ≥ 1 : Xn = a}.
What is the probability that the chain visits the state d before time T ?
• 2.3 [10 points] Find an equilibrium distribution for the Markov chain defined by Q.
• 2.4 [10 points] Is the uniform distribution µ with
µ→ = [µa µb µc µd] = [ 11 1 14 4 4 an equilibrium distribution of the Markov chain defined by Q?
• 2.5 [10 points] Find the 1-step transition matrix P for the Hastings-Metropolis Markov chain which (i) has equilibrium distribution µ given by the uniform distribution above and (ii) has as its proposal chain the Markov chain defined by Q above.
Q Bonus A Markov chain in R is a discrete time Markov chain Xn ∞n=0 in the continuum state space X = R. For most Markov chains in R, the 1-step transition probabilities Pxy
are all 0, so a new quantity is needed to describe these Markov chains. For any two states x, y in R, the 1-step transition density is a function K(x, y) of (x, y) in R2 so thatˆ b
P (Xn+1 in [a, b] | Xn = x) =K(x, y)dy.This K(x, y) is also called the Markov kernel or stochastic kernel of the Markov chain.
• X.1 [X points] Consider the chain {Xn}∞n=0 in X = R with 1-step transition densities K( ) = 1if |y − x| ≤ 0.5 otherwise Find P (X2 > 0.5 | X0 = 0).
• X.2 [X points] Consider the chain {Xn}∞n=0 in X = R with 1-step transition densities 1 − (y−x)2 K(x, y) = √2πe 2 Find P (X2 > 0.5 | X0 = 0).
CS 340 Milestone One Guidelines and Rubric Overview: For this assignment, you will implement the fundamental operations of create, read, update,
Retail Transaction Programming Project Project Requirements: Develop a program to emulate a purchase transaction at a retail store. This
7COM1028 Secure Systems Programming Referral Coursework: Secure
Create a GUI program that:Accepts the following from a user:Item NameItem QuantityItem PriceAllows the user to create a file to store the sales receip
CS 340 Final Project Guidelines and Rubric Overview The final project will encompass developing a web service using a software stack and impleme
