• Write your full name, “Homework 4”, and the date at the top of the first page.
• Show all work, including each step of your solution, to earn maximal partial credit.
• Each question has multiple parts. Write legibly and neatly. Box your final answers.
• Use Genius Scan or a similar application to convert your solutions to .pdf format.
• Submit a single .pdf file to Gradescope under the assignment “Homework 4”.
• You are encouraged to work together (on Piazza) but must write up your own solutions.
Assignment (5 Problems: 20 + 20 + 20 + 20 + 20 = 100 points total.)
Q Problem 1 The Markov chain X = {Xn}∞n=0 has state space X = {a, b} and 0.9 0.10.02 0.98
• 1.1 [5 points] Compute P (X2 = a|X0 = a).
• 1.2 [5 points] Compute P (X3 = b|X0 = a).
• 1.3 [5 points] Is this Markov chain irreducible?
• 1.4 [5 points] Is the state b recurrent?
Q Problem 2 The Markov chain X = {Xn}∞n=0 has state space X = {a, b, c} and
P = 0.5 0 0.5
0.5 0.5 0
• 2.1 [5 points] Compute P (X2 = a|X0 = a).
• 2.2 [5 points] Compute P (X3 = c|X0 = b).
• 2.3 [5 points] Is this Markov chain irreducible?
• 2.4 [5 points] Is the state b recurrent?
Q Problem 3 The Markov chain X = {Xn}∞n=0 has state space X = {a, b, c} and
0.8 0.1 0.1
P = 0.6 0.2 0.2
0.4 0.3 0.3
• 3.1 [5 points] Compute P (X2 = a|X0 = a).
• 3.2 [5 points] Compute P (X3 = c|X0 = b).
• 3.3 [5 points] Is this Markov chain irreducible?
• 3.4 [5 points] Is the state b recurrent?
Q Problem 4 The Markov chain X = {Xn}∞n=0 has state space X = {a, b, c, d, e} and
0.5 0 0.5 0 0
0.25 0.5 0.25 0 0
P = 0.5 0 0.5 0 0
0 0 0 0.5 0.5
0 0 0 0.5 0.5
• 4.1 [10 points] Find the communicating classes of the Markov chain.
• 4.2 [10 points] Determine if each communicating class is recurrent or transient.
Q Problem 5 The Markov chain X = {Xn}∞n=0 has state space X = {a, b, c, d, e} and
1 0 0 0 0
0.5 0 0.5 0 0
P = 0 0.5 0 0.5 0
• 5.1 [10 points] Find the communicating classes of the Markov chain.
• 5.2 [10 points] Determine if each communicating class is recurrent or transient.
Q Bonus [X points] If the Markov Chain in Problem 5 has an initial state X0 = c, what is the probability that X5 is in the set {b, c, d}?
Note: looking back at Lecture 8 and at Example 6 in §4.1 of [Ross], if one identifies a, b, c, d, e = 0, 1, 2, 3, 4 , this Markov chain is a probability model for the experiment of gambling $1 at each time step n = 0, 1, 2, . . . – with independent 50% chances of winning each time – starting with $2 and hoping to reach $4 overall before ending up bankrupt at $0. This Bonus problem is therefore asking you: what are the chances that the gambler has not reached the target amount of $4 nor gone bankrupt by the 5th gamble?
CS 340 Milestone One Guidelines and Rubric Overview: For this assignment, you will implement the fundamental operations of create, read, update,
Retail Transaction Programming Project Project Requirements: Develop a program to emulate a purchase transaction at a retail store. This
7COM1028 Secure Systems Programming Referral Coursework: Secure
Create a GUI program that:Accepts the following from a user:Item NameItem QuantityItem PriceAllows the user to create a file to store the sales receip
CS 340 Final Project Guidelines and Rubric Overview The final project will encompass developing a web service using a software stack and impleme
