Probability Models
Homework 8
Assignment (4 Problems: 25 + 25 +25 + 25 = 100 points total.)
Q Problem 1 Let X(t) be the birth-death process defined as a continuous time Markov chain in Example 1 in Lecture 21 with state space X = a, b and 1-step transition rates λ = qab = 5 and µ = qba = 1.
• 1.1 [10 points] Given that X(0) = a, what is the probability that the chain stays in the state a for all times t in the time interval [0, 0.3]?
• 1.2 [15 points] Given that X(0) = a, what is the probability that the chain is in the state b at time t = 0.8 and that also the chain has only jumped from a to bonce in the time interval [0, 0.8]?
Problem 2 Let X(t) be the continuous time Markov chain defined in Example 2 in Lecture 21 with state space X = {a, b, c} and infinitesimal generator
Q−6 3 3
Q = 4 −12 8
15 3 −18
which encodes the 1-step transition rates qij and decay rates vi by qii = −vi = − j/=i qij.
• 2.1 [10 points] Determine the 3 × 3 matrix P with entries
Pij =qij k/=i qikif i j0 if i = j.
• 2.2 [15 points] Given that X(0) = b, what is the probability that this continuous time Markov chain visits the state a before visiting c? Equivalently, what is the probability that this process started at b makes its first jump from b to a (and not from b to c)?
Problem 3 Let X(t) be the birth-death process from Example 1 in Lecture 22 with state space X = {0, 1} and infinitesimal generator
Q = −λ λ
Q µ −µ
where λ, µ > 0 are two positive constants. This model generalizes the chain in Problem 1. In Lecture 22, we used Kolmogorov’s backward equations to derive the formula
P00(t) = P (X(t) = 0|X(0) = 0) = µ + λ e−(λ+µ)t λ + µ λ + µ
for the transition probabilities P00(t) at any time t 0. Kolmogorov’s insight was that in order to solve for the unknown P00(t), one has to temporarily enlarge the number of unknown transition probabilities one cares about – specifically, to P00(t) and P10(t) – in order to solve for both unknowns at once and hence solve for P00(t).
• 3.1 [5 points] For any λ, µ, what is the transition probability P00(0) at t = 0?
• 3.2 [10 points] If µ = 0 but λ > 0, what is the transition probability P00(t)?
• 3.3 [10 points] If µ = λ > 0, what is the limiting transition probability limt→∞ P00(t)?
Problem 4 In the context of Problem 3, in Lecture 22 we derived a formula for P00(t) by studying two unknown transition probabilities P00(t) and P10(t). For birth-death processes with more than two states, we also learned that one has to consider more than two unknown transition probabilities. Consider the M/M/1 queue, the birth-death process with λ0 = λ1 = λ2 = • • • = λ µ1 = µ2 = µ3 = • • • = µ
for birth and death rates λ, µ > 0.
• 4.1 [15 points] State the system Kolmogorov backward equations for the transition probabilities of the M/M/1 queue. You do not have to solve this system.
• 4.2 [10 points] Simplify your answer in the case µ = 0.
Bonus [X points] For any birth-death process with birth and death rates λn, µn > 0 at the state n in X = N, consider the quantity πn defined by the formula π = λ0λ1 • • • λn−1 .
Evaluate πn in the case of the birth-death process with rates λn = Ln µn = Mn for some M, L > 0 satisfying L < M .
CS 340 Milestone One Guidelines and Rubric Overview: For this assignment, you will implement the fundamental operations of create, read, update,
Retail Transaction Programming Project Project Requirements: Develop a program to emulate a purchase transaction at a retail store. This
7COM1028 Secure Systems Programming Referral Coursework: Secure
Create a GUI program that:Accepts the following from a user:Item NameItem QuantityItem PriceAllows the user to create a file to store the sales receip
CS 340 Final Project Guidelines and Rubric Overview The final project will encompass developing a web service using a software stack and impleme
