Homework #6
Goal: EM Algorithm
For this homework, submit your R code for this assignment electronically via Canvas. Note the deadline on Canvas. In addition to the R code, you must also turn in your typed-in answers. Please submit your R code and the typed-up answers as separate documents. Points will be deducted if R code is very poorly formatted.
1. For this problem, you will be deriving and implementing the EM algorithm for a Gaussian Mixture Model. Each part will guide you through a step in implementing the EM algorithm. A Gaussian Mixture Model is where the data Y comes from one of several normal distributions with probabilities πk, i.e.
Yi ∼ N (µk, σ2) (1)
for groups k in 1, . . . , K. We can write the probability that Yi = y as
P (Yi = y) = Σ πkP (Yi = y|Zi = k) = Σ πkN (Yi; µk, σ2) (2)
k=1 k=1
where Zi denotes what group that Yi comes from. We are interested in estimating θ = (µk, σ2, πk).
This problem is well-known and there are extensive online notes de- tailing solutions. You may use these online resources, but you must understand what happens at each step. That is why homework will be graded primarily on the amount of work you show and your expla- nations. Show ALL steps in your derivation. This homework is less about the coding and more about the theory, since the hard part of EM algorithm is the theoretical component.
(a) Write down the joint likelihood of Y1, . . . , Yn. This is the observed likelihood, since we are missing our latent variables Zi. Explain why this joint likelihood is difficult to maximize directly with respect to θ.
(b) Now assume we know Zi. Write down the complete-data likeli- hood and the corresponding complete-data log-likelihood.
(c) Now take the conditional expectation to find Q(θ|θ(t)). There will be an expectation that depends on µ(t), σ2(t) and π(t), which
you can denote γj(xi) for i ∈ 1, . . . , n. The update step for that
term is given
(d) Maximize this conditional expectation with respect to µk, σ2, and
πk. Your update steps should be the following:
Σn γj(xi)xi
(e) You now have update steps for all parameters in the model. Clearly write each update step. Make sure you include the update step derived in step (c).
(f) Fill in the code provided in HW 6 Guide.R for K = 2. Note that for K = 2, you can keep track of only π and (1 − π), and similarly, just one vector of γ(xn) since γ1(xi) = 1 − γ1(xi).
(g) Run your code on the data set for K = 2. hw6 prob1 data.txt found on Canvas. Clearly provide estimates for θ (and only θ, not the γ values).
CS 340 Milestone One Guidelines and Rubric Overview: For this assignment, you will implement the fundamental operations of create, read, update,
Retail Transaction Programming Project Project Requirements: Develop a program to emulate a purchase transaction at a retail store. This
7COM1028 Secure Systems Programming Referral Coursework: Secure
Create a GUI program that:Accepts the following from a user:Item NameItem QuantityItem PriceAllows the user to create a file to store the sales receip
CS 340 Final Project Guidelines and Rubric Overview The final project will encompass developing a web service using a software stack and impleme
