Data science is a collaborative activity. While you may talk with others about the homework, we ask that you write your solutions individually. If you do discuss the assignments with others please include their names at the top of your submission.
Normal equations
1. (12 points) In the lecture, we discussed a geometric argument for the normal equations we solve to get the least squares estimator θˆ:
X⊤(Y − Xθ) = 0.
Here, we are using X to denote the design matrix:
where 1n is the n-vector of all 1s and Xj is the n-vector [x1,j, . . . , xn,j]⊤ holding all observations of the jth variable. We use Y to denote the response vector as an n 1 matrix (note that there would be nothing wrong with writing y Xθ, we would just interpret the vector y as a length-n column vector, which is the same as an n 1 matrix).
The normal equations are the estimating equations for OLS, i.e. (multiple) linear regres- sion under the least squares loss. As we discussed in class, while there can be multiple values of θ ∈ Rp+1 that satisfy these equations, there is always at least one solution.
To build intuition for these equations and relate them to the SLR estimating equations, we will derive them mathematically in several ways.
(a) (2 points) Show that finding the optimal estimator θˆ by solving the normal equa- tions is equivalent to requiring that the residual vector e = Y − Xθˆ should average to zero, and should be orthogonal to Xj (i.e., should have zero dot product with Xj) for every j, that is, show that
Use calculus to show that any θ = [θ0, θ1, ..., θp]⊤ that minimizes the MSE must solve the normal equations.
Hint: remember that, at a minimum of MSE, the partial derivatives of MSE with respect to every θi must all be zero.
CS 340 Milestone One Guidelines and Rubric Overview: For this assignment, you will implement the fundamental operations of create, read, update,
Retail Transaction Programming Project Project Requirements: Develop a program to emulate a purchase transaction at a retail store. This
7COM1028 Secure Systems Programming Referral Coursework: Secure
Create a GUI program that:Accepts the following from a user:Item NameItem QuantityItem PriceAllows the user to create a file to store the sales receip
CS 340 Final Project Guidelines and Rubric Overview The final project will encompass developing a web service using a software stack and impleme
