logo Hurry, Grab up to 30% discount on the entire course
Order Now logo

Ask This Question To Be Solved By Our ExpertsGet A+ Grade Solution Guaranteed

expert
Steven HughesEducation
(5/5)

908 Answers

Hire Me
expert
Jordan CarterEngineering
(5/5)

922 Answers

Hire Me
expert
Rakesh RastogiManagement
(5/5)

865 Answers

Hire Me
expert
Leroy BicknellBusiness
(5/5)

519 Answers

Hire Me
R Programming
(5/5)

Why should the data be partitioned into training and validation sets? What will the training set be used for? What will the validation set be used for?

INSTRUCTIONS TO CANDIDATES
ANSWER ALL QUESTIONS

1. Predicting Boston Housing Prices. The file BostonHousing.csv contains information collected by  the US Bureau of the Census concerning housing in the area of Boston, Massachusetts. The dataset includes information on 506 census housing tracts in the Boston area. The goal is to predict the median house price in new tracts based on information such as crime rate, pollution, and number of rooms. The dataset contains 13 predictors, and the response is the median house price (MEDV). The table below describes each of the predictors and the response.

 

Variable Description

 

CRIM Crime Rate

ZN Percentage of residential land zoned for lots over 25,000 sq. ft. INDUS Percentage of land occupied by non-retail business

CHAS Does tract bound Charles River (1 if yes and 0 if no) NOX Nitric oxide concentration (parts per 10 million)

RM Average number of rooms per dwelling

AGE Percentage of owner-occupied homes built prior to 1940 DIS Weighted distance to five Boston employment centers RAD Index of accessibility to radial highways

TAX Full-value property tax rate per $10,000 PTRATIO Pupil-to-teacher ratio by town

LSTAT Percentage of lower status of the population MEDV Median Value of homes in $1000s CAT.MEDV MEDV grouped into two categories

 

a. Why should the data be partitioned into training and validation sets? What will the training set be used for? What will the validation set be used for?

b. Fit a multiple linear regression model to the median house price (MEDV) as a function of CRIM, CHAS, and RM. Write the equation for predicting the median house price from the predictors in the model.

c. Using the estimated regression model, what median house price is predicted for a tract in the Boston area that does not bound the Charles River, has a crime rate of 0.1, and where the average number of rooms per house is 6?

d. Reduce the number of predictors:

i. Which predictors are likely to be measuring the same thing among the 13 predictors? Discuss the relationships among INDUS, NOX, and TAX.

ii. Produce a correlation heatmap for the 12 numerical predictors and search for highly correlated pairs. These have potential redundancy and can cause multicollinearity. Choose which ones to remove based on these results.

iii. Divide the housing set into a training and validation set (dedicate 60% to the training and 40% to the validation set). In order to reduce the remaining predictors, use the regsubsets() function of the leaps package as follows: Run regsubsets() with exhaustive search on the training set. Using adjusted R2s as a metric, choose the top three models from the exhaustive search. Then use each of these models separately to predict the validation set. Compare RMSE and MAPE derived from the validation set. Finally, describe the best model.

 

iv. Using the step() function of the base package run a bidirectional search on the training set. Compare the best model produced by the step() function with the best model produced by the procedure in the previous question. Is there a need to use the validation set to confirm which model is the best one?

(5/5)
Attachments:

Related Questions

. The fundamental operations of create, read, update, and delete (CRUD) in either Python or Java

CS 340 Milestone One Guidelines and Rubric  Overview: For this assignment, you will implement the fundamental operations of create, read, update,

. Develop a program to emulate a purchase transaction at a retail store. This  program will have two classes, a LineItem class and a Transaction class

Retail Transaction Programming Project  Project Requirements:  Develop a program to emulate a purchase transaction at a retail store. This

. The following program contains five errors. Identify the errors and fix them

7COM1028   Secure Systems Programming   Referral Coursework: Secure

. Accepts the following from a user: Item Name Item Quantity Item Price Allows the user to create a file to store the sales receipt contents

Create a GUI program that:Accepts the following from a user:Item NameItem QuantityItem PriceAllows the user to create a file to store the sales receip

. The final project will encompass developing a web service using a software stack and implementing an industry-standard interface. Regardless of whether you choose to pursue application development goals as a pure developer or as a software engineer

CS 340 Final Project Guidelines and Rubric  Overview The final project will encompass developing a web service using a software stack and impleme