Description
You have been supplied with the datafile all_weather_data.csv which contains data from multiple weather stations. This data has been collected from Mesonet.
The locations that have been collected can be seen below.
The dataset needs to be analyzed using PCA, but it contains missing values and anomalies that must be removed before the analysis can be performed. It is your task to clean the dataset into a form that can be fed to PCA for analysis. After that, you will be doing a PCA analysis on the dataset.
Part 1: Data processing
Import the csv data into a table in a database on your machine. You can use SQL, workbench or python to achieve this. Show the process you did to achieve
Plot a timeseries plot of selected variables for station FGBT and FMMI. For each station plot heat_index_set_1d together with air_temp_set_1 in one plot and wind_speed_set_1 together with wind_gust_set_1 on another. All in all, provide 4 plot images (2 for each station). Below is an example for the station
Create a query that gives the name of all stations that have at least one row of values for heat_index_set_1d
Find columns that have more that have at least 90% values (not having more than 5% NULL values). The columns that have less then 10% missing values are to be used for analysis. You can choose to use SQL or Python. Show how you proceed and list the columns that have too many missing
Create a view that shows only the columns that have more than 90% non-null values. This view should also remove all rows that have any null values. Provide the SQL code to create this
The data we have in its current for is not normalized and therefore contains a lot of redundant data. Create a normalized model. Provide an image of the model and an explanation for which variables broke what normalization
Part 2: PCA analysis
Once you have your cleaned data file, carry out a PCA on the data matrix. All the weather measurements will be your observations, all the weather properties (temperature, dew point etc.) will be your variables. The station IDs will be your labels.
Carry out the PCA using an existing PCA algorithm in Python, or write your own. With the hoggorm algorithm you can easily plot the scores and loadings https://hoggorm.readthedocs.io/en/latest/
With the scikit-learn you can plot the observations of the different station IDs with different colours.
https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.PCA.html
You can see an example of how to apply the PCA algorithms in the python script in the lecture notes.
Make a figure showing a score plot and a loadings plot with PC1 and PC2.
Based on these plots, discuss shortly (2-5 lines) about the weather at different geographical locations.
CS 340 Milestone One Guidelines and Rubric Overview: For this assignment, you will implement the fundamental operations of create, read, update,
Retail Transaction Programming Project Project Requirements: Develop a program to emulate a purchase transaction at a retail store. This
7COM1028 Secure Systems Programming Referral Coursework: Secure
Create a GUI program that:Accepts the following from a user:Item NameItem QuantityItem PriceAllows the user to create a file to store the sales receip
CS 340 Final Project Guidelines and Rubric Overview The final project will encompass developing a web service using a software stack and impleme
