Introduction
For this project, you will perform a data linkage on two real-world datasets (Part 1) and explore different classification algorithms (Part 2).
The project is to be coded in Python 3. Eight (8) relevant data sets can be downloaded from LMS and are available on Juptyer Hub:
ˆ datasets for Part 1: Data Linkage
csv
csv
amazon google csv
amazon csv
google csv
amazon google truth csv
ˆ datasets for Part 2: Classification:
csv
csv
Please note that your solution program should behave the same way as intended if a similar but different input dataset in an identical format is used. Hard-coding any part of solution will attract a heavy mark-deduction.
Part 1 - Data Linkage (12 marks)
Amazon and Google both have product databases. They would like to link the same products in order to perform joint market research.
Marking guide for Task 1-A:
Na¨ıve data linkage without blocking (4 marks)
mark Not reproducible - code does not pass the test script given: this could be due to the source file is not named py and the output file task1a.csv is not produced.
The output is hardcoded or the output is completely incorrect.
mark The output is in the format as At least one of Precision or Recall are below 0.6.
marks The output is in the format as specified. Both Precision and Recall are above 6.
marks The output is in the format as specified. Both Precision and Recall are above 8.
marks The output is in the format as specified. Both Precision and Recall are above 9.
Marking guide for Task 1-B
Blocking for efficient data linkage (4 marks)
mark Not reproducible - code does not pass the test script given: this could be due to the source file is not named py and the output files amazon blocks.csv and google blocks.csv are not produced; or
The output is hardcoded or the output is completely incorrect, or The blocking method is not linear in time complexity.
mark The output is in the format as specified. PC is below 0.4 and/or RR is below 6.
marks The output is in the format as specified. PC is above 0.4 and RR is above 0.6.
marks The output is in the format as specified. PC is above 0.6 and RR is above 0.8.
marks The output is in the format as specified. PC is above 0.8 and RR is above 0.9.
Marking guide for Task 1-C:
Report on the Data Linkage Project (4 marks)
Scoring method: How the product comparison works
1 mark the comparison functions, final scoring function and threshold are appropriate and are clearly and succinctly described, including any preprocessing applied.
0.5 mark Description of the product comparison is understandable but somewhat unclear. It contains insufficient or excessive unnecessary or incorrect information. Missing some key information on the methods applied.
mark The description misses key information about the product com- parison method and/or is
Evaluation of the overall performance of the product comparison
mark Evaluation of the performance is clear and succinct; the potential improvements are sensible and not overly
0.5 marks Evaluation of the performance lacks sufficient details or contains excessive information to give clarity and/or the potential improvements are not convincing or are overly simplistic.
mark Evaluation of the performance is substantially incorrect and/or the potential improvements are incorrect or
Blocking method: How the blocking method works
mark The blocking method is appropriate and is clearly and succinctly described; includes any preprocessing
0.5 mark Description of the blocking method is understandable but some- what unclear. It contains insufficient or excessive unnecessary information or some incorrect information.
mark The description misses key information about the blocking method and/or is
Evaluation of the blocking method
mark Evaluation of the blocking performance is clear and succinct and the overall performance relates well to the measures
The potential improvements are sensible and convincing.
The time-complexity of the blocking method is correct and clearly indicated or can be clearly inferred.
0.5 mark Evaluation of the blocking performance is somewhat unclear. It lacks sufficient details or contains excessive information to give clarity; or
Overall performance is not well related to the measures used; or The potential improvements are not convincing; or
The time-complexity of the blocking method and its correctness are unclear.
0 mark The evaluation of the blocking performance is substantially incor- rect, or the potential improvements are incorrect or missing.
Marking guide for Task 2-A:
Comparing Classification Algorithms (3 marks)
Please note that your solution program should behave the same way as intended if a similar but different input dataset in an identical format is used. Hard-coding any part of the solution will attract a heavy mark-deduction.
Failure to pass the test script will result in 0 mark for the task.
mark Not reproducible - code does not pass the test script given: this could be due to the source file is not named py and the output file task2a.csv is not produced.
The classification accuracy of the three classifiers are not printed to stan- dard output.
The output is hardcoded or the output is completely incorrect.
mark The output file csv is correct but the classification accuracies are substantially incorrect.
marks The classification accuracies for some of the classifiers are slightly
marks The output file csv is produced and has correct number of rows/columns and has correct values.
The classification accuracies are correct for all three classifiers.
Marking guide for Task 2-B:
Feature Engineering and Selection(6 marks)
Please note that your solution program should behave the same way as intended if a similar but different input dataset in an identical format is used. Hard-coding any part of the solution will attract a heavy mark-deduction.
Failure to pass the test script will result in 0 mark for the task.
Overall
mark Not reproducible - code does not pass the test script given: this could be due to the source file not being named py.
Feature engineering – interaction terms
mark Evidence of correct implementation of interaction term pairs. Output that support the correct implementation produced and the outputs are
0.5 mark Evidence of partially correct implementation of interaction term pairs. Output that support the implementation produced and the outputs are partially correct.
0 mark No output is produced as supporting evidence of implementation of this sub- task (even if the sub-task is implemented in task2b.py)
Feature engineering – clustering
2 marks Evidence of correct implementation of feature engineered from clustering. Outputs that support the correct implementation produced and the outputs are correct.
Evidence of implementing a good choice of k for k-means clustering. Outputs that support a good selection of k produced.
1 mark Implementation of feature engineering from clustering and choice of k for k- means clustering are largely correct but contain some minor issues.
Appropriate outputs as evidence of the implementation are produced but choice of k is hard-coded or inappropriate.
mark No output is produced as supporting evidence of implementation of this sub- task (even if the sub-task is implemented in py)
CS 340 Milestone One Guidelines and Rubric Overview: For this assignment, you will implement the fundamental operations of create, read, update,
Retail Transaction Programming Project Project Requirements: Develop a program to emulate a purchase transaction at a retail store. This
7COM1028 Secure Systems Programming Referral Coursework: Secure
Create a GUI program that:Accepts the following from a user:Item NameItem QuantityItem PriceAllows the user to create a file to store the sales receip
CS 340 Final Project Guidelines and Rubric Overview The final project will encompass developing a web service using a software stack and impleme
