logo Use SA10RAM to get 10%* Discount.
Order Now logo

Ask This Question To Be Solved By Our ExpertsGet A+ Grade Solution Guaranteed

expert
Roshan NehraPhilosophy
(5/5)

876 Answers

Hire Me
expert
Antenaina SandyMarketing
(5/5)

530 Answers

Hire Me
expert
Jaida PatrickTechnical writing
(5/5)

637 Answers

Hire Me
expert
Craig AndersonEngineering
(5/5)

834 Answers

Hire Me
Java Programming

Create a MapReduce program called MovieRatings that will print all the average ratings for a movie based on user feedback

INSTRUCTIONS TO CANDIDATES
ANSWER ALL QUESTIONS

Assignment 5

The goal of this assignment is to learn to write a more complicated MapReduce program for processing big data sets in parallel using MapReduce framework.

  • Create a MapReduce program called MovieRatings that will print all the average ratings for a movie based on user feedback

  • The program should work on MovieLens  data  set, which can be downloaded from http://www.grouplens.org/node/73.       Download the 100k data (ml-100zip) set into your Downloads folder. Extract the ml-100k.zip file into  ~/Downloads  folder  itself.  For this particular assignment, the program will use the u.data and u.item files under the extracted folder. Technically speaking, you really do not have to do this step if you had already done assignment 3. Because this was data preparatory step for assignment 3.

  • On running the MovieRatings MapReduce program, it should process the data file, which contains  a list of movie ratings by users and also process u.item file, which has movie related information. In the u.data file, each user has rated at least 20 movies.   Users and items are numbered consecutively  from 1.  The u.data  file is randomly ordered and the fields are tab separated with list of fields being user id, item id, rating and timestamp. The u.item field contains sveral pieces of information  about each movie. The u.item file is pipe(|) separated. Unfortunately, the text of the README file is incorrect in terms of field separator for u.item, but it has the correct information about  the  fields  and their  order.  Please look at the actual file and also the README file.

  • The program should take 2 The first argument should be the directory where the movielens user data set

(u.data) and movie information (u.item) files are placed (called /movie-and-ratings in hdfs), and the second argument is the name of directory where the results will be placed in HDFS (called /movie-rating-result).

  • The result of the program should be a list of all movies rated with the following information per line -- movie id, movie title, release date, IMDB URL, average rating, total number of unique users rated, total number of ratings for the movie

  • Use 2 reducers by setting the number of reducers to 2.

  • Write a combiner for efficiency, if possible

  • Define a custom counter that has a running count of total records processed in map, total number of unique movies in

  • Before you begin, please split the input file into 5 files called udata, u2.data, u3.data, u4.data and u5.data, each having 20000 records. Use the Unix split command. Reference: https://kb.iu.edu/d/afar

  • All the input files - 5 user ratings file and the movie information file should be gzip compressed before processing.

  • Reducer output files should be

What to turn in?

  • Jar file that can be run at command line

  • A zip of NetBeans or Eclipse project

  • A sample run with output as a Word I have given you a template for assignment 4 solution. Follow the guidelines, fill the document and submit.

Related Questions

. The fundamental operations of create, read, update, and delete (CRUD) in either Python or Java

CS 340 Milestone One Guidelines and Rubric  Overview: For this assignment, you will implement the fundamental operations of create, read, update,

. Develop a program to emulate a purchase transaction at a retail store. This  program will have two classes, a LineItem class and a Transaction class

Retail Transaction Programming Project  Project Requirements:  Develop a program to emulate a purchase transaction at a retail store. This

. The following program contains five errors. Identify the errors and fix them

7COM1028   Secure Systems Programming   Referral Coursework: Secure

. Accepts the following from a user: Item Name Item Quantity Item Price Allows the user to create a file to store the sales receipt contents

Create a GUI program that:Accepts the following from a user:Item NameItem QuantityItem PriceAllows the user to create a file to store the sales receip

. The final project will encompass developing a web service using a software stack and implementing an industry-standard interface. Regardless of whether you choose to pursue application development goals as a pure developer or as a software engineer

CS 340 Final Project Guidelines and Rubric  Overview The final project will encompass developing a web service using a software stack and impleme

Get Free Quote!

403 Experts Online