Applications of Big Data & Feature Extraction - IT Assignment Help

Assignment Task -                 
 

1 Description 

• For this assignment, you will need to create a complete program to perform sentiment classic
• cation for movie reviews from feature extraction to classification.
• For a given review, your program should be able to predict whether it is positive i.e. like the movie, or negative, i.e. dislike the movie. 

2 About the data 

You will use a large movie review dataset containing a set of 25,000 movie reviews for training, and 25,000 for testing. You can download the data from view under the assignments folder named aclImdb.zip. You can also visit the following website for more information about the dataset: or download data directly from there. Unzip the data to your local directory. In the aclImdb/ directory created by the zip le (∼500MB), you will and the following three items (among others): 

1. train/ — feature ?les and raw text ?les for the training set 

2. test/ — feature ?les and raw text ?les for the testing set 

3. README — the readme ?le for more information on the dataset 

Read the README ?le carefully for descriptions of the text ?les that contain the reviews and their naming convention. ?e directories with which we are concerned here are: 

1. ./aclImdb/train/pos — raw text ?les of positive reviews in the training set 2. ./aclImdb/train/neg — raw text ?les of negative reviews in the training set 3. ./aclImdb/test/pos — raw text ?les of positive reviews in the test set.

3 Task 1. Feature extraction 

Use the MapReduce model to convert all text data into matrices. Convert ratings to vectors. ?ese will be used for classi?cation in Task 2. Use TF-IDF to vectorize the text ?les. See previous practical classes and lecture materials for TF-IDF. One step further though is to represent each text ?le (review) as a very long and sparse vector as the following. Assume words list is the ?nal list of distinct words contained in all reviews and its length is N. ?en each review will be a vector of length N, with each position associated with the word in wordlist and the value being either 0, if the corresponding word is absent in the review or the word’s TF-IDF. For example, if wordlist = [‘word1’, ‘word2’, ‘word3’, ‘word4’] and review 1 contains word1 and word4, then the vector representation of review 1 is [0.1, 0, 0, 0.4] assuming TF-IDF of word 1 and word 4 in review 1 is 0.1 and 0.4 respectively. Note that TF is calculated from one single document while IDF is obtained from all documents in the collection. 

4 Task 2. Classic cation

Construct a classification model for review sentiment prediction, meaning that: given a customer movie review (taken from the test set), your program should be able to predict whether it is positive or negative.ere is no limitation on how many classi?ers and what speci?c model you should use. You can simply pick one that works for you for this task, either from those covered in lecture and practical class materials or any other classi- ?ers from any python packages. A good starting point is the scikit-learn (i.e. sklearn) package. 

This IT Assignment has been solved by our IT Experts at My Uni Paper. Our Assignment Writing Experts are efficient to provide a fresh solution to this question. We are serving more than 10000+Students in Australia, UK & US by helping them to score HD in their academics. Our Experts are well trained to follow all marking rubrics & referencing style.

Be it a used or new solution, the quality of the work submitted by our assignment Experts remains unhampered. You may continue to expect the same or even better quality with the used and new assignment solution files respectively. There’s one thing to be noticed that you could choose one between the two and acquire an HD either way. You could choose a new assignment solution file to get yourself an exclusive, plagiarism (with free Turnitin file), expert quality assignment or order an old solution file that was considered worthy of the highest distinction.