MapReduce and Streaming Data Analytics Skills for Advanced Insights

Assignment Task

Purpose

The purpose of this assessment is to evaluate your learning and knowledge of using the MapReduce technique mentioned in Week 1, finding similar items and mining data streams. Your Task Your task is to complete the following exercises:

 1. Friend Recommendation System (Stanford) 

Write a MapReduce program in Spark (see Overview Module for download instructions) that implements a simple “People You Might Know” social network friendship recommendation algorithm. The key idea is that if two people have a lot of mutual friends, then the system should recommend that they connect with each other.

2. S-curve (exercise 3.4.1 in Leskovec, Rajaraman and Ullman) 

Evaluate the S-curve 1 − (1 − sr ) b for s = 0.1, 0.2, . . ., 0.9, for the following values of r and b;

• r=3 and b=10.
• r=6 and b=20.
• r=5 and b=50.

3. Filtering Streams (similar to Exercises of 4.3 in Leskovec, Rajaraman and Ullman)

1. For the situation of the running example of Section 4.3.1 in Leskovec, Rajaraman and Ullman with changed conditions (10 billion bits, 2 billion members of the set S). Calculate the false-positive rate when using three hash functions. Do the same for four hash functions.

2. As a function of n, the number of bits and m the number of members in the set S, what number of hash functions minimizes the false-positive rate?

This Data Science has been solved by our PhD Experts at My Uni Paper.