DSCI617: Analysis of NASA Server Logs" Set up Environment, Management Assignment Help

Assignment Task:

TASK:

Instructions for Code Cells Any set of instructions you see with a blue bar to the left will provide instructions for creating a single code cell. Part A: Set up Environment In this part of the project, we will set up our environment. We will begin with some import statements. Create a code cell to import matplotlib.pyplot (using the standard alias). Also import the SparkSession class. No other packages or imported tools should be used in this project. We will now create our SparkSession and SparkContext objects. In a new code cell, create a SparkSession object named spark and a SparkContext object named sc. Part B: Load and Process Data In this project, you will be working with a file containing one month of server log data collected from NASA.gov in August 1995. This file is located at the path /FileStore/tables/NASA_server_logs_Aug_1995.txt. An example of a record contained in this file is shown below. piweba4y.prodigy.com [01/Aug/1995:00:00:10] "GET /images/launchmedium.gif" 200 11853 Each line in this file contains 6 pieces of information, separated by spaces. These pieces of information are, in order:

1. The IP address or DNS hostname of the server making the request.

2. The date and time of the request.

3. The request type. This has four possible values: GET, POST, and HEAD.

4. The location of the resource being requested.

5. The HTTP status code the server sent back to the client.

6. The number of bytes transferred to the client. This is set as a hyphen character '-' in some records. Notice that the request type and resource location are surrounded by a pair of double quotes. We begin by loading the data file into an RDD. Load the contents of the server log file into an RDD named nasa_raw. Print the number of elements in this RDD. We will now display the first few elements of the RDD. Print the first 10 elements of the RDD, with each element appearing on a different line of output. We will now process each line of server information by removing the double quotes, tokenizing the strings, replacing the hyphens that appear for the number of bytes with zeros, and coercing the values into appropriate datatypes. Perform the following steps in a single code cell.

1. Create a function named process_row(). This function should accept a single parameter named row, which is intended to represent individual elements of the nasa_raw RDD. The function should perform the follow processing tasks on the string contained in row, in the order described. a. Use the string replace() method to replace double quotes with empty strings. b. Tokenize the string on space characters using the split() method.

c. If the last token (indicating bytes) is equal to a hyphen, replace it with 0. d. Coerce the bytes token to an integer. (Note that the status code could be interpreted as an integer, but we will leave it as a string to more easily reflect that this is categorical information). e. Return the resulting list of tokens.

2. Apply the process_row() function to the elements of nasa_raw to create a new RDD named nasa. The new RDD should contain the same number of elements as nasa_raw, but these elements should be lists instead of strings.

3. Persist the nasa RDD to memory.

4. Print the first 10 elements of the nasa RDD, with each element appearing on a different line of output.

This DSCI 617 Management Assignment has been solved by our Management experts at My Uni Paper. Our Assignment Writing Experts are efficient to provide a fresh solution to this question. We are serving more than 10000+ Students in Australia, UK & US by helping them to score HD in their academics. Our Experts are well trained to follow all marking rubrics & referencing style.

Be it a used or new solution, the quality of the work submitted by our assignment experts remains unhampered. You may continue to expect the same or even better quality with the used and new assignment solution files respectively. There’s one thing to be noticed that you could choose one between the two and acquire an HD either way. You could choose a new assignment solution file to get yourself an exclusive, plagiarism (with free Turnitin file), expert quality assignment or order an old solution file that was considered worthy of the highest distinction.