BIG DATA AND DATA ANALYTICS - University Of Newcastle Australia - Data Analytics Assignment

Internal Code: MAS5794

Data Analytics Assignment:

Questions: 1) Use R to perform a multiple linear regression that regresses MEDV on CRIM (per capita crime rate by town), RM (average number of rooms per dwelling), NOX (nitric oxides concentration; parts per 10 million), DIS (weighted distances to five Boston employment centres), and AGE (proportion of owner-occupied units built prior to 1940). Interpret the coefficients and report the results of the regression in APA style (including a regression table and reporting of F-values). 2) Use R to create a new factor variable called NOXCAT that categorizes the suburbs into towns with LOW, MEDIUM, and HIGH nitric oxides concentration (based on the variable NOX). The categorization should be as follows: - LOW (<= 30% Quantile) - MEDIUM (> 30% Quantile & <= 70% Quantile) - HIGH (> 70% Quantile) Then, use ggplot to create a boxplot that shows MEDV for the different values of NOXCAT (LOW, MEDIUM, HIGH). 3) The newly created variable NOXCAT is a categorical variable with three possible values (LOW, MEDIUM, and HIGH). Use R to manually create a set of dummy variables (for different values of NOXCAT) and then regress MEDV on the different NOX categories. The coding of the dummy variables in the regression should be such that the intercept reflects the MEDV value of suburbs in the MEDIUM category. Interpret the coefficients. 4) Use ggplot() to create a scatterplot of MEDV by LSTAT. Add a linear fit (red), a quadratic fit (green), and a cubic fit (blue) to the plot. 5) Use Leave-One-Out Cross-Validation (LOOCV) to compare a linear model, a quadratic model, a cubic model, and a quartic model to regress MEDV on LSTAT. Interpret the results based on the mean- squared error (MSE). 6)  Use 11-fold cross-validation to compare 8 different degrees of polynomials to regress MEDV on LSTAT. Use ggplot() to plot the mean squared error (MSE) over the 8 different degrees of polynomials. Interpret the results based on the MSE. Why is 11-fold cross- validation in this particular case advantageous compared to 10-fold cross-validation?