Comparison of Logistic Regression, Neural network, and Random Forest Assignment

Assignment Task

Questions

i. Variance/Bias Tradeoff : Below, you are provided two classifiers, and you need to identify the tradeoff between variance and bias in each case (i.e. for g. compare the second classifier to the first and ideltify if the bias/variance is lower or higher). Provide a justification as to why that is the case.

• Logistic Regression vs Neural Network
• Logistic Regression vs Decision Tree
• Decision Tree vs Random Forest
• Decision Tree vs Gradient Boosted Tree
• Logistic Regression vs 1NN classifier

ii. Bagging with Linear Regression: Imagine that instead of Random Forest, which performs bagging on decision trees, we perform bagging on a linear regression model. What will be the algorithm? Will this identical to running a single linear regression? Also, comment on the bias and variance of the bagged linear regression model in comparison with a simple linear regression.

iii. Gradient Boosting: In class, we studied gradient boosted decision trees with the squared L2 Loss. Provide the algorithm for gradient boosting if instead of the squared L2 loss, we use the Logistic Loss and the Hinge Loss.

iv Neural Networks with Logistic Loss and ReLU non-linearities: In class, we derived the back-propogation and gradient descent expressions for Squared L2 Loss (linear regression loss) with sigmoid Derive the reccursive expressions if instead, we have the logistic loss (assume binary classification) and ReLU non-linearities.

This CS 6375 - IT Computer Science has been solved by our PhD Experts at My Uni Paper.