CSE251A - Introduction to Machine Learning Algorithms

Assignment Task

The error rate of nearest neighbor

Give an example of a binary classification dataset with points (x, y) for which the 1-NN classifier does not have zero training error (that is, it makes mistakes on the training set). You should plot the three points and show where the error is.

Gradient Descent - Linear Regression

Consider a house rent prediction problem where you are supposed to predict price of a house based on just its area. Suppose you have n samples with their respective areas, x (1), x(2 x(n) their true house rents y (1), y(2)y(n) Let’s say, you train a linear regressor that predicts f x (I) = θ0 + θ1x (I) The parameters θ0 and θ1 are scalars and are learned by minimizing- ing mean-squared-error loss through gradient descent with a learning rate α. 

1. Express the loss function(L) in terms of x(I) y(I) , n, θ0, θ1.

2. Compute

3. Compute

4. Write update rules for θ0 and θ1

Gradient Descent - Linear Regression with L1 Regularization

Consider the same house rent prediction problem where you are supposed to predict the price of a house based on just its area. Suppose you have n samples with their respective areas, x(1), x(2) x(n) , their true house rents y (1), y(2) y(n) Let’s say, you train a linear regres- sor that predicts f(x (I) ) = θ0 + θ1x (I) The parameters θ0 and θ1 are scalars and are learned by minimizing mean-squared-error loss with L1-regularization through gradient descent with a learning rate α and the regularization strength constant λ.

Implementing a Linear Regression Model

from Scratch Now, you will implement a linear regression model from scratch. We have provided a skeleton code file (i.e. LinearRegression.py) for you to implement the algorithm as well as a notebook file (i.e. Linear Regression.ipynb) for you to conduct experiment and answer relevant ques- tions. Libraries such as numpy and pandas may be used for auxiliary tasks (such as matrix 2 multiplication, matrix inversion, and so on), but not for the algorithms. That is, you can use numpy to implement your model, but cannot directly call libraries such as scikit-learn to get a linear regression model for your skeleton code.

This CSE251A – Engineering has been solved by our PHD Experts at My Uni Paper.