Linear Regression scratch

Lecture 28:- Linear Regression scratch

Implementing linear regression from scratch involves building the model and optimizing it without using any existing machine learning libraries like scikit-learn. Here, we'll implement simple linear regression, which deals with one independent variable. We'll use the method of least squares to find the best-fitting line.

Let's assume we have a dataset with n data points, where x is the independent variable, and y is the dependent variable.

Steps to implement linear regression from scratch:

Define the Model: The model for simple linear regression is represented by the equation y = mx + b, where m is the slope (coefficient) and b is the y-intercept.

Find Coefficients: To find the coefficients m and b, we'll calculate them using the formula:

Make Predictions: Once we have the coefficients, we can use the model to make predictions for new data points.

Evaluate the Model: We'll use a performance metric like R-squared to evaluate how well the model fits the data.

Here's a Python implementation of simple linear regression from scratch:

pythonCopy code

def mean(values):    return sum(values) / float(len(values)) def simple_linear_regression(x, y):    # Calculate the mean of x and y    x_mean, y_mean = mean(x), mean(y)        # Calculate the slope (m) and y-intercept (b)    numerator = sum((xi - x_mean) * (yi - y_mean) for xi, yi in zip(x, y))    denominator = sum((xi - x_mean)**2 for xi in x)    m = numerator / denominator    b = y_mean - m * x_mean        return m, b def predict(x, m, b):    return [m * xi + b for xi in x] def r_squared(y_true, y_pred):    mean_y = mean(y_true)    ss_total = sum((yi - mean_y)**2 for yi in y_true)    ss_res = sum((yi - ypi)**2 for yi, ypi in zip(y_true, y_pred))    r2 = 1.0 - (ss_res / ss_total)    return r2 # Example usage: x = [1, 2, 3, 4, 5] y = [2, 3, 4, 2, 3] # Find coefficients m, b = simple_linear_regression(x, y) # Make predictions predictions = predict(x, m, b) # Evaluate the model r_squared_value = r_squared(y, predictions) print("Slope (m):", m) print("Y-Intercept (b):", b) print("Predictions:", predictions) print("R-squared:", r_squared_value)