Linear R scratch - Part 3

Lecture 30:- Linear R scratch - Part 3

In the previous implementations, we covered finding the coefficients (slope and intercept) of the linear regression model and making predictions using forward propagation. Now, let's proceed with implementing the remaining parts:

Cost Function (Mean Squared Error): The cost function measures the error between the predicted values and the actual values. In linear regression, the Mean Squared Error (MSE) is commonly used as the cost function. It calculates the average of the squared differences between the predicted and actual values.

Gradient Descent: Gradient Descent is an optimization algorithm used to find the optimal values for the model's parameters (slope and intercept) that minimize the cost function (MSE). It iteratively updates the parameters in the opposite direction of the gradient of the cost function to reach the minimum.

Let's add the cost function and gradient descent to our implementation:

pythonCopy code

def mean(values):    return sum(values) / float(len(values)) def simple_linear_regression(x, y):    # ... (same as previous implementation) ... def predict(x, m, b):    return [m * xi + b for xi in x] def mean_squared_error(y_true, y_pred):    n = len(y_true)    mse = sum((yi - ypi)**2 for yi, ypi in zip(y_true, y_pred)) / n    return mse def gradient_descent(x, y, m, b, learning_rate, epochs):    n = float(len(y))    for _ in range(epochs):        y_pred = predict(x, m, b)        # Calculate the gradients for m and b        dm = (-2/n) * sum(x * (y - y_pred))        db = (-2/n) * sum(y - y_pred)        # Update m and b using the gradients and learning rate        m -= learning_rate * dm        b -= learning_rate * db    return m, b # Example usage: x_train = [1, 2, 3, 4, 5] y_train = [2, 3, 4, 2, 3] # Find coefficients using simple linear regression m, b = simple_linear_regression(x_train, y_train) # Make predictions x_test = [6, 7, 8] predictions = predict(x_test, m, b) # Calculate Mean Squared Error (MSE) mse = mean_squared_error(y_train, predict(x_train, m, b)) print("Mean Squared Error (MSE):", mse) # Apply Gradient Descent to further optimize the model learning_rate = 0.01 epochs = 1000 m_optimized, b_optimized = gradient_descent(x_train, y_train, m, b, learning_rate, epochs) print("Optimized Slope (m):", m_optimized) print("Optimized Y-Intercept (b):", b_optimized)

In this updated implementation, we added the Mean Squared Error (MSE) function to calculate the error between the predicted and actual values. We also introduced the Gradient Descent function to optimize the model's parameters (m and b) using an iterative update process based on the gradients of the cost function.

Gradient Descent allows the model to learn from the training data and minimize the error, resulting in improved predictions and better-fitted line. The learning rate and the number of epochs are hyperparameters that control the speed and convergence of the optimization process. They need to be chosen carefully to ensure stable convergence.

Note that this is still a simplified version of linear regression with gradient descent. In real-world scenarios, it's recommended to use libraries like scikit-learn, which provide more advanced features, optimizations, and robustness for handling various data and modeling tasks.