Line fitting the matrix way

This in-class exercise walks you through the following steps to fit a line to several datasets:

  1. Create the datasets.
  2. Construct design matrices.
  3. Compute the weights using matrix inversion.
  4. Plot the data and the fitted line.

Linear (line) models

Task 1: Identifying parameters

  1. Run the cell below to load the neccessary libraries and to construct the datasets.

  2. Identify the inputs and the labels of each dataset. 

import numpy as np
import matplotlib.pyplot as plt

dataset_1 = np.array([[10, 2], [9, 6]])
dataset_2 = np.array([[5, 1], [7, 5]])
dataset_3 = np.array([[4, 2], [5, 3]])
Task 2: Identifying parameters and constructing the design matrix
  1. Complete the separate_inputs_labels function below. The function should take a dataset as input and return the inputs and labels separated into separate variables. The function should return a matrix X containing the inputs and an array y containing the labels. Use the function to obtain the inputs and labels for each dataset.
Hint

Slicing might be helpful here.

  1. Complete the code below and construct the design matrix for the other datasets. Print your results.
def separate_inputs_labels(dataset):
    """
    This function takes a dataset as input and returns the inputs and labels.
    
    Parameters:
    dataset (numpy array): The dataset to be separated.
    
    Returns:
    X (numpy array): The input matrix.
    y (numpy array): The labels array.
    """
    
    ...
    # return the results as a tuple
    return X, y

# construct the design matrix    
X1_design = None

# Print the datasets
print("X1 = \n",X1)
print("y1 =", y1)
print("Design Matrix for Dataset 1:\n", X1_design)
X1 = 
 [[10]
 [ 9]]
y1 = [2 6]
Design Matrix for Dataset 1:
 [[10.  1.]
 [ 9.  1.]]
Task 3: Solve for model parameters
  1. Find the inverse of the design matrix for each dataset constructed above.
  2. Calculate the model weights, then print your results.
  3. Use the plot_model function to plot your results.
  4. Visually inspect the plots and interpret the meaning and influence of each term.
print("Weights for Dataset 1:", wieghts1)

# Function to plot data points and fitted line
def plot_model(X, y, wieghts, dataset_name):
    # Plot the data points
    plt.scatter(X, y, color='blue', label='Given Points')
    
    # Extend x_vals range to include zero for correct y-intercept visualization
    x_vals = np.linspace(0, max(X) + 1, 100)
    y_vals = wieghts[0] * x_vals + wieghts[1]
    
    # Plot the fitted line
    plt.plot(x_vals, y_vals, color='red', label=f'Line: y = {wieghts[0]:.2f}x + {wieghts[1]:.2f}')
    
    # Plot the y-intercept
    plt.scatter(0, wieghts[1], color='green', zorder=5, label=f'Y-intercept (0, {wieghts[1]:.2f})')
    
    # Add title and labels
    plt.title(dataset_name)
    plt.xlabel('X')
    plt.ylabel('y')
    plt.legend()
    plt.grid(True)
    plt.show()
    

plot_model(X1, y1, wieghts1, 'Dataset 1')
Weights for Dataset 1: [-4. 42.]

If you have more time, complete the next task.

Task 4: A new dataset

Run the cell below to define a new dataset.

  1. Identify the inputs and the labels, then reuse the code from previous tasks to construct a design matrix.
  2. Calculate the inverse of the design matrix. This step should result in an error. What are the possible reasons for getting this error?

For pedagogical reasons, next week we will return to this dataset, as you will have the necessary tools to fit a model for this scenario.

dataset_4 = np.array([[9, 1], [7, 2],[3, 8], [1, 3],[4, 3]])