Linear Algebra

This exercise introduces fundamental linear algebra operations in Numpy and how to use them to solve linear systems of equations. The goal is to get familiarized with the concepts of linear algebra and how to use them in Numpy. The following topics will be covered:

  • Performing matrix operations (elementwise operations, transpose, multiplication, inverse).
  • Properties of matrix multiplication and inverse.
  • Representing linear equations in matrix form.
  • Solving linear equations using the matrix inverse.

The cell below defines matrices A , B , C , D , E that are used throughout the exercise:

import numpy as np
import matplotlib.pyplot as plt
# Define matrices to be used in the tasks:
A = np.array([
    [1, 0.5, 1/3, 0.25],
    [0.5, 1/3, 0.25, 0.2],
    [1/3, 0.25, 0.2, 1/6],
    [0.25, 0.2, 1/6, 1/7]
])

B = np.array([
    [-16, 15, -14, 13],
    [-12, 11, -10, 9],
    [-8, 7, -6, 5],
    [-4, 3, -2, 1]
])

C = np.array([
    [1, 1/2, 1/3, 1/4],
    [1/2, 1/3, 1/4, 1/5],
    [1/3, 1/5, 1/7, 1/9],
    [1/4, 1/7, 1/8, 1/9],
])

D = np.array([
    [2, 4, 5/2],
    [-3/4, 2, 0.25],
    [0.25, 0.5, 2]
])

E = np.array([
    [1, -0.5, 3/4],
    [3/2, 0.5, -2],
    [0.25, 1, 0.5]
])

D_inv = np.linalg.inv(D)
E_inv = np.linalg.inv(E)

Matrix Operations

Task 1: Elementwise difference
  1. Calculate $A-B$ in the code cell below.
# Write your solution here
Task 2: Multiplication properties
  1. Calculate $AC$ and $CA$ in the code cell below. (You may use either np.dot or the @ operator).
  2. Explain why the results are different.
# Write your solution here
Task 3: Elementwise multiplication - Hadamard product
  1. Calculate the elementwise multiplication of $A$ and $C$ using the * operator.
  2. Explain the difference between the * and @ operators.
# Write your solution here
Task 4: Inverses
  1. Use np.linalg.inv to calculate the inverse of $A$ and $C$.
  2. Verify that $AA^{-1}=I$ and $CC^{-1}=I$. If the results differ from your expectations, argue why this is the case.
Hint

The question relates to the limitations of floating point numbers.

# Write your solution here

Properties

Task 5: Inverse properties

Use the code cell below to verify that:

  1. $D^{-1}E^{-1} = (ED)^{-1}$
  2. $D^{-1}E^{-1} \neq (DE)^{-1}$
# Write your solution here
Task 6: The determinant
  1. Calculate the determinant of $A$, $B$, and $C$ using np.linalg.det .
  2. Determine which of the matrices $A,B,C$ have an inverse.
  3. Calculate the inverses of the matrices using np.linalg.inv . Explain what happens and how this is related to your answer in (2).
# Write your solution here
Task 7: Transpose
  1. Verify that $(D^{-1})^\top$ and ${D^\top}^{-1}$ are equal.
Hint

The transpose of a matrix A in Numpy can be calculated with A.T .

# Write your solution here

Linear equations

Matrices can represent systems of linear equations

$$ Ax=b $$

where $A$ is the coefficient matrix, $x$ vector of unknowns, and $b$ is a vector of the dependent variables.

A solution can be found using

$$ \begin{align*} A^{-1}Ax&=A^{-1}b\\ x &= A^{-1}b. \end{align*} $$
Task 8: Solving linear equation systems

For each of the following sets of linear equations determine whether a unique solution exits. Recall that the determinant can be used to determine whether a matrix has an inverse. Your answers have to be submitted in Grasple :

a)

$$ \begin{align*} 2x + 3y &= -1\\ x + y &= 0\\ \end{align*} $$

b)

$$ \begin{align*} 1x + 0y &= 5\\ 0x + 1y &= 7\\ \end{align*} $$

c)

$$ \begin{align*} 0x + y &= -1\\ -2x + -3y &= 2\\ \end{align*} $$

d)

$$ \begin{align*} x + -3y + 3z &= 0.5\\ x - 5y + 3z& = 0.5\\ 6z + -6y + 4x &= 1. \end{align*} $$

e)

$$ \begin{align*} 2x + 3y + 4z &= 2\\ x + 4z + y &= -2\\ 4z + 5y + 2x &= 3. \end{align*} $$

f)

$$ \begin{align*} x + y + z &= 2\\ 2x + 2z + 2y &= -2\\ 3z + 3y + 3x &= 3. \end{align*} $$ 
# Write your solutions here

Matrix multiplication

For an $N\times D$ matrix $A$ and a $D\times K$ matrix $B$, the matrix multiplication (or matrix product) is a new $N\times K$ matrix $R$. Elements $R_{ij}$ of $R$ can be calculated using the following formula

$$ R_{ij} = \sum_{d=1}^D A_{id}B_{dj}. $$

In other words, it is the dot product of the $i$'th row vector of $A$ and the $j$'th column vector of $B$.

Task 9: Implementing matrix multiplication

Implement matrix multiplication in the matmul function in the code cell below. You may use either Python lists or Numpy arrays, but the intention is to not use Numpy's built-in functions for matrix multiplication (i.e., np.dot , @ , np.matmul , etc.). You may, however, use np.dot for the purpose of computing the inner product between row and column vectors.

Hint

It might be helpful to calculate the correct result by hand first, to make debugging easier.

def matmul(a, b):
    # Implement this function
    ...
    

ma = [
    [1, 2, 3],
    [4, 5, 6],
    [7, 8, 9]
]

mb = [
    [5, 4, 9],
    [2, 1, 7],
    [8, 0, 1]
]

matmul(ma, mb)