import filtering_util
import numpy as np
import matplotlib.pyplot as plt
from scipy.ndimage import gaussian_filter1d

Filtering gaze data

This exercise introduces different techniques for processing and analyzing gaze data using filtering. In particular the exercise removing noise and identifying fixations and saccades in the signal. A fixation is defined as an instance with little or no eye movement. A saccade is an eye movement between two fixations as illustrated in Figure 1 for an example (image from researchGate ).

The visualizations presented throughout the exercise are based on data collected from test_subject_3 using the grid pattern.

The exercise contains functions in the filtering_util.py file to create plots.

test_subject = 'test_subject_3'

csv_file =f'../W06/data/output/{test_subject}/grid/pupil_coordinates.csv'

pupil_coor= filtering_util.load_csv1(csv_file)

filtering_util.plot_pupil_coor(pupil_coor['px'], pupil_coor['py'], 'Original')
plt.show()

The next task involves visualizing the pupil coordinates signals px and py for the gaze grid pattern. In this task, a $1D$ Gaussian filter will be applied to smooth the signals individually. Figure 2 illustrates the Gaussian $1D$ filters and the corresponding filtered signal. The cell below convolves a Gaussian filter to gaze data.

#Write your reflections here...

def apply_gauss(data, sigma = 3):
    filtered = gaussian_filter1d(data, sigma=sigma, order = 0)
    return filtered



px = np.asarray(pupil_coor['px'])
py = np.asarray(pupil_coor['py'])

# Write your implementation

filtering_util.plot_x_and_y(filtered_x, filtered_y)

#Write your reflections here...

Partials derivatives

#Write your reflections here...

The following task is about implementing a method to find derivatives a signal using the filter $[-1, 1]$.

The function plot_x_and_y_complete from filter_util.py takes two dictionaries one for each coordinates and plots the signals. The dictionaries should contain keys value pairs of the original and processed signal.

def get_partial_derivatives(data):
    """
    Convolve 1D data with a [-1, 1] filter.

    Args:
        data (numpy array): 1D array of data.

    Returns:
        numpy array: Convolved result.
    """
    # Write your implementation

der_x = get_partial_derivatives(filtered_x)
der_y = get_partial_derivatives(filtered_y)

x_collected = {'underlaying': px, 'filtered': filtered_x, 'derivative': der_x}
y_collected = {'underlaying': py, 'filtered': filtered_y, 'derivative': der_y}

filtering_util.plot_x_and_y_complete(x_collected, y_collected)

#Write your reflections here...

The following task are about detecting events (fixations and saccades) in eye signals using derivative filters.

def saccade_detection(der):
    """
    Detects saccades (non-zero changes) in the derivative data.
    Parameters:
    der (1 x N numpy array): Representing the derivative of the signal.
    Returns:
    saccades (list): A list of indices where there are non-zero changes (saccades) in `x_dev`.
    """
    saccades = []
    # Write your implementation
return saccades


x_saccades = ...
y_saccades = ...

x_saccade = {'underlaying': px, 'filtered': filtered_x, 'derivative filtered': der_x, 'saccade': x_saccades}
y_saccade = {'underlaying': py, 'filtered': filtered_y, 'derivative filtered': der_y, 'saccade': y_saccades}

filtering_util.plot_x_and_y_complete(x_saccade, y_saccade)

def fixation_filtering(px, py, x_saccades, y_saccades):
    """
    Removes noise from the input data based on detected saccades. If a spike is detected
    at a given index, the previous valid value is used to replace the current value.
    Parameters:
        px (1 x N nupy array): Representing the x coordinates (input signal).
        py (1 x N nupy array): Representing the y coordinates (input signal).
        x_saccades (list): A list of indices where saccades are detected in the x direction.
        y_saccades (list): A list of indices where saccades are detected in the y direction.
    Returns:
    px_new (list): A list of the cleaned x coordinates.
    py_new (list): A list of the cleaned y coordinates.
    """
    px_cleaned = []
    py_cleaned = []
    # Write your implementation
return px_cleaned, py_cleaned



cleaned_px, cleaned_py = ...
der_x_cleaned = ...
der_y_cleaned = ...
x_saccades_cleaned = ...
y_saccades_cleaned = ...

x_collected['cleaned'] = cleaned_px
y_collected['cleaned'] = cleaned_py

x_collected['derivative cleaned'] = der_x_cleaned
y_collected['derivative cleaned'] = der_y_cleaned

x_collected['saccade detected'] = x_saccades_cleaned
y_collected['saccade detected'] = y_saccades_cleaned

filtering_util.plot_x_and_y_complete(x_collected, y_collected, 5) #5 is a scaling factor on the values of the derivative for display purposes
filtering_util.plot_pupil_coor(px, py, 'Original')
filtering_util.plot_pupil_coor(cleaned_px, cleaned_py, 'Cleaned')
plt.show()

#Write your reflections here...

The next step is to utilize the charateristics of the derivative of the smoothed signal to detect the frames (pupil coordinates) corresponding to each screen calibration point, so the data can be sorted in corresponding input and label pairs.

def get_frames_pupil_movement(cleaned_der, data):
    """
    Identifies segments of pupil movement based on changes in the cleaned derivative.
    Parameters:
        cleaned_der (list or numpy array): A 1D array of the derivative values of the cleaned data, indicating changes in pupil position over time.
        data (list or numpy array): The original data corresponding to the pupil movement.

    Returns:
        dict ( {string: tuple(int, int)} ): A dictionary where the keys are calibration point numbers (as strings) and the values are tuples
              containing the start and end frame numbers for each segment.
    """
    frames = {}
# Write your implementation

filtering_util.plot_pupil_coordinates(frames, px, py, cleaned_px, cleaned_py)

Analyzing results

In the following tasks the results of the filtering will be analyzed using mean, variance and covariance at each calibration target for both raw data and filtered data.

filtering_util.plot_statistics_comparison(px, py, cleaned_px, cleaned_py, frames)

Inspecting own data

# Write your reflections here...

Event detection using gradients

In this task you will explore the combined signal (changes in x and y direction simultainously) and use the gradient for event detection. Recall the definition for the gradient of the function $f(x,y)$: $$ \nabla f(x, y) = \left( \frac{\partial f(x,y)}{\partial x}, \frac{\partial f(x,y)}{\partial y} \right) $$ You will use the gradient magnitude, the length of the gradient defined by:

def gradient_mangitude(x,y):
    """
    Calculates the gradient mangitude in a 2D space.
    Args:
        x (N x 1 numpy array): Array of x coordinates.
        y (N x 1 numpy array): Array of y coordinates.
    
    Returns:
        (N x 1 numpy array): The Euclidean distance from the origin to the point (x, y).
    """
    # Write your implementation

magnitude = gradient_mangitude(der_x, der_y)
magnitude_cleaned = gradient_mangitude(der_x_cleaned, der_y_cleaned)

collected_signal = {}
collected_magnitude = {}
collected_signal['x'] = px
collected_signal['y'] = py
collected_signal['x cleaned'] = cleaned_px
collected_signal['y cleaned'] = cleaned_py
collected_magnitude['magnitude of gradient'] = magnitude
collected_magnitude['magnitude of gradient cleaned'] = magnitude_cleaned



filtering_util.plot_x_and_y_complete(collected_signal, collected_magnitude, 5) #5 is a scaling factor on the values of the derivative for display purposes

# Write your reflection here...