Data Collection

Overview

This exercise is about collecting a small dataset of eye images. It is made as a tutorial and is to be used for later exercises and assignments.

Data collection is a fundamental part of the machine learning pipeline. A solid understanding of this process is essential to apply techniques effectively and interpret data appropriately.

You can always return to this exercise to capture new data.

Read all instructions carefully before beginning the data collection process. At least one group member must complete the data collection. If the collected data is not shared within the group, each member is required to perform their own data collection.

Figure 1:

Data collection setup

Figure 2:

Four predefined calibration patterns

Equipment

Figure 1 illustrates the setup. It includes an infrared (IR) camera with four IR lights and a headrest for stabilization of the head. You will use it to find the features of the eye to estimate where people are looking (gaze).

Process

The intention is that a test subject is seated in a chair with a screen in front and places the chin in the headrest.

Running the script, the test subject will be presented with four calibration patterns, grid , circle , line , and random , as shown in Figure 2. During the calibration procedure, a single target will appear and will move around in the predefined patterns. The test subject gaze at the target on the screen as it moves. Each fixation point is held for 2 seconds before moving to the next target.

Run

Task 1: Data collection 1
  1. Locate the file collect_data.py .
  2. Open a terminal and navigate to the location of the file.
  3. Run the script by typing python collect_data.py
Figure 3:

Intended camera view

Figure 4:

Type in information

Figure 5:

Press r to start calibration

Figure 6:

Gaze at the target

Figure 7:

The print statements of the program in Mac terminal

When the script is executed, a window displays camera feedback alongside a dialog for entering participant information.

Task 2: Data collection 2

This task is about running the data collection.

After the session, the script processes the raw video to extract frames. This takes a few minutes, with progress shown in the terminal (see Figure 7 for the expected output)

  1. Adjust the camera to capture the left eye, ensuring all four glint reflections are visible, as shown in Figure 3.
  2. Complete the dialog form with participant details. For participants involved in multiple sessions, the script reuses existing information, see Figure 4.
  3. After entering the details, a gray screen appears with instructions to begin calibration as shown in Figure 5. Press r to start recording. For each of the four calibration patterns, maintain gaze on the target displayed on the screen and follow its trajectory, as shown in Figure 6.

Keep the head as stable as possible using the headrest. You are allowed to blink during the session.

Once processing completes, a folder structure as shown in Figure 8 has been generated with the following hierarchy:

  1. Python script for data collection and 'session' folder.
  2. Test subjects recorded and file metadata.json.
  3. Session(s) for the corresponding test subject.
  4. Session divided into the different calibration patterns and full raw session video.
  5. Event log and screen coordinates csv files, 'frames' folder.
  6. Frames extracted from the video of the calibration pattern.

The sessions folder contains all collected data, organized into separate folders for each participant and trials as well as a metadata.json file containing metadata for all sessions. Each trial folder contains the .mp4 file with raw video data and subfolders for each calibration pattern. Each pattern contains two .csv files:

  1. fixation_coord.csv , which includes target coordinates in pixels and normalized format
  2. event_log.csv , which contains start and stop times and fixation intervals for each target in the pattern.
Figure 8:

Folder structure