Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update IMU figures, and rotation description #702

Merged
merged 1 commit into from
Aug 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file not shown.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file not shown.
10 changes: 7 additions & 3 deletions neon/data-collection/data-streams/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,16 +31,16 @@

The gaze estimation algorithm is based on end-2-end deep learning and provides gaze data robustly without requiring a calibration. You can find a high-level description as well as a thorough evaluation of the accuracy and robustness of the algorithm in our [white paper](https://zenodo.org/doi/10.5281/zenodo.10420388).

## Fixations & Saccades

Check warning on line 34 in neon/data-collection/data-streams/index.md

View workflow job for this annotation

GitHub Actions / ✍️ Check spelling

Unknown word (Saccades)

Available in: <Badge>Pupil Cloud</Badge><Badge>Neon Player</Badge>
The two primary types of eye movements exhibited by the visual system are fixations and saccades. During fixations, the eyes are directed at a specific point in the environment. A saccade is a very quick movement where the eyes jump from one fixation to the next. Properties like the fixation duration are of significant importance for studying gaze behavior.

Check warning on line 37 in neon/data-collection/data-streams/index.md

View workflow job for this annotation

GitHub Actions / ✍️ Check spelling

Unknown word (saccades)

Check warning on line 37 in neon/data-collection/data-streams/index.md

View workflow job for this annotation

GitHub Actions / ✍️ Check spelling

Unknown word (saccade)

![Fixations](./fixations.jpg)

Fixations and saccades are calculated automatically in Pupil Cloud after uploading a recording and are included in the recording downloads. The deployed fixation detection algorithm was specifically designed for head-mounted eye trackers and offers increased robustness in the presence of head movements. Especially movements due to vestibulo-ocular reflex are compensated for, which is not the case for most other fixation detection algorithms. You can learn more about it in the [Pupil Labs fixation detector whitepaper](https://docs.google.com/document/d/1CZnjyg4P83QSkfHi_bjwSceWCTWvlVtbGWtuyajv5Jc/export?format=pdf) and in our [publication](https://link.springer.com/article/10.3758/s13428-024-02360-0) in *Behavior Research Methods* discussing fixation detection strategies.

Check warning on line 41 in neon/data-collection/data-streams/index.md

View workflow job for this annotation

GitHub Actions / ✍️ Check spelling

Unknown word (saccades)

Check warning on line 41 in neon/data-collection/data-streams/index.md

View workflow job for this annotation

GitHub Actions / ✍️ Check spelling

Unknown word (vestibulo)

We detect saccades based on the fixation results, considering the gaps between fixations to be saccades. Note, that this assumption is only true in the absence of smooth pursuit eye movements. Additionally, the fixation detector does not compensate for blinks, which can cause a break in a fixation and thus introduce a false saccade.

Check warning on line 43 in neon/data-collection/data-streams/index.md

View workflow job for this annotation

GitHub Actions / ✍️ Check spelling

Unknown word (saccades)

Check warning on line 43 in neon/data-collection/data-streams/index.md

View workflow job for this annotation

GitHub Actions / ✍️ Check spelling

Unknown word (saccades)

Check warning on line 43 in neon/data-collection/data-streams/index.md

View workflow job for this annotation

GitHub Actions / ✍️ Check spelling

Unknown word (saccade)

The downloads for gaze mapping enrichments ([Reference Image Mapper](/pupil-cloud/enrichments/reference-image-mapper/#export-format), [Marker Mapper](/pupil-cloud/enrichments/marker-mapper/#export-format)) also include mapped fixations, i.e. fixations in reference image or surface coordinates respectively.

Expand Down Expand Up @@ -86,18 +86,22 @@

A fusion engine also combines these values with magnetometer readings to estimate the module's absolute orientation relative to magnetic north and gravity as a quaternion. Note that in order to obtain precise absolute yaw readings, the magnetometer needs to be [calibrated](/data-collection/calibrating-the-imu/).

The IMU is located in the top bar of the module and is sampled at 110 Hz. Its coordinate system is oriented with the x-axis pointing to the right, the y-axis pointing in front, and the z-axis pointing upwards.
The IMU is located in the top bar of the module and samples at 110 Hz. Its coordinate system is oriented with the x-axis pointing to the right, the y-axis pointing in front, and the z-axis pointing upwards.

![IMU Coordinate System](./imu-xyz-black.jpg)

When relating data from the IMU to things visible in the scene camera, it may be necessary to align their respective 3D coordinate systems. The IMU's coordinate system is rotated by 102° around the x-axis in relation to the scene camera's coordinate system.

![IMU Scene Camera](./imu-scene_camera-black.jpg)
![IMU Scene Camera](./imu-scene-camera-black.jpg)

::: tip
Note that leftward rotations in the IMU coordinate system are positive, whereas they are negative in the scene camera and 3D eye state coordinate systems.
:::

### Euler Angles

When exporting recordings from Pupil Cloud or Neon Player the IMU's orientation in Euler angles (i.e. roll, pitch, and yaw) is also available.

Pitch is defined as a rotation around the x-axis with a value range of -90° to +90°. Yaw and roll are rotations around the y- and z-axis, respectively, with value ranges of -180° to +180°.

![IMU Pitch, Yaw, Roll](./imu-pitch_yaw_roll-black.jpg)
![IMU Pitch, Yaw, Roll](./imu-pitch-yaw-roll-black.jpg)
Loading