Merge pull request #704 from pupil-labs/new_scanpath_alpha_lab

New scanpath alpha lab

alpha-lab/cards.json

@@ -51,13 +51,13 @@
   },
   {
     "title": "Generate Scanpath Visualisations",
-    "details": "Generate both static and dynamic scanpath visualisations using exported data from Pupil Cloud's Reference Image Mapper.",
+    "details": "Generate both static and dynamic scanpath visualisations using exported data from Pupil Cloud's Reference Image Mapper or Manual Mapper.",
     "link": {
       "text": "View",
-      "href": "/alpha-lab/scanpath-rim/#generate-static-and-dynamic-scanpaths-with-reference-image-mapper"
+      "href": "/alpha-lab/scanpath-rim/#generate-static-and-dynamic-scanpaths"
     },
-    "image": "/alpha-lab/gen-scanpath-rim.webp",
-    "category": "Reference Image Mapper"
+    "image": "/alpha-lab/scanpath_image_nad.webp",
+    "category": "Gaze Mapping"
   },
   {
     "title": "Uncover Gaze Behaviour on Phones",

alpha-lab/scanpath-rim/index.md

@@ -1,5 +1,5 @@
 ---
-description: Generate static and dynamic scanpaths with Reference Image Mapper
+description: Generate static and dynamic scanpaths with Reference Image Mapper and Manual Mapper
 permalink: /alpha-lab/scanpath-rim/
 tags: [Pupil Invisible, Neon, Cloud]
 ---
@@ -8,64 +8,60 @@ tags: [Pupil Invisible, Neon, Cloud]
 
 <TagLinks />
 
-<Youtube src="7V3X4XmbRAM"/>
+<Youtube src="F52WXh1txVs"/>
 
 ::: tip
-Picture this: Build and customize scanpath visualizations with your Reference Image Mapper exports!
+Picture this: Build and customise scanpath visualisations with your Reference Image Mapper or Manual Mapper exports!
 :::
 
-## Visualizing Gaze Exploration With Scanpaths
-Scanpaths are graphical representations of gaze over time. They provide a glimpse into how the observer has focused their attention on different aspects of a scene, making them a valuable tool for understanding a person's visual attention and perception. The video above illustrates this concept. It shows:
-- Fixation locations, visualized as blue numbered circles
-- Fixation durations, which are mapped to the size of the circles. Longer fixations correspond to larger circles
-- Saccades, represented by blue lines connecting the fixations
-- Saccade amplitudes, reflected by the length of the lines. Longer lines correspond to larger saccade amplitudes, indicating larger shifts in gaze
+## Visualising Gaze Exploration With Scanpaths
+Scanpaths are graphical representations of gaze behaviour over time. They show how an observer has focused attention on different aspects of a scene, making them a valuable tool for understanding a person's visual attention and perception. The video above illustrates this concept. It shows:
+- Fixation locations - Visualised as numbered circles
+- Fixation durations - Mapped to the size of the circles. Longer fixations correspond to larger circles
+- Saccades - Represented by lines connecting sequential fixations
+- Saccade distances (between sequential fixations) - Reflected by the length of the lines. Longer lines correspond to bigger gaze shifts
 
-In this guide, we will show you how to generate both static and dynamic scanpath visualisations using your Reference 
-Image Mapper exported data.
+In this guide, we will show you how to generate static and dynamic scanpath visualisations using your Reference Image Mapper or Manual Mapper exported data.
 
 ::: tip
-Before continuing, ensure you are familiar with the [Reference Image Mapper](https://docs.pupil-labs.com/neon/pupil-cloud/enrichments/reference-image-mapper/) 
-enrichment. Check out [this explainer video](https://www.youtube.com/watch?v=ygqzQEzUIS4&t=56s) for reference.
+Before continuing, ensure you are familiar with the [Reference Image Mapper](https://docs.pupil-labs.com/neon/pupil-cloud/enrichments/reference-image-mapper/) and the [Manual Mapper](https://docs.pupil-labs.com/neon/pupil-cloud/enrichments/manual-mapper/) enrichments.
 :::
 
 ## Extending Current Tools
-The [Reference Image Mapper](https://docs.pupil-labs.com/neon/pupil-cloud/enrichments/reference-image-mapper/) enrichment available in Pupil Cloud is a tool that maps gaze onto
-2D images and can subsequently generate heatmaps. However, it currently does not support the production of scanpath visualizations.
-Thus, we chose to develop a script that shows you how to build your own scanpaths using Reference Image Mapped data.
+The [Reference Image Mapper](https://docs.pupil-labs.com/neon/pupil-cloud/enrichments/reference-image-mapper/) and the [Manual Mapper](https://docs.pupil-labs.com/neon/pupil-cloud/enrichments/manual-mapper/) enrichments available in Pupil Cloud are tools that map fixations onto 2D images. Following the completion of these enrichments, you can generate heatmaps and Areas of Interest. However, they currently do not support the production of scanpath visualisations. Thus, we chose to provide an easy click-and-run solution that allows you to build your own scanpaths using the enrichment data. 
 
+To automate the process as much as possible, you can download your data directly via the Pupil Cloud API. Alternatively, you can manually download the data and upload the unzipped folder to your Google Drive. 
 
 ## Steps
-1. Run a [Reference Image Mapper enrichment](https://docs.pupil-labs.com/neon/pupil-cloud/enrichments/reference-image-mapper/) and download the results
-2. Download [this script](https://gist.github.com/elepl94/9f669c4d81e455cf2095957831219664) and follow the [installation instructions](https://gist.github.com/elepl94/9f669c4d81e455cf2095957831219664#installation)
+1. Run a [Reference Image Mapper](https://docs.pupil-labs.com/neon/pupil-cloud/enrichments/reference-image-mapper/) or a [Manual Mapper](https://docs.pupil-labs.com/neon/pupil-cloud/enrichments/manual-mapper/) enrichment.
+2. Data from your enrichment needs to be loaded onto Google Drive. You can choose between one of two options:
+   1. Use a Pupil Cloud API token to have the enrichment loaded into Google Drive automatically. You need to obtain a developer token from Pupil Cloud (click [here](https://cloud.pupil-labs.com/settings/developer) to obtain yours).
+   2. Download the enrichment manually from Pupil Cloud and then upload it to your Google Drive.
+3. Access our **[Google Colab Notebook](https://colab.research.google.com/drive/13FIZroc7ckyOs7APSSskqo7ew2JAZR-q?usp=sharing)** and carefully follow the instructions.
+
+<div class="mb-4" style="display:flex;justify-content:center;">
+ <a
+  href="https://colab.research.google.com/drive/13FIZroc7ckyOs7APSSskqo7ew2JAZR-q?usp=sharing"
+>
+  <img
+    src="https://img.shields.io/static/v1?label=&message=Open in Google Colab&color=blue&labelColor=grey&logo=Google Colab&logoColor=#F9AB00"
+    alt="Open in Google Colab"
+  />
+</a>
+</div>
+
+If you want to work with the code locally, feel free to download it from this [GitHub Repository](https://gist.github.com/nadje/7a8f1e5590ab9022be4253a5b0d8d238).
 
 ## Review the Scanpaths
-<Youtube src="X43aTIRjwgQ" />
-
-After the script has completed its execution, you'll find the resulting scanpath visualizations stored in a newly created 
-sub-folder named "scanpath." For each participant, you will obtain a reference image with the scanpath superimposed on it. 
-You will also find a video featuring a dynamic scanpath overlay. Finally, if you had multiple participants, an aggregated 
-visualization combining all participants' scanpaths will be available, enabling a more comprehensive overview of the subjects'
-gaze behavior.
-
-<div style="display: flex; justify-content: space-between;">
-    <div style="flex: 1; margin-right: 10px;">
-        <div style="width: 100%; text-align: center;">
-            <img src="./Jack_scanpath.jpeg" alt="Jack Scanpath" style="width: 100%; height: 100%;">
-        </div>
-    </div>
-</div>
-<br />
-<div style="display: flex; justify-content: space-between;">
-    <div style="flex: 1; margin-right: 10px;">
-        <div style="width: 100%; text-align: center;">
-            <img src="./general_scanpath.jpeg" alt="General Scanpath" style="width: 100%; height: 100%;">
-        </div>
-    </div>
-</div>
+After running the notebook, you'll find the following files in your folder:
+- **Static scanpath visualisation**: Saved as `scanpath_image.png`, showing the reference image used for your enrichment with the scanpath superimposed on it. The scanpath will include the participant(s) you selected and the number of fixations chosen for visualisation. If you have multiple participants, an aggregated visualisation combining all participants' scanpaths will be available, enabling a more comprehensive overview of the subjects' gaze behaviour.
+
+- **Dynamic scanpath visualisation**: A video featuring a dynamic scanpath overlay will be saved in your folder, separately for each wearer, e.g., `scanpath_video_nadia.mp4`. 
+
+![Static scanpath](./scanpath_image_nad.png)
 
 ::: tip
-If you need assistance in using this application, reach out to us via email at [[email protected]](mailto:[email protected]), on our [Discord server](https://pupil-labs.com/chat/), or visit our [Support Page](https://pupil-labs.com/products/support/) for dedicated support options.
+Need guidance in further customising your scanpath visualisations? Reach out to us via email at [[email protected]](mailto:[email protected]), on our [Discord server](https://pupil-labs.com/chat/), or visit our [Support Page](https://pupil-labs.com/products/support/) for dedicated support options.
 :::
 
 <style scoped>

neon/.vitepress/config.mts

@@ -102,6 +102,10 @@ let theme_config_additions = {
             text: "Offset Correction",
             link: "/data-collection/offset-correction/",
           },
+          {
+            text: "Measuring the IED",
+            link: "/data-collection/measuring-ied/",
+          },
           {
             text: "Scene Camera Exposure",
             link: "/data-collection/scene-camera-exposure/",
@@ -228,6 +232,10 @@ let theme_config_additions = {
             text: "IMU Timeline",
             link: "/neon-player/imu-timeline/",
           },
+          {
+            text: "Eye State Timeline",
+            link: "/neon-player/eye-state-timeline/",
+          },
           {
             text: "Gaze Offset Correction",
             link: "/neon-player/gaze-offset-correction/"

neon/data-collection/data-format/index.md

@@ -129,7 +129,7 @@ This file contains [saccades](/data-collection/data-streams/#fixations-saccades)
 
 
 ## 3d_eye_states.csv
-This file contains [3D eye states](/data-collection/data-streams/#_3d-eye-states) as well as [pupil diameter](/data-collection/data-streams/#pupil-diamaters) data.
+This file contains [3D eye states](/data-collection/data-streams/#_3d-eye-states) as well as [pupil diameter](/data-collection/data-streams/#pupil-diameters) data.
 
 
 | Field                     | Description | 

neon/data-collection/measuring-ied/index.md

@@ -0,0 +1,15 @@
+# Measuring the Inter-Eye-Distance (IED)
+
+The wearer's IED can be set in the Neon Companion app for applications requiring 
+precise pupillometry or eye-state measurements. 
+This does not require prior calibration of the device. However, the accuracy of Neon’s 3D eye-state and pupil-size 
+measurements can be enhanced by correctly setting the IED for each wearer.
+
+To add the wearer's IED, input the value into their ‘Wearer Profile’ in the Neon Companion app before 
+starting 
+a recording. The default IED is set to 63 mm, which is the average for adults.
+
+Here's a simple way to measure IED:
+1. Ask the person to look at a distant object (to avoid convergence)
+2. Hold a ruler in front of their eyes and measure from the center of one pupil to the center of the
+other

neon/neon-player/eye-state-timeline/index.md

@@ -0,0 +1,21 @@
+# Eye State Timeline
+
+This plugin visualizes [3D eye state](/data-collection/data-streams/#_3d-eye-states) and [pupil diameter](/data-collection/data-streams/#pupil-diameters) data. 
+
+![Eye State Timeline](./eye-state-timeline.webp)
+
+::: info
+The data will only be visualized if Eye State computation was enabled in the Neon Companion app during recording. 
+::: 
+
+## Export Format
+Results exported to `3d_eye_states.csv` with the following fields:
+| Field                     | Description | 
+| ------------------------- | -------- | 
+| **section id**            | Unique identifier of the corresponding section.     |
+| **recording id**          | Unique identifier of the recording this sample belongs to.     |
+| **timestamp [ns]**        | UTC timestamp in nanoseconds of the sample. Equal to the timestamp of the eye video frame this sample was generated with. |
+| **pupil diameter left [mm]**   | Physical diameter of the pupil of the left eye. |
+| **pupil diameter right [mm]**   | Physical diameter of the pupil of the right eye. |
+| **eye&nbsp;ball&nbsp;center&nbsp;left&nbsp;x&nbsp;[mm]**<br /> **eye ball center left y [mm]**<br /> **eye ball center left z [mm]**<br /> **eye&nbsp;ball&nbsp;center&nbsp;right&nbsp;x&nbsp;[mm]**<br /> **eye&nbsp;ball&nbsp;center&nbsp;right&nbsp;y&nbsp;[mm]**<br /> **eye ball center right z [mm]** | Location of left and right eye ball centers in millimeters in relation to the scene camera of the Neon module. For details on the coordinate systems see [here](/data-collection/data-streams/#_3d-eye-states). |
+| **optical axis left x**<br /> **optical axis left y**<br /> **optical axis left z**<br /> **optical axis right x**<br /> **optical axis right y**<br /> **optical axis right z** | Directional vector describing the optical axis of the left and right eye, i.e. the vector pointing from eye ball center to pupil center of the respective eye. For details on the coordinate systems see [here](/data-collection/data-streams/#_3d-eye-states). |

neon/neon-player/fixation-detector/index.md

@@ -23,4 +23,3 @@ Results exported to `fixations.csv` with the following fields:
 | **duration [ms]** | Duration of the fixation in milliseconds.     |
 | **fixation x [px]** | Float value representing the x-coordinate of the fixation in world camera pixel coordinates. This position is the average of all gaze samples within the fixation.     |
 | **fixation y [px]** | Same as "fixation x [px]" but for the y-coordinate.     |
-| **fixation y [px]** | Same as "fixation x [px]" but for the y-coordinate.     |