Skip to content

Commit

Permalink
Merge pull request #723 from pupil-labs/cloud_2024q4_updates
Browse files Browse the repository at this point in the history
Cloud 2024q4 updates
  • Loading branch information
marc-tonsen authored Nov 29, 2024
2 parents 9e28a8e + 7addbea commit e798576
Show file tree
Hide file tree
Showing 44 changed files with 98 additions and 5 deletions.
1 change: 1 addition & 0 deletions invisible/.vitepress/config.mts
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,7 @@ let theme_config_additions = {
{ text: "Projects", link: "/pupil-cloud/projects/" },
{ text: "Workspaces", link: "/pupil-cloud/workspaces/" },
{ text: "Offset Correction", link: "/pupil-cloud/offset-correction/" },
{ text: "Add-ons", link: "/pupil-cloud/add-ons/" },
],
},
{
Expand Down
Binary file added invisible/pupil-cloud/add-ons/device_storage.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions invisible/pupil-cloud/add-ons/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
<!--@include: ../../../neon/pupil-cloud/add-ons/index.md-->
Binary file added invisible/pupil-cloud/add-ons/link_device.mp4
Binary file not shown.
Binary file added invisible/pupil-cloud/add-ons/link_device.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added invisible/pupil-cloud/add-ons/register_addon.mp4
Binary file not shown.
Binary file not shown.
Binary file not shown.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file added invisible/pupil-cloud/projects/create_project.mp4
Binary file not shown.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions neon/.vitepress/config.mts
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,7 @@ let theme_config_additions = {
{ text: "Projects", link: "/pupil-cloud/projects/" },
{ text: "Workspaces", link: "/pupil-cloud/workspaces/" },
{ text: "Offset Correction", link: "/pupil-cloud/offset-correction/" },
{ text: "Add-ons", link: "/pupil-cloud/add-ons/" },
],
},
{
Expand Down
Binary file added neon/pupil-cloud/add-ons/device_storage.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
41 changes: 41 additions & 0 deletions neon/pupil-cloud/add-ons/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# What Are Add-ons?
Pupil Cloud add-ons allow you to selectively upgrade services for your eye tracking device. Add-ons can be purchased through the Pupil Labs [website](https://pupil-labs.com/products/cloud/pricing). Each add-on is bound to a device.

## Register an Add-on
Typically registering an add-on can be done by clicking on a link that we send in an email when you purchase the add-on.

To manually register an add-on:

1. Click on the top right account avatar in the navigation bar, and then “Account Settings”
2. In the Cloud Add-ons tab, click on “Register Add-on”
3. Enter your activation key and your activation date
4. Confirm your add-on activation

<video width="100%" controls>
<source src="./register_addon.mp4" type="video/mp4">
</video>

## Link a Device

Depending on the add-on you purchased, they can be linked with a fixed number of eye tracking device. Linking a device with an add-on unlocks the functionality of the add-on for the device.

To link a device with a registered add-on, click `Link device` in the table. The drop-down menu for the hardware serial will contain the serials of all the devices that have uploaded recordings to one of your workspaces before. You can also obtain the serial of a new device via the Companion app and enter it here.

<video width="100%" controls>
<source src="./link_device.mp4" type="video/mp4">
</video>

## Devices View

The Devices view shows devices that have uploaded to workspaces you own. Here you can check how much storage each device is using in and compare this to their quota.

You can also the workspaces each device has uploaded to by double clicking on a device in the table.

![Devices Linking Devices](./link_device.png)

![Devices Storage](./device_storage.png)

::: info
**Enabling recording anonymization on upload:**<br>
Anonymizing recording on upload requires an Anonymization Add-on. If you have registered an Anonymization Add-on and linked a device to it you can you create a new workspace and enable `Face Blurring`. Any recording uploaded to this workspace will be anonymized.
:::
Binary file added neon/pupil-cloud/add-ons/link_device.mp4
Binary file not shown.
Binary file added neon/pupil-cloud/add-ons/link_device.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added neon/pupil-cloud/add-ons/register_addon.mp4
Binary file not shown.
Binary file not shown.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
8 changes: 8 additions & 0 deletions neon/pupil-cloud/enrichments/face-mapper/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,14 @@ Additionally, this enrichment also calculates the location of the most important

The face detection algorithm used under the hood for this enrichment is [RetinaFace](https://github.com/deepinsight/insightface/tree/master/detection/retinaface).

## Validate the Mapping

After the enrichment is successfully run, the recording timeline contains visualizations that indicate when faces were detected.

<video width="100%" controls>
<source src="./validate_facemapper.mp4" type="video/mp4">
</video>

## Export Format

### face_positions.csv
Expand Down
Binary file not shown.
4 changes: 4 additions & 0 deletions neon/pupil-cloud/enrichments/index.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
# What Are Enrichments?
Enrichments are tools that allow you to explore and visualize your data in new ways. These tools require complex algorithms and significant computational resources. Now, we put them at your fingertips, as you can run them with just a few clicks in Pupil Cloud. With enrichments, you can add new dimensions and derive new insights from your data. We can't wait to see how you use them!

<video width="100%" controls>
<source src="./create_enrichment.mp4" type="video/mp4">
</video>

## What Can I Do With Enrichments?
From mapping your gaze to real-world features of the environment, like surfaces, 3d objects, and faces, to aggregating and visualizing your data, enrichments can help you get the most out of your recordings. See below for a list of all available enrichments and their use cases.

Expand Down
Binary file modified neon/pupil-cloud/enrichments/manual-mapper/image_1_mapper.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified neon/pupil-cloud/enrichments/manual-mapper/image_2_mapper.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
10 changes: 8 additions & 2 deletions neon/pupil-cloud/enrichments/manual-mapper/index.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
# Manual Mapper

![Manual Mapper header image](./manual_mapper_header.png)
<video width="100%" controls>
<source src="./manualmapper_header.mp4" type="video/mp4">
</video>

The Manual Mapper offers a simple interface for manually mapping fixations to a reference image, aiding gaze data visualization and analysis. Click on the reference image to map a fixation and the recording auto-advances to the next one. Fixations outside the image can be marked by clicking beyond the image boundary.

Expand All @@ -11,14 +13,18 @@ The Manual Mapper offers a simple interface for manually mapping fixations to a
- The field `Fixation` shows the ID of the current fixation.
- The field `Recording Mapping Progress` indicates how many fixations have been mapped within the specified enrichment section.

In the example below, we manually map the fixations detected between the `epoch_start` and `epoch_end` events. This section includes 44 fixations in total, and we currently map the fixation with ID 72.
In the example below, we manually map the fixations detected between the `manual.start` and `manual.end` events. This section includes 101 fixations in total, and we currently map the fixation with ID 15.

![Manual Mapper image one](./image_1_mapper.png)

You will see all fixations and their durations visualized in the timeline below.

![Manual Mapper image two](./image_2_mapper.png)

::: tip
**Move between unmapped fixations**<br>
Use <kbd>Shift+A</kbd> to move to the previous unmapped fixation, and <kbd>Shift+D</kbd> to move to the next unmapped fixation.
:::

## Export Format

Expand Down
Binary file not shown.
22 changes: 21 additions & 1 deletion neon/pupil-cloud/enrichments/marker-mapper/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,10 @@ Note that the markers require a white border around them to be robustly detected

## Selecting Markers in the Cloud

<video width="100%" controls>
<source src="./mm_define.mp4" type="video/mp4">
</video>

By default, all visible markers are used for surface definition when creating a surface. You can add or remove markers by clicking on them. Markers that are part of the definition are shown in green, while others are shown in red.

Note that when adding a new marker to the surface definition, another marker that is already part of the definition has to be visible within the same video frame.
Expand All @@ -46,9 +50,25 @@ The Marker Mapper maps gaze points to a 2D surface and returns them in surface c
The mapper may return values outside of the surface, which yields values smaller than 0 or larger than 1, indicating that the corresponding gaze was not on the surface at that time.

::: tip
**Orientation:** The red border marks the top edge of the surface. You can rotate the surface by clicking on the "Rotate Surface" button.
**Orientation:** The red border marks the top edge of the surface. You can rotate the surface by clicking on "Rotate Surface" under the ellipsis next to 'Surface Defined'.
:::

## Select image to be used in Marker Mapper visualizations

<video width="100%" controls>
<source src="./mm_imageupload.mp4" type="video/mp4">
</video>

You can select the image to be used in Marker Mapper visualizations. You can either pick a frame from a scene video, or you can upload a dedicated image that contains the surface. Images should contain the selected markers, and will be cropped according to the surface definition. The selected image will be used in the side-by-side view and as the background for AOI and heat map visualizations.

## Validate the Mapping

After the enrichment is successfully run, the recording timeline contains visualizations that indicate when surface was localized, and when gaze was on the surface.

<video width="100%" controls>
<source src="./mm_enrichment_viz.mp4" type="video/mp4">
</video>

## Export Format

### gaze.csv
Expand Down
Binary file not shown.
Binary file not shown.
Binary file not shown.
4 changes: 3 additions & 1 deletion neon/pupil-cloud/enrichments/reference-image-mapper/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,9 @@ Why not try replicating the above examples? Or even try it with your own use cas

## Validate the Gaze Mapping

![Reference Image Mapper in Pupil Cloud](./rim-in-cloud.png)
<video width="100%" controls>
<source src="./validate_rim.mp4" type="video/mp4">
</video>

To check if gaze has been mapped successfully, use the side-by-side view:

Expand Down
Binary file not shown.
Binary file added neon/pupil-cloud/projects/create_project.mp4
Binary file not shown.
5 changes: 5 additions & 0 deletions neon/pupil-cloud/projects/index.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
# Projects

<video width="100%" controls>
<source src="./create_project.mp4" type="video/mp4">
</video>

A project is a set of recordings that are grouped for joined processing and analysis. You create projects from the Recordings view in Drive by selecting one or more recordings and clicking "create project from recording". You can also add projects to existing recordings.

Within a project, you can annotate recordings with events and create [enrichments](/pupil-cloud/enrichments/) to power your analysis.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion neon/pupil-cloud/visualizations/areas-of-interest/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ The AOI Editor allows you to draw areas of interest (AOIs) on top of the referen

### AOI Editing and Drawing

Upon completion of the enrichment, access the main view of the enrichment by navigating to the **`Enrichments`** tab and selecting **`Edit AOIs`** under the section **`Tools`**.
Upon completion of the enrichment, access the main view of the enrichment by navigating to the **`Enrichments`** tab and selecting **`Edit`** next to **`Areas of Interest`**.

![Edit AOIs](./AOI_enrichment_view.png)

Expand Down
4 changes: 4 additions & 0 deletions neon/pupil-cloud/workspaces/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,10 @@ Only accounts that have been invited to become members of a workspace are grante

Public sharing of data with people outside of a workspace is **not** currently possible.

### Workspace privacy settings
Privacy settings can be set up upon workspace creation, and cannot be changed once the workspace has been created. F
![Workspace settings](./workspace_settings.png)

### Frequently Asked Questions

**When should I use workspaces?**
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit e798576

Please sign in to comment.