Pupil Capture, Player, and Service release

Overview

We are pleased to announce the release of Pupil v1.6!

Download the latest bundle and let us know what you think via the #pupil channel 😄

Features

Audio Playback in Player -- #1111

Starting with v1.6, Pupil Player supports playback of recorded audio. Audio playback only works if the playback speed is set to 1x. Mute functionality and audio wave visualization will be implemented in the next release.

Developer notes

The audio playback implementation is based on portaudio and its pyaudio wrapper. Installing these will be required to run from source. See the developer doc changes for details.

Audio Capture for Windows -- #1110

We are very happy to add support for audio capture on Windows!

iMotions Exporter -- #1118

We added a plugin in Pupil Player that will enable you to export Pupil data to iMotions data format. You can import the exported Pupil data into iMotions software.

Experimental Fingertip Calibration -- #1108

This experimental calibration routine allows you to self-calibrate by using your fingertip as a reference point. This method should be used for prototyping and experimentation only. The detection robustness is not yet production grade, but we will put a lot more effort into this going forward. We wanted to release the idea so that you can try it out, suggest improvements, and provide feedback feedback.

Accuracy Visualizer -- #1126

Accuracy Visualizer is now a default plugin. This plugin will visualize the calibrated area. You can turn the visualization off in the plugin's settings menu.

Bugfixes

• HoloLens Relay: Fix gaze topic format change regression -- 6534ec
• UVC Source: Use hardware timestamps for Pupil Labs cameras only -- #1123
• Automatic detection and reinitialization if 200Hz eye camera exhibits stripes: issue #1116 -- #1024. If you are still experiencing this behavior after updating to v1.6, please open an issue and we will follow up with you to debug.

Pupil Capture, Player, and Service release

Overview

We are pleased to announce the release of Pupil v1.5!

Download the latest application bundle and let us know what you think via the #pupil channel 😄

Features

Capture

• Indicate annotation firing with a short log message. This provides realtime feedback for users that fire annotations via Pupil Remote -- #1080
• Screen-based calibration plugins: Session-persistent monitor selection -- #1091
• Capture allows creation of recordings without an active world video source, requires pyndsi>=0.4 -- #1101.
• Enable eye video recordings by default - #1101
• Enable/disable eye video recordings using the recording.should_start notification -- #1101

Player

Trim section and export improvements -- #1093

• Remove trim section indicators from video and raw data exporters
• Add trim section indicators to general settings, showing both:
  • relative recording time
  • frame indices
• Add export_info.csv file to export folder. This csv file contains the following information:
  • Player version
  • Data format version
  • Export start date and time
  • Frame index trim range
  • Relative time trim range
  • Absolute time trim range (world timestamp range)
• We no longer use trim range with frame indices as a naming convention
• We now use incrementally increasing folder names with format %d%d%d, e.g.:
  • <recording>/exports/000/
  • <recording>/exports/001/
  • <recording>/exports/002/

Confidence threshold consistency -- #1100

• Previously, all binocular mappers dismissed pupil data with confidence lower than 0.6 to prevent bad binocular mapping. Now, low confidence pupil data is mapped monocularly.
• Introduction of a user settable global minimum calibration confidence threshold. This threshold sets the lower bounds of the pupil data confidence that is used for calibration. This value was previously fixed to 0.8 in Pupil Capture and to 0.0 in Pupil Player -- effectively using low confidence pupil data for calibration. This threshold can be set in all calibration plugin menus.

These changes were introduced based on #1052 and #1047.

Minimum data confidence: Lower bounds gaze data by confidence that is used to generate visualizations and high level data, e.g. fixations. This does not effect the raw data export. Following data points are effected by it:

• Fixations
• Offline surface heatmaps
• Circle, cross, light points and polyline visualizations

Bugfixes

• Fix Fake_Source compatibility with Frame Publisher -- #1085
• Service does not crash when starting Frame_Publisher via a notification -- #1099
• Frame_Publisher only warns once on encountering an incompatible format instead of warning each time -- #1099
• Fix an issue where Fake_Source would return frames at 2 fps in Capture if it was restored from the session settings. -- #1085
• Offline Fixation Detector: Correct upper limit of fixation lenghts to 4000ms -- #1084
• Avoid ZeroDivisionError when calculating hdpi factors on Windows -- #1090
• Avoid positioning windows by default behind the Windows task bar -- #1103
• Fix bug where export folders were written into the recording folder itself instead of into the exports subfolder
• UVC control settings are correctly restored from session settings and on reconnect -- #1099 and #1101
• Avoid crash on recordings with less than 10 world frames -- #1101
• Player always checks for recordings without world videos instead of only checking on upgrading the recording format - #1101
• Fixed a bug that crashed Player when removing a surface - #1106
• Fixed a bug that crashed Capture on restoring a minimized window on MS Windows 10 when log display was loaded - #1097

Pupil Capture, Player, and Service release

v1.4 Overview

We are pleased to announce the release of Pupil v1.4!

Download the latest bundle and let us know what you think via the #pupil channel 😄

Features

Pupil Player

• Support for Pupil Mobile recordings without world video -- #1071
• Annotation csv exports include user-defined fields -- #1061
• Trim calibration and mapping ranges using trim marks -- #1057
• Add Jump to previous fixation thumb button to the Offline Fixation Detector plugin -- #1072
• Export ranges are based on time instead of frame indices. The export folder will be named appropriately -- #1072
• Resuming playback at the end of the trim section will seek to the beginning of the trim section instead of continuing playing after the trim section. This allows you to play back the trimmed section repeatedly without having to seek manually -- #1072
• Pupil Player default plugin list -- d40a3b
  • Remove the Vis Scan Path
  • Add the Raw Data Exporter
• FakeCapture (aka Test Image) frame content now displays the Pupil Labs gradient.

Bug Fixes

• Improve Player timeline drawing performance by caching the drawn data -- #1063, #1068
• Improve performance while playback in Player is paused -- 2df34d
• Remove bug where the Vis Scan Path plugin would generate increasingly bad results with each playback iteration -- bf1977
• Catch one of reasons that lead to the GLFW: Cannot create window error on Windows

Feedback

As always we hope you enjoy the new release and please let us know if you discover any bugs. We look forward to your feedback!

Pupil Capture, Player, and Service release

v1.3 Overview

We are pleased to announce the release of Pupil v1.3!

There are a lot of new features - especially for Pupil Player - that we are excited to share with the Pupil community.

Screenshot of Pupil Player demonstrating the timeline graphs features during a "blink" event.

Download the latest bundle and let us know what you think via the #pupil channel 😄

Features

General

• Text input - Support for cut/copy/paste and double click to select all in text input fields - #1027
• Collapsed menu - Main menu is now collapsed by default#1019. You can expand the menu by clicking any of the the icons on the right hand side of the screen.
• Pupil detection - Disable coarse detection for eye videos with width less than 200 pixels #1022
• Pupil Service UI #1016

Pupil Capture

• 200hz eye camera - Improved support for the new Pupil Labs 200Hz eye cameras
• Online pupil detection opt-out - This feature enables one to opt-out of online pupil detection in Pupil Capture #1009. If you use this feature, you should record eye videos so that you can use offline pupil detection and calibration in Pupil Player (Recorder > Record eye).
• Real world start time - Add record start time as Unix timestamp to info.csv #1017 - this enables users to calculate when the recording started in real-world time.
• Single marker calibration - Add display modes for Single Marker Calibration #1021

Pupil Player

• Time based seeking in Player #1041
• Improved Player seek control
• Add more Player timelines
• Offline blink detection #1040

Offline Calibration Improvements

• Disabled automatic search for calibration markers
• Added option to cancel calibration marker seach
• Added option to use detected calibration marker as natural features for manual filtering
• Improved caching of detected data

Improved Player Seek Control

• Highlight elements on hover that can be activated when being clicked or dragged
• Playback modes and speeds #1023
  • Frame-by-frame seeking when paused
  • Decrease and increase playback speed while playing
• Point & click somewhere on the seek bar to jump to this position

Player Timelines

• Added pupil confidence graph in the timeline #1040
• Added 2d pupil diameter (pixels) graph in the timeline diameter, 2d, in pixels #1040
• Added blink events graph in the timeline #1040
• Added recorded FPS, for world and eye videos as a graph in the timeline #1051

Bug Fixes

• Fixed visual bug where offline gaze mapping would be stuck at 99% #1014
• Fixed crash when changing a text field in the Accuracy Visualizer menu #1007
• Fix passing a recording to Pupil Player as a command line argument #1006
• Fix bug where main menu would not expand #1006
• Improved ROI handling #1031

Feedback

As always we hope you enjoy the new release and please let us know of any bugs. We look forward to your feedback!

200hz video Pupil mobile bundle support

This has been added with v1.3-9! Please update to this version and re-download the recordings from your device.

Pupil Capture, Player, and Service release

v1.2 Overview

The new v1.2 release improves overall stability and includes support for the new 200Hz eye cameras. We highly recommend upgrading Pupil Player, Service and Capture to v1.2.

Features

• Support for the new 200Hz eye cameras (#983)
• Default eye video size to qVGA or smaller for better performance.
• Bundle support for RealSense 3D on macOS and Windows
• Add Hololens Relay - UDP relay for Pupil to HoloLens communication
• Improve calibration marker detection, see below for details (#961, #965)
• Offline Surface Tracker - Add confidence column to gaze export (#941, #945)
• Refactor Video Export procedure (#927)
• Refactor Batch Exporter (#931)

Bug Fixes

• Improve Retina display support (#935)
• Improve Camera Intrinsics Estimation stability (2028f1e, #969)
• Fix Offline Fixation Detector (#950)
• Improve HMD Calibration (3d) stability (#970)
• Include pyndsi v0.3.3 which fixes recording Pupil Mobile streams (#910 #976 #981)

Calibration marker detection

The calibration marker detection accuracy and efficiency has been improved. The procedure works best with our new markers:

Feedback

As always we hope you enjoy the new release and please do let us know of any bugs. We look forward to your feedback!

Pupil Capture, Player, and Service release

v1.1 Overview

We are pleased to announce the release of Pupil v1.1 - this release happily coincides with the 100th release of Pupil software!

There are a lot of new features in this release and we are especially excited to introduce Pupil's new GUI!

We highly recommend upgrading Pupil Player, Service and Capture to v1.1.

A note on versioning

TL;DR Please don’t read too deeply into version numbers. We will be deploying new software on a continuous basis and version numbers will continue to increment accordingly.

The core team had a healthy debate on about version numbering. Should we release v1.0 or v0.10.? This could seem like a trivial debate about naming conventions, but we also realize that there is a lot of “baggage” associated with versioning. We acknowledge that many people see v1.0 as a big step for a project. It often signifies a level of maturity of the project and community. Pupil is certainly growing into a platform with a healthy group of contributors and researchers around the world who depend on Pupil software and hardware. Pupil is certainly more mature now compared to where it was 5 years ago. But we want emphasize that Pupil is an active project that will always be in flux. We will always need feedback from the community, there will be bugs and together we will fix them! This version is just one small step forward of many to come. 😄

Prior to this release we were (ab)using Semantic Versioning. For those not familiar with the Semantic Versioning, the version numbers follow the format major.minor.patch. From a technical perspective, we should have released a major version when we first defined the plugin structure, and another major version whenever we redefined the plugin interface, and yet another major version when we defined the “API” with the IPC Backbone in v0.8.

Where’s v1.0?

With all the changes that we made with the new UI, it looks like we flew right past v1.0 and went straight to v1.1! There are more than 100 commits between the v1.0 and v1.1 tags, so a minor bump was required.

Features

• New GUI - A lot of work went into redesigning the GUI in Pupil Capture and Pupil Player. There are ton of new features; too many to list! Just dive in and let us know what you think via chat or make an issue if you find a bug!
  • Pupil Capture - Checkout the docs for a high level overview of the new GUI structure here
  • Pupil Player - The most significant GUI changes are in Pupil Player. The new GUI greatly simplifies the interface. No more floating plugin windows cluttering the window! We also introduce an expandable events timeline so that all temporal events can be clearly labeled and correlated with the timeline. For a high level overview check out the docs.
• Plugin Manager - The central place for loading and unloading other plugins. No more tedious Select to loaddrop down menus! 😄
• Pupil diameter history graph - You can now visualize changes in pupil diameter over time in Pupil.
• Improvements to libuvc, our usb video backend. Considerably more accurate timestamp generation and more robust failure behaviour on usb disconnect and recovery.

Bug Fixes

• Pupil Mobile file import - Pupil Player can now properly read and display files recorded by Pupil Mobile that were saved on the Android device (see #873)
• Fixation detector - Substantially decreased number of false negative detections
• Realsense backend - Improved failure behaviour on usb disconnect and recovery. Added pointcloud visualizer window.
• HMD calibration - Improved Network API based on feedback from HMD-eyes developers.
• Manual marker calibration - Improved detection accuracy of markers.

Backwards incompatible changes

We revised the plugin API with v1.0 in order to ensure that all plugins work consistently with the new GUI. Please be aware that some of these changes are incompatible with the old API. This means that old plugins need updating before they can run correctly in v1.1. Please see the these pull request notes on details about the changes. We also updated the plugin guide accordingly.

Disabled Plugins

These plugins will be disabled due to legacy incompatibilities:

• Batch Exporter
• Marker Auto Trim Marks

We will re-add them on demand and when we have time to refactor them.

We will work on refactoring these plugins and will release an up-to-date version with the next release. Follow development progress and express your interest in these features in the following issues:

• #911
• #914

Windows Driver Installation

Drivers are automatically installed and updated! Please let us know if this does not work for you so that we can improve the installer!

Feedback

As always we hope you enjoy the new release and please do let us know of any bugs. We look forward to your feedback!

Linux bundle has been updated to support Ubuntu 17.10.

Pupil Capture, Player, and Service release

v0.9.14 Overview

New feature release. We recommend updating Pupil Player and Capture to v0.9.14!

Features

• 3d World camera - We have added support for librealsense which enables Pupil Capture to capture depth and rgb streams from the Intel RealSense R200. Read more about this feature in the docs. Currently only supported when running from source. Bundle support is coming soon!
• Widows Driver Auto-Installer - Windows drivers are automatically installed when you run Pupil Capture on Windows 10.
• Accuracy Test - The accuracy test and visualizer has been refactored. This plugin enables one to easily test and visualize accuracy directly after a calibration. See #808 and #812.
• Diameter History Plugin - New plugin for Pupil Capture that displays a graph of pupil diameter for each eye using diameter data from the 3d model. See #820.
• Camera intrinsics - Refactored all distortion code. Added fisheye distortion models to support wide angle lenses and added more pre-set calibrations. See #789 for further explanation and discussion.
• Real-time Fixation Detector for Pupil Capture - refactored fixation detector to work in 2d and 3d mode alike. Reduced CPU load and added visual feedback.
• Real-time heat maps in Pupil Player - test the heatmap feature of our surface-tracker in real-time.
• Blink detector visualization - Update for the blink detector plugin. Blinks are now visualized in Pupil Capture by a screen shade on blink onset.
• Drag and drop support for video files in Pupil Capture - Select the Video File Source manager and drag and drop video files into the world and eye windows in Pupil Capture.
• Looped video playback for file source
• Stability and performance improvements of the libuvc usb video backend.
• Pupil detector size settings now scale with video resolution.
• Improved monocular 3d gaze mapper accuracy by allowing one more degree of freedom in the mapping function.
• Numerous small improvements in the Eye Video Overlay plugin for Pupil Player

Bug Fixes

• Frame publisher - a number of smaller fixes
• .mkv file support in the offline pupil detector
• Surface tracking marker detector fixes - a number of smaller fixes
• ROI settings are persistent again.

Windows Driver Installation

Drivers are now automatically installed and updated! Please let us know if this does not work for you so that we can improve the installer!

Feedback

As always we hope you enjoy the new release and please do let us know of any bugs. We look forward to your feedback!

Pupil Capture, Player and Service release

v0.9.13 Overview

New feature release. We recommend updating Pupil Player and Capture to v0.9.13!

Features

• Fully synced audio: Audio is recorded with timestamps and merged with video when exported in Pupil Player. (Mac and Linux only for now).
• Pupil Player now supports recordings made locally with Pupil Mobile.
• Offline pupil detection: You can now do post-recording pupil (re)-detection. Just make sure you record the eye video in capture.
• Offline Calibration using markers: If you include calibrations in your recording you can calibrate after using this method.
• Offline Calibration using natural features: You can calibrate post-recording by clicking known gaze locations.
• Pupil Capture now has a reset-settings-to-defaults button.
• Reverted back to old presets for 3d detection as they create more stable models.
• Pupil Time Sync is no longer a default plugin. This should be turned on when using groups and/or Pupil Mobile!

Bug Fixes

• Fixed a bug that allowed more than 2 eye processes to run.
• Improved libuvc performance on Windows (reduces CPU load).
• Numerous small fixes.

Deprecated/removed features:

• Removed manual gaze correction. Let us know if this feature is really required.

Hotfixes within this release (v0.9.13-48)

• Fixed offline surface Tracker
• Fixed demo recording import
• Improved libuvc shutdown procedure

Documentation and Demo

We are still short on docs for the new player methods. Check out the demo video here: https://www.youtube.com/watch?v=lPtwAkjNT2Q&list=PLi20Yl1k_57q2KXGIf125hdUrlDx79-xA

Windows Driver Installation

If you are installing Pupil on Windows 10 for the first time or after a windows update, please see Windows 10 driver setup instructions here.

Feedback

As always we hope you enjoy the new release and please do let us know of any bugs. We look forward to your feedback!

Windows release coming soon!

Pupil Capture, Player and Service release

v0.9.12 Overview

Bugfix and feature release. We recommend updating Pupil Player and Capture to v0.9.12!

Features

• Audio capture is now its own plugin. Audio packets are now timestamped and made available as events.
• Pupil Mobile now syncs its time with Pupil Capture.
• Protocol versioning for time-sync and NDSI (the protocol we use to talk to pupil mobile).
• H264 capture to file from pupil-mobile network stream is now officially supported.

Bug Fixes

• Increased timestamp granularity for Windows capture backend to avoid frames with the same timestamp.
• Fixed a bug that double applied gaze correction in the exported video.

Windows Driver Installation

If you are installing Pupil on Windows 10 for the first time, please see Windows 10 driver setup instructions here.

Pupil Capture, Player and Service release

v0.9.10 Overview

Bugfix and feature release. We recommend updating Pupil Player and Capture to v0.9.10!

Rewrite of Pupil time sync

• Rewrote the time sync plugin and defined a public protocol draft.
• We have added example implementation scripts of clients in pupil-helpers.

Bug Fixes

• Fixed three bugs in the offline surface tracker.

Windows Driver Installation

If you are installing Pupil on Windows 10 for the first time, please see Windows 10 driver setup instructions here.