Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added docs for template access vie RT API #698

Merged
merged 2 commits into from
Jul 8, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 59 additions & 0 deletions neon/real-time-api/tutorials/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@

```python
from datetime import datetime
import matplotlib.pyplot as plt

Check warning on line 104 in neon/real-time-api/tutorials/index.md

View workflow job for this annotation

GitHub Actions / ✍️ Check spelling

Unknown word (matplotlib)

Check warning on line 104 in neon/real-time-api/tutorials/index.md

View workflow job for this annotation

GitHub Actions / ✍️ Check spelling

Unknown word (pyplot)
import cv2
from pupil_labs.realtime_api.simple import discover_one_device

Expand All @@ -118,8 +118,8 @@
print(f"For the left eye x, y, z: {gaze_sample.optical_axis_left_x}, {gaze_sample.optical_axis_left_y}, {gaze_sample.optical_axis_left_z} and for the right eye x, y, z: {gaze_sample.optical_axis_right_x}, {gaze_sample.optical_axis_right_y}, {gaze_sample.optical_axis_right_z}.")

scene_image_rgb = cv2.cvtColor(scene_sample.bgr_pixels, cv2.COLOR_BGR2RGB)
plt.imshow(scene_image_rgb)

Check warning on line 121 in neon/real-time-api/tutorials/index.md

View workflow job for this annotation

GitHub Actions / ✍️ Check spelling

Unknown word (imshow)
plt.scatter(gaze_sample.x, gaze_sample.y, s=200, facecolors='none', edgecolors='r')

Check warning on line 122 in neon/real-time-api/tutorials/index.md

View workflow job for this annotation

GitHub Actions / ✍️ Check spelling

Unknown word (facecolors)

Check warning on line 122 in neon/real-time-api/tutorials/index.md

View workflow job for this annotation

GitHub Actions / ✍️ Check spelling

Unknown word (edgecolors)
```
The output data would look as follows:
```
Expand Down Expand Up @@ -147,7 +147,7 @@

imu_sample = device.receive_imu_datum()

dt = datetime.fromtimestamp(imu_sample.timestamp_unix_seconds)

Check warning on line 150 in neon/real-time-api/tutorials/index.md

View workflow job for this annotation

GitHub Actions / ✍️ Check spelling

Unknown word (fromtimestamp)
print(f"This IMU sample was recorded at {dt}")

print(f"It contains the following data:")
Expand Down Expand Up @@ -175,7 +175,7 @@

## Camera Calibration

You can receive camera calibration parameters using the [`get_calibration`](https://pupil-labs-realtime-api.readthedocs.io/en/stable/api/simple.html#pupil_labs.realtime_api.simple.Device.get_calibration) method. Especially the scene camera matrix and distortion coefficients are useful for undistorting the scene video.

Check warning on line 178 in neon/real-time-api/tutorials/index.md

View workflow job for this annotation

GitHub Actions / ✍️ Check spelling

Unknown word (undistorting)

```python
from pupil_labs.realtime_api.simple import discover_one_device
Expand All @@ -199,12 +199,71 @@
print(calibration["left_distortion_coefficients"][0])
```

## Template Data
You can access the response data entered into the template questionnaire on the phone and also set those responses remotely.

Using the [`get_template`](https://pupil-labs-realtime-api.readthedocs.io/en/stable/api/simple.html#pupil_labs.realtime_api.simple.Device.get_template) method, you can receive the definition of the template containing all questions and sections.

```python
template = device.get_template()
fstring = "{i.id}\t{i.title}\t{i.widget_type} \t{i.choices}"

Check warning on line 209 in neon/real-time-api/tutorials/index.md

View workflow job for this annotation

GitHub Actions / ✍️ Check spelling

Unknown word (fstring)
print("\n".join(fstring.format(i=i) for i in template.items))

Check warning on line 210 in neon/real-time-api/tutorials/index.md

View workflow job for this annotation

GitHub Actions / ✍️ Check spelling

Unknown word (fstring)
```
```
e3b94cc7-dce4-4781-a818-f769574c31d2 Section 1 SECTION_HEADER []
a54e85aa-5474-42f8-90c0-19f40e9ca825 Question 1 TEXT []
59c01ff9-972e-42c7-8b6f-5ef4e11df5d8 Section 2 SECTION_HEADER []
3c7d620f-9f98-4556-92dd-b66df329999c Question 2 PARAGRAPH []
6169276c-91f4-4ef9-8e03-45759ff61477 Question 3 CHECKBOX_LIST ['a', 'b', 'c']
33059b82-63b7-4c4e-9bab-a27f7724bd1e Question 4 RADIO_LIST ['1', '2', '3']
```

Using the [`get_template_data`](https://pupil-labs-realtime-api.readthedocs.io/en/stable/api/simple.html#pupil_labs.realtime_api.simple.Device.get_template_data) method, you can receive the responses currently saved in the template.

```python
data = device.get_template_data()
print("\n".join(f"{k}\t{v}" for k, v in data.items()))
```
```
6169276c-91f4-4ef9-8e03-45759ff61477 ['a']
3c7d620f-9f98-4556-92dd-b66df329999c An example paragraph.
a54e85aa-5474-42f8-90c0-19f40e9ca825 An example short text.
33059b82-63b7-4c4e-9bab-a27f7724bd1e ['1']
```

Using the [`post_template_data`](https://pupil-labs-realtime-api.readthedocs.io/en/stable/api/simple.html#pupil_labs.realtime_api.simple.Device.post_template_data) method, you can set the template responses remotely.

```python
questionnaire = {
"6169276c-91f4-4ef9-8e03-45759ff61477": "b",
"3c7d620f-9f98-4556-92dd-b66df329999c": "A different paragraph.",
"a54e85aa-5474-42f8-90c0-19f40e9ca825": "Another short text.",
"33059b82-63b7-4c4e-9bab-a27f7724bd1e": "3",
}
device.post_template_data(questionnaire)
```

You can also retrieve individual questions by their ID using the [`get_question_by_id`](https://pupil-labs-realtime-api.readthedocs.io/en/stable/api/models.html#pupil_labs.realtime_api.models.Template.get_question_by_id) method and check the validity of a response using the [`validate_answer`]( https://pupil-labs-realtime-api.readthedocs.io/en/stable/api/models.html#pupil_labs.realtime_api.models.TemplateItem.validate_answer) method.

```python
question = template.get_question_by_id("6169276c-91f4-4ef9-8e03-45759ff61477")
question.validate_answer(["invalid_option"])
```
```
pupil_labs/realtime_api/models.py", line 346, in validate_answer
raise InvalidTemplateAnswersError(self, answers, errors)
pupil_labs.realtime_api.models.InvalidTemplateAnswersError: Question 3 (6169276c-91f4-4ef9-8e03-45759ff61477) validation errors:
location: ('6169276c-91f4-4ef9-8e03-45759ff61477', 0)
input: invalid_option
message: Value error, 'invalid_option' is not a valid choice from: ['a', 'b', 'c']
```

## Troubleshooting

If you are having trouble connecting to your Neon device via the real-time API, consider the following points:

1. Make sure the Neon Companion app and the device you are using to access the API are connected to the same local network.
1. For discovery the local network must allow MDNS and UDP traffic. In large public networks this may be prohibited for security reasons.

Check warning on line 266 in neon/real-time-api/tutorials/index.md

View workflow job for this annotation

GitHub Actions / ✍️ Check spelling

Unknown word (MDNS)

- You may still be able to connect to Neon using its IP address. You can find the IP address in the WiFi settings of the phone. Once you have it, you can connect like this:

Expand Down
Loading