Skip to content

Commit

Permalink
Fix ups for final release.
Browse files Browse the repository at this point in the history
  • Loading branch information
@morriscb
morriscb committed Oct 18, 2023
1 parent a4c6066 commit 54fbd6a
Show file tree
Hide file tree
Showing 6 changed files with 136 additions and 24 deletions.
24 changes: 23 additions & 1 deletion allensdk/brain_observatory/behavior/behavior_project_cache/behavior_project_cache.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
from pathlib import Path
import pandas as pd
import numpy as np
import warnings

from allensdk.api.warehouse_cache.cache import Cache
from allensdk.brain_observatory.behavior.behavior_ophys_experiment import \
Expand All @@ -25,6 +26,10 @@
.project_cache_base import ProjectCacheBase


class UpdatedStimulusPresentationTableWarning(UserWarning):
pass


class VBOLimsCache(Cache):
"""
A class that ineherits from the warehouse Cache and provides
Expand Down Expand Up @@ -121,6 +126,23 @@ def __init__(
version=version,
cache=cache)

warnings.warn(
message="As of AllenSDK version 2.16.0, the latest Visual "
"Behavior Ophys data has be significantly updated from "
"previous releases. Specifically the user will need to "
"update all processing of the stimulus_presentations "
"tables. These tables now include multiple stimulus types "
"delineated by the columns `stimulus_block` and "
"`stimulus_block_name`. The previous are stored in the "
"block name containing 'change_detection' and can be "
"accessed in the pandas table thusly: "
"`stimulus_presentations["
"stimulus_presentations.stimulus_block_name.str.contains"
"('change_detection')]`",
category=UpdatedStimulusPresentationTableWarning
)


@classmethod
def cloud_api_class(cls):
return BehaviorProjectCloudApi
Expand Down Expand Up @@ -346,7 +368,7 @@ def get_natural_movie_template(self, n_workers=None) -> pd.DataFrame:
frames as presented to the mouse. The DataFrame is indexed with the
same frame index as shown in the stimulus presentation table.
The processing of the movie requires signicant processing and its
The processing of the movie requires significant processing and its
return size is very large so take care in requesting this data.
Parameters
Expand Down
2 changes: 1 addition & 1 deletion ...vatory/behavior/behavior_project_cache/project_apis/data_io/behavior_project_cloud_api.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ def sanitize_data_columns(


class BehaviorProjectCloudApi(BehaviorProjectBase, ProjectCloudApiBase):
MANIFEST_COMPATIBILITY = ["0.0.0", "2.0.0"]
MANIFEST_COMPATIBILITY = ["1.0.0", "2.0.0"]

def __init__(
self,
Expand Down
96 changes: 79 additions & 17 deletions allensdk/brain_observatory/behavior/behavior_session.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1072,8 +1072,10 @@ def stimulus_presentations(self) -> pd.DataFrame:
for a given duration, typically 250 ms) and whose columns are
presentation characteristics.
Adds trials_id to the stimulus table if the column is not already
present.
For VBO, the latest release adds additional
stimuli to the table divided into stimulus blocks. The block previously
released is the one whose stimulus block name starts with
"change_detection".
Returns
-------
Expand All @@ -1083,40 +1085,89 @@ def stimulus_presentations(self) -> pd.DataFrame:
and whose columns are presentation characteristics.
dataframe columns:
stimulus_block: (int)
Index of the the different stimuli presented to the mouse
in order.
stimulus_block_name [VBO only]: (int)
Name of the individual stimulus blocks describing briefly
what was shown to the mouse.
stimulus_presentations_id [index]: (int)
identifier for a stimulus presentation
(presentation of an image)
image_index [VBO only]: (int)
image index (0-7) for a given session,
corresponding to each image name
image_name: (string)
Name of the image presented to the mouse if applicable.
This is indexes in to the stimulus_templates object.
movie_frame_index [VBO only]: (int)
Index into the natural movie presented. Use
the method get_raw_natural_movie or if you the image
stretched and warped to the monitor use
get_natural_movie_template. (Warning this is a very compute
intensive process).
duration: (float)
duration of an image presentation (flash)
in seconds (stop_time - start_time). NaN if omitted
start_frame: (int)
image presentation start frame
end_frame: (float)
image presentation end frame
start_time: (float)
image presentation start time in seconds
end_time: (float)
image presentation end time in seconds
image_index: (int)
image index (0-7) for a given session,
corresponding to each image name
start_frame: (int)
image presentation start frame
end_frame: (int)
image presentation end frame
is_change: (bool)
True if the image stimulus has changed. <NA> if not
an image stimulus block.
is_image_novel: (bool)
True if the image stimulus is novel. <NA> if not
an image stimulus block.
omitted: (bool)
True if no image was shown for this stimulus
presentation
presentation. <NA> if not an image stimulus block.
movie_repeat [VBO only]: (int)
Number of times the movie has been repeated during the
natural movie block. -99 if not a movie block.
flashes_since_change: (bool)
Number of times the image has been flashed in between when
it has changed.
trials_id: (int)
Id to match to the table Index of the trials table.
If not present, it will be added.
active: (bool)
Stimulus block overlaps with the trials.
is_sham_change: (bool)
True if the stimulus overlaps with a catch trial.
stimulus_index [VBN only]: (int)
0 for gabor_20_deg_250ms, 1 for flash_250ms. -99 otherwise
position_x [VBN only]: (float)
X position of gabor grating when shown. NaN if not
applicable.
position_y [VBN only]: (float)
Y position of gabor grating when shown. NaN if not
applicable.
color [VBN only]: (float)
Value of the color int flash_250ms stimulus. Either -1 or
1. NaN if not applicable.
rewarded [VBN only]: (bool)
Merge of trials data. True if the mouse was rewarded during
the stimulus. <NA> if not applicable to stimulus block.
orientation [VBN only]: (float)
Orientation of the gabor grating in degrees.
temporal_frequency [VBN only]: (float)
Temporal_frequency of the gabor grating. NaN if not
applicable.
spatial_frequency [VBN only]: (float)
spatial_frequency of the gabor grating. NaN if not
applicable.
contrast [VBN only]: float
Contrast value of the gabor grating. NaN if not applicable.
"""
table = self._stimuli.presentations.value
table = table.drop(columns=["image_set", "index"], errors="ignore")
table = table.rename(columns={"stop_time": "end_time"})

if (
"trials_id" not in table.columns
and "stimulus_block" in table.columns
):
table["trials_id"] = compute_trials_id_for_stimulus(
table, self.trials
)
return table

@property
Expand Down Expand Up @@ -1279,6 +1330,17 @@ def trials(self) -> pd.DataFrame:
change_image_name: (string)
name of image that is changed to at the change time,
on go trials
change_time: (float)
Time in the session at which the change occurred.
response_latency [VBO only]: (float)
Delay between change and first lick in seconds. NaN if
no change occurred.
change_frame: (int)
Frame in the session at which the change occurred. -99 if
no change.
change_time_no_display_delay [VBN only]: (float)
Time of the change in seconds, before the display lag is
accounted for and applied.
"""
return self._trials.data

Expand Down
1 change: 1 addition & 0 deletions allensdk/brain_observatory/behavior/data_objects/stimuli/presentations.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@ def __init__(
[
"flashes_since_change",
"image_index",
"movie_repeat",
"movie_frame_index",
"repeat",
"stimulus_index",
Expand Down
6 changes: 4 additions & 2 deletions doc_template/visual_behavior_ophys_nwb.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ Below is a brief description and location for several types of data in the NWB f
| +----------------------------------------------------------------------------+--------------------------------------------------------------------+ |
| | Subject species | /general/subject/species | |
| +----------------------------------------------------------------------------+--------------------------------------------------------------------+ |
| | Subject identifier | /general/subject/subject_id | |
| | Subject identifier | /general/subject/subject_id | | |
+--------------------------+----------------------------------------------------------------------------+--------------------------------------------------------------------+ |
| Metadata (General) | Date of NWB file creation | /file_create_date | |
| +----------------------------------------------------------------------------+--------------------------------------------------------------------+ |
Expand All @@ -112,7 +112,9 @@ Below is a brief description and location for several types of data in the NWB f
| +----------------------------------------------------------------------------+--------------------------------------------------------------------+ |
| | Institution | /general/institution | |
| +----------------------------------------------------------------------------+--------------------------------------------------------------------+ |
| | Dataset keywords | /general/keywords | |
| | Dataset keywords | /general/keywords |
| +----------------------------------------------------------------------------+--------------------------------------------------------------------+ |
| | Allen project_code | /general/project_code |
| +----------------------------------------------------------------------------+--------------------------------------------------------------------+ |
| | Additional metadata | /general/metadata | |
| | | | |
Expand Down
31 changes: 28 additions & 3 deletions doc_template/visual_behavior_optical_physiology.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -310,6 +310,18 @@ DATA FILE CHANGELOG

**v1.1.0**

Removed data:
- Removed truncated behavior sessions with ids: 934610593, 958310218, 975358131, 1011688792
- Removed behavior with incorrect image presentations: 931566300
- This results in the removal several other subordinate data:
- ophys session removed: 931566300
- ophys experiments removed: 932372699, 932372701, 932372705, 932372707, 932372711
- The above identifiers have also been removed from their respective metadata
tables. Columns such as prior_exposures however, retain knowledge of these
sessions.
- The cell ROIs associated with the above experiments have also been removed
from the cell ROI metadata table.

Metadata Changes

- Better consistency of integer typing throughout.
Expand Down Expand Up @@ -338,10 +350,23 @@ NWB Data Changes
taken.
- Enforced better and more consistent typing between the metadata tables and
the session metadata.
- All datetimes in NWBs and metadata tables are now explicitly UTC timezone.
- All date/times in NWBs and metadata tables are now explicitly UTC timezone.
- Stimulus presentations tables now contain data from all stimuli not just
the active/passive image stimulus. These new stimuli are delineated by
stimulus_block and stimulus_block_name. The previous image behavior stimulus
is accessible as the block with a name containing "change_detection".
- New columns in the stimulus_presentations table:
- active, is_image_novel, is_sham_change, movie_frame_index, movie_repeat,
stimulus_block, stimulus_block_name, stimulus_name
- is_image_novel, is_sham_change, movie_frame_index, movie_repeat,
stimulus_block, stimulus_block_name, stimulus_name, active
- New trials columns:
- change_time, change_frame, response_latency

Supplemental cache data

- New accessors in VisualBehaviorOphysProjectCache to download and cache the natural
movie presented to the mice. Additional accessor to convert the movie to
the format as shown on the screen during the session. (Warning this
conversion is compute intensive).

**v1.0.1**

Expand Down

0 comments on commit 54fbd6a

Please sign in to comment.