Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add conversion script for conditioning sessions #11

Merged
merged 20 commits into from
Oct 29, 2024
Merged

Conversation

alessandratrapani
Copy link
Collaborator

@alessandratrapani alessandratrapani commented Oct 23, 2024

Stub NWBfile: https://catalystneuro-processing.s3.us-east-2.amazonaws.com/cai-lab-to-nwb/Ca_EEG3-4_NeutralExposure.nwb

Add conversion script for conditioning sessions

  • add stub conversion option for all data stream
  • add conversion script for one session
  • add conversion script for all sessions

extras

  • Replace motion correction util fun with a proper interface that add the MotionCorrection container
  • Add metadata from the paper (subject age, general ophys metadata)

@alessandratrapani alessandratrapani self-assigned this Oct 23, 2024
@alessandratrapani alessandratrapani changed the base branch from main to refactor_conversion_pipeline October 24, 2024 17:00
Base automatically changed from refactor_conversion_pipeline to main October 24, 2024 18:11
@alessandratrapani alessandratrapani changed the title Add conversion script for (non-offline) sessions Add conversion script for conditioning sessions Oct 28, 2024
@alessandratrapani alessandratrapani marked this pull request as ready for review October 28, 2024 15:48
Copy link
Collaborator

@h-mayorquin h-mayorquin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did the first reading . I will run this on my side and check your file to see if it works for me.

@@ -37,5 +37,22 @@ NWBFile:
- Zaki, Yosif
Subject:
species: Mus musculus
age: TBD # in ISO 8601, such as "P1W2D"
sex: TBD # One of M, F, U, or O
age: P12W/P15W # in ISO 8601, such as "P1W2D"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this from the paper?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but I will confirm it in the upcoming meeting. I need it defined to run the conversion

optical_channel:
- name: OpticalChannel
description: Green channel of the microscope.
emission_lambda: 513.0
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this generic for Miniscope?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope, it's related to the indicator they are using. Since there is no specified emission/excitation lambda wrote the FPbase values. I will also confirm this with them in the upcoming meeting

@@ -50,6 +61,10 @@ def add_to_nwbfile(self, nwbfile: NWBFile, **conversion_options) -> NWBFile:
edf_reader = read_raw_edf(input_fname=self.file_path, verbose=self.verbose)
data, times = edf_reader.get_data(picks=list(channels_dict.keys()), return_times=True)
data = data.astype("float32")
# TODO select the correct time range
if stub_test:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good idea.


# Add Imaging
folder_path = experiment_dir_path / session_id / date_str / time_str
miniscope_folder_path = get_miniscope_folder_path(folder_path)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, this might fail if thE meatdata is not correct but is more automatized.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I advice to nest the print statements within an if verbose.

@h-mayorquin
Copy link
Collaborator

I could not run this on my side. I am getting this error:

  File "/home/heberto/development/cai-lab-to-nwb/.venv/lib/python3.11/site-packages/neuroconv/tools/roiextractors/roiextractors.py", line 1754, in add_segmentation_to_nwbfile
    add_background_plane_segmentation_to_nwbfile(
  File "/home/heberto/development/cai-lab-to-nwb/.venv/lib/python3.11/site-packages/neuroconv/tools/roiextractors/roiextractors.py", line 1175, in add_background_plane_segmentation_to_nwbfile
    background_ids = segmentation_extractor.get_background_ids()
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/heberto/development/roiextractors/src/roiextractors/segmentationextractor.py", line 177, in get_background_ids
    return list(range(self.get_num_background_components()))
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'NoneType' object cannot be interpreted as an integer

It seems that there is a method missing on minian? Any ideas?

@alessandratrapani
Copy link
Collaborator Author

I could not run this on my side. I am getting this error:

  File "/home/heberto/development/cai-lab-to-nwb/.venv/lib/python3.11/site-packages/neuroconv/tools/roiextractors/roiextractors.py", line 1754, in add_segmentation_to_nwbfile
    add_background_plane_segmentation_to_nwbfile(
  File "/home/heberto/development/cai-lab-to-nwb/.venv/lib/python3.11/site-packages/neuroconv/tools/roiextractors/roiextractors.py", line 1175, in add_background_plane_segmentation_to_nwbfile
    background_ids = segmentation_extractor.get_background_ids()
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/heberto/development/roiextractors/src/roiextractors/segmentationextractor.py", line 177, in get_background_ids
    return list(range(self.get_num_background_components()))
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'NoneType' object cannot be interpreted as an integer

It seems that there is a method missing on minian? Any ideas?

Yeah, unfortunately it's on roiextractor side, I solved this small bug here: catalystneuro/roiextractors#378
But I forgot to pin to the roiextractor main

@h-mayorquin
Copy link
Collaborator

OK, this ran well and the file looks reasonable.

I would use f-strings instead of "".format() but that's me. Any reason you prefer format? I am merging this so we can focus on the next one.

@h-mayorquin h-mayorquin merged commit b1e98a4 into main Oct 29, 2024
@h-mayorquin h-mayorquin deleted the conversion_script branch October 29, 2024 22:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants