Fixes and improvements (by mtnhuck) #13

chrisgorgo · 2018-01-13T02:17:31Z

No description provided.

scanner_software_versions_pd get from SoftwareVersions edited at line 165: dict_append(image03_dict, 'scanner_software_versions_pd', metadata.get("HardcopyDeviceSoftwareVersion", "")) to dict_append(image03_dict, 'scanner_software_versions_pd', metadata.get("SoftwareVersions", "")) image_slice_thickness SliceThickness Inserted at line 170 dict_append(image03_dict, 'image_slice_thickness', metadata.get("SliceThickness", ""))

Edited to include image_orientation image_orientation: modified from https://stackoverflow.com/questions/34782409/understanding-dicom-image-a ttributes-to-get-axial-coronal-sagittal-cuts and rordenlab/dcm2niix#153 (comment) photomet_interpret: from DICOM/JSON field PhotometricInterpretation

Added a notes section talking about required experiment_id field

chrisgorgo · 2018-01-13T02:20:01Z

@yarikoptic coudl you have a look?

yarikoptic

I have left some confirmations and questions to @mtnhuck so I could make an informed recommendation ;)

yarikoptic · 2018-01-13T15:03:53Z

README.md

+
+## Notes:
+Column experiment_id must be manually filled in for now.
+This is based on experiment ID's received from NDA after setting the study up through the NDA website [here](https://ndar.nih.gov/user/dashboard/collections.html).


am I reading it right @mtnhuck that it is pretty much yet another "mapping" but now for the task/runs? unfortunately that url requires a login so can't assess. Could you please give an example?

This is a number that comes from the NDA archive, ie they provide you with a number for your experiment. I'm still waiting to hear back from them if it is at the session level or the scan level (ie would resting-state need a different experiment number than a face-perception task?). An example number which I generated on their website is 867.

Ok, let's wait for them to reply and then make it an option or mapping

According to the .gov ppl, experiment_id is per "experiment within the study" so resting state would get one and face-perception would get another.

As agreed later on, this is the id for user to provide. We have a preliminary patch ready to submit (will be done after this PR is merged) to provide the mapping from BIDS task id to experiment_id via a simple .tsv file.

yarikoptic · 2018-01-13T15:07:51Z

bids2nda/main.py

@@ -162,11 +162,22 @@ def run(args):
        dict_append(image03_dict, 'image_modality', "MRI")
        dict_append(image03_dict, 'scanner_manufacturer_pd', metadata.get("Manufacturer", ""))
        dict_append(image03_dict, 'scanner_type_pd', metadata.get("ManufacturersModelName", ""))
-        dict_append(image03_dict, 'scanner_software_versions_pd', metadata.get("HardcopyDeviceSoftwareVersion", ""))
+        dict_append(image03_dict, 'scanner_software_versions_pd', metadata.get("SoftwareVersions", ""))


+1 -- I do not see the HardcopyDeviceSoftwareVersion present anywhere.
If they were interested in all "versions", then we could also extend it with ConversionSoftwareVersion but since they do not care -- "better" for us, worse for reproducibility ;)

I think this one is settled

yarikoptic · 2018-01-13T15:08:25Z

bids2nda/main.py

        dict_append(image03_dict, 'magnetic_field_strength', metadata.get("MagneticFieldStrength", ""))
        dict_append(image03_dict, 'mri_echo_time_pd', metadata.get("EchoTime", ""))
        dict_append(image03_dict, 'flip_angle', metadata.get("FlipAngle", ""))
        dict_append(image03_dict, 'receive_coil', metadata.get("ReceiveCoilName", ""))
+        dict_append(image03_dict, 'image_slice_thickness', metadata.get("SliceThickness", ""))


Should be ok, strange thing they do not define units anywhere in their schema

It looks like in the image03.txt they provide on their website they define acceptable attributed and mm is standard for most.

yarikoptic · 2018-01-13T15:08:56Z

bids2nda/main.py

        dict_append(image03_dict, 'magnetic_field_strength', metadata.get("MagneticFieldStrength", ""))
        dict_append(image03_dict, 'mri_echo_time_pd', metadata.get("EchoTime", ""))
        dict_append(image03_dict, 'flip_angle', metadata.get("FlipAngle", ""))
        dict_append(image03_dict, 'receive_coil', metadata.get("ReceiveCoilName", ""))
+        dict_append(image03_dict, 'image_slice_thickness', metadata.get("SliceThickness", ""))
+        dict_append(image03_dict, 'photomet_interpret', metadata.get("PhotometricInterpretation", ""))


Should be ok -- I think they do not define possible values so whatever we have might be as good

yarikoptic · 2018-01-13T15:12:31Z

bids2nda/main.py

+        elif plane[1] == 1:
+            dict_append(image03_dict, 'image_orientation.', "Coronal")
+        elif plane[2] == 1:
+            dict_append(image03_dict, 'image_orientation.', "Axial")


ha ha ;) I wondered if that is what you guessed or was informed logic (I didn't look into what they exactly mean yet)? ;)
If "logic" is informed/correct, then it might as well be just

dict_append( image03_dict, 'image_orientation.', ['Sagital', 'Coronal', 'Axial'][np.argmax(plane[:3])] )

i.e. simply go after maximal one. Not sure if it could happen that either two of them >= 0.5 resulting to rounding to 1 in more than one, or none (all round to 0). Going after maximum would remove the ambiguity and make code shorter/without duplication.

Also not sure if the sign couldn't be negative -- then may be also first wrap it with np.abs() (I think it shouldn't alter the logic of defining the slicing)

e.g. in your case:

sub-sid000447/ses-1/func/sub-sid000447_ses-1_task-selfvalence_run-03_bold.json: "ImageOrientationPatient": [0.99910127013954, -0.0366068325451, -0.0213680091858, 0.03960414059805, 0.98585875034027, 0.16283131279032], sub-sid000448/ses-1/anat/sub-sid000448_ses-1_acq-MPRAGE_T1w.json: "ImageOrientationPatient": [-0.004293648487, 0.95983065031276, 0.28054712135906, 0.05798604985148, 0.28031661899416, -0.9581545862429], sub-sid000448/ses-1/dwi/sub-sid000448_ses-1_acq-DTIXhardi64_dwi.json: "ImageOrientationPatient": [0.9983085405984, -0.0120986643356, 0.05686545603079, 0.00679536266945, 0.99568534650845, 0.09254465837924],

so -- was T1 coronal and the diffusion and bold Sagital? (I would think they were Axial, but I am not familiar with your protocols) Is it just a matter of swapping Axial and Sagital? ;)

yarikoptic · 2018-01-13T23:27:39Z

@chrisfilo please add me to the "Contributors" of a kind so I could push fixes to the PR

chrisgorgo · 2018-01-13T23:33:29Z

Done. Try now.

yarikoptic · 2018-01-14T03:19:04Z

bids2nda/main.py

+
+        plane = metadata.get("ImageOrientationPatient")
+        get_orientation = lambda place: ['Axial','Coronal','Sagittal'][np.argmax(plane[:3])]
+        dict_append(image03_dict, 'image_orientation.',get_orientation(plane))


after looking at this data -- I think this is not sufficient, and produces incorrect result. Let's wait for Matthew to reply on #12 (I will buzz him). FWIW here is my code which is bit more detailed but I think I am missing the point that it should be paired with affine or smth. nibabel/orientations.py actually is probably having the right function to use ;-)

def cosine_to_orientation(img_ornt_pat): """Deduce slicing from cosines From http://nipy.org/nibabel/dicom/dicom_orientation.html#dicom-voxel-to -patient-coordinate-system-mapping From Section C.7.6.1.1.1 we see that the "positive row axis" is left to right, and is the direction of the rows, given by the direction of last pixel in the first row from the first pixel in that row. Similarly the "positive column axis" is top to bottom and is the direction of the columns, given by the direction of the last pixel in the first column from the first pixel in that column. Let's rephrase: the first three values of "Image Orientation Patient" are the direction cosine for the "positive row axis". That is, they express the direction change in (x, y, z), in the DICOM patient coordinate system (DPCS), as you move along the row. That is, as you move from one column to the next. That is, as the column array index changes. Similarly, the second triplet of values of "Image Orientation Patient" (img_ornt_pat[3:] in Python), are the direction cosine for the "positive column axis", and express the direction you move, in the DPCS, as you move from row to row, and therefore as the row index changes. Parameters ---------- img_ornt_pat: list of float Values of the ImageOrientationPatient field Returns ------- {'Axial', 'Coronal', 'Sagital'} """ # we do not care about the signs img_ornt_pat = np.abs(img_ornt_pat) # we need to figure out first leading dimension and the 2nd d1 = int(np.argmax(img_ornt_pat[:3])) d2 = int(np.argmax(img_ornt_pat[3:])) ds = tuple(sorted([d1, d2])) try: return { (0, 1): 'Axial', (1, 2): 'Sagital', (0, 2): 'Coronal' }[ds] except KeyError: raise RuntimeError( "Could not deduce the image orientation of %r with both directions " "being detected as %r. Give us a use-case to fine tune" % (img_ornt_pat, ds) )

As I'm sure y'all know, the affine from the DICOM tells you the orientation of the image acquisition relative to the patient.

You get the affine from the ImageOrientationPatient, which defines two columns, and your interpretation of the third column. There are two options for the third column (assuming it's orthogonal to the other two) - the cross product of the two columns you have, or the negative cross product. If you have the per-slice positions, you can work out which of these applies - when you don't (for 3D images or greater), you've either got to fish these out of the private fields (Siemens Mosaic) or know what you're doing with the multislice DICOM format (I don't know that very well).

Once you've got the affine, you can work out the orientation, with the nibabel orientations code.

Thank you @matthew-brett!
Unfortunately I am not grasping the full complexity of the situation ATM and for now will just improve this PR with my logic.

I am taking that back! I found that my commit message said "(not correct)" so I guess I would indeed need to get a better grasp of the situation ;-) Will push another tentative solution in a second, which might be a better start (if not a finish ;)).

yarikoptic · 2018-01-14T03:42:17Z

tried to push into @mtnhuck 's branch but permissions denied. do not see checkmark to be able to accept pushes from collaborators -- do you see it @chrisfilo

here is my branch to be merged with few tuneups https://github.com/yarikoptic/BIDS2NDA/tree/enh-mtnhuck2 - I "removed" assignment of the slice_orientation for now since IMHO it is not fully correct yet (#12)

chrisgorgo · 2018-01-14T03:46:52Z

I elevated your permissions further. Could you try again?

(I don't see the checkmark, but that's because I made the PR)

Updated for SliceThickness and PhotometricInterpretation

… fieldmaps

make fixes

Shotgunosine · 2018-08-22T16:27:50Z

Hi all, just wondering what's going on with this PR.

mtnhuck · 2018-08-23T10:34:02Z

I believe that experiment_id is the experiment ID from your NIH dashboard. You group of studies, say for a particular grant, would have a collection ID and the experiment ID would from a specified set of subjects who performed a certain sequence of tasks during a scan session.

…

On Wed, Aug 22, 2018 at 4:14 PM Yaroslav Halchenko ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In README.md <#13 (comment)>: > @@ -32,3 +32,7 @@ The is the file format produced by the GUID Tool: one line per subject in the fo ## Example outputs See [/examples](/examples) + +## Notes: +Column experiment_id must be manually filled in for now. +This is based on experiment ID's received from NDA after setting the study up through the NDA website [here](https://ndar.nih.gov/user/dashboard/collections.html). According to the .gov ppl, experiment_id is per "experiment within the study" so resting state would get one and face-perception would get another. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#13 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGhtiDdtR8yVXKWlXzNgwtUizyJa6RhAks5uTbu6gaJpZM4RdEND> .

-- Jeremy Huckins Department of Psychological & Brain Sciences Dartmouth College

yarikoptic · 2019-06-21T18:41:48Z

bids2nda/main.py

@@ -194,6 +200,8 @@ def run(args):
        dict_append(image03_dict, 'image_resolution1', nii.header.get_zooms()[0])
        dict_append(image03_dict, 'image_resolution2', nii.header.get_zooms()[1])
        dict_append(image03_dict, 'image_resolution3', nii.header.get_zooms()[2])
+        dict_append(image03_dict, 'image_slice_thickness', nii.header.get_zooms()[2])
+        dict_append(image03_dict, 'photomet_interpret', metadata.get("global",{}).get("const",{}).get("PhotometricInterpretation",""))


This beast I am not sure even where to get from. In our use-case we just adjusted the code to provide 'NA' as a default value.

I pinged the NIH help desk for some guidance on the interpretation of this field. Will report back.

…consts section apparently it is not fully flattened by/after heudiconv

… (not correct!)

…2niix if available

* gh-mtnhuck/master: fix photomet and image_orientation additions Finally got it to pull PhotometricInterpretation but doesn't work for fieldmaps Update references Conflicts: bids2nda/main.py - melded together definition for image_slice_thickness so it picks it up from metadata field first and if not available, then via nibabel

codecov · 2019-06-24T02:21:43Z

Codecov Report

❗ No coverage uploaded for pull request base (master@2b264fb). Click here to learn what that means.
The diff coverage is 60.86%.

@@            Coverage Diff            @@
##             master      #13   +/-   ##
=========================================
  Coverage          ?   14.61%           
=========================================
  Files             ?        2           
  Lines             ?      219           
  Branches          ?        0           
=========================================
  Hits              ?       32           
  Misses            ?      187           
  Partials          ?        0

Impacted Files	Coverage Δ
bids2nda/tests/test_helpers.py	`100% <100%> (ø)`
bids2nda/main.py	`13.02% <52.63%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2b264fb...735eab0. Read the comment docs.

yarikoptic · 2019-06-24T02:22:45Z

ok, pushed some changes. couldn't stop so even enabled testing so my two lines of tests do not go untested ;) After we find some reasonably small dataset to test on, we could establish some crude smoke and even functionality testing (in subsequent PRs)

yarikoptic · 2019-06-26T15:53:05Z

ok, I think the world will not become a worse place (if not better) with these changes, so I will go ahead and merge

mtnhuck added 3 commits January 12, 2018 17:09

Slightly modified readme.md

4fa8695

Added a notes section talking about required experiment_id field

yarikoptic reviewed Jan 13, 2018

View reviewed changes

Edits to try to fix image_orientation

e19b5ed

yarikoptic reviewed Jan 14, 2018

View reviewed changes

BF: better to have TODO than an incorrect value for now

882effc

mtnhuck and others added 4 commits January 14, 2018 00:04

Update references

ef32caf

Updated for SliceThickness and PhotometricInterpretation

Finally got it to pull PhotometricInterpretation but doesn't work for…

80ee7ff

… fieldmaps

fix photomet and image_orientation additions

e4eb087

Merge pull request #1 from leej3/master

9a0dd89

make fixes

yarikoptic reviewed Jun 21, 2019

View reviewed changes

yarikoptic added 7 commits June 21, 2019 14:54

BF: acquire SliceThickness and PhotometricInterpretation from global.…

36a2695

…consts section apparently it is not fully flattened by/after heudiconv

ENH: figure out slice orientation from "ImageOrientationPatientDICOM"…

77e9bcf

… (not correct!)

RF: use a solution from SO for figuring out orientation

ce99948

ENH: prefer/use ImageOrientationPatientDICOM filled out by recent dcm…

e517ccc

…2niix if available

RF: move test under tests/test_helpers.py

dc8d63e

ENH: rudimentary .travis.yml to run the mighty single unittest

735eab0

yarikoptic mentioned this pull request Jun 24, 2019

Rough code adding some support for manifest #18

Open

yarikoptic merged commit 300fd08 into bids-standard:master Jun 26, 2019

ljchang mentioned this pull request Feb 25, 2022

Minor bug fixes #37

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fixes and improvements (by mtnhuck) #13

Fixes and improvements (by mtnhuck) #13

chrisgorgo commented Jan 13, 2018

chrisgorgo commented Jan 13, 2018

yarikoptic left a comment

yarikoptic Jan 13, 2018

mtnhuck Jan 13, 2018

yarikoptic Jan 13, 2018

yarikoptic Aug 22, 2018

yarikoptic Jun 21, 2019

yarikoptic Jan 13, 2018

yarikoptic Jun 21, 2019

yarikoptic Jan 13, 2018

mtnhuck Jan 13, 2018

yarikoptic Jan 13, 2018

yarikoptic Jan 13, 2018

yarikoptic Jan 13, 2018

yarikoptic Jan 13, 2018

yarikoptic commented Jan 13, 2018

chrisgorgo commented Jan 13, 2018

yarikoptic Jan 14, 2018

matthew-brett Jan 14, 2018

yarikoptic Jun 21, 2019

yarikoptic Jun 21, 2019

yarikoptic commented Jan 14, 2018

chrisgorgo commented Jan 14, 2018

Shotgunosine commented Aug 22, 2018 •

edited

Loading

mtnhuck commented Aug 23, 2018 via email

yarikoptic Jun 21, 2019

Shotgunosine Jun 21, 2019

yarikoptic Jun 24, 2019

codecov bot commented Jun 24, 2019 •

edited

Loading

yarikoptic commented Jun 24, 2019

yarikoptic commented Jun 26, 2019

Fixes and improvements (by mtnhuck) #13

Fixes and improvements (by mtnhuck) #13

Conversation

chrisgorgo commented Jan 13, 2018

chrisgorgo commented Jan 13, 2018

yarikoptic left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yarikoptic commented Jan 13, 2018

chrisgorgo commented Jan 13, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yarikoptic commented Jan 14, 2018

chrisgorgo commented Jan 14, 2018

Shotgunosine commented Aug 22, 2018 • edited Loading

mtnhuck commented Aug 23, 2018 via email

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Jun 24, 2019 • edited Loading

Codecov Report

yarikoptic commented Jun 24, 2019

yarikoptic commented Jun 26, 2019

Shotgunosine commented Aug 22, 2018 •

edited

Loading

codecov bot commented Jun 24, 2019 •

edited

Loading