Create a git-annex repo for data from beijing #14

plbenveniste · 2024-05-28T14:43:03Z

The data is stored in duke/mri/beijing/MS-NMO科研序列纵向数据梳理-仇小路.
The steps are the following:

A git-annex repo should be created :
The data should be Bidsfied
The data should be pushed to the repo
A PR should be opened
The PR should be merged.

@jcohenadad What name should be used for the git-annex repo ? beijing-ms-nmo-qu-xiaolu ?

@mguaypaq Could you create the corresponding git-annex repo ?

The text was updated successfully, but these errors were encountered:

jcohenadad · 2024-05-28T15:16:21Z

What name should be used for the git-annex repo ? beijing-ms-nmo-qu-xiaolu ?

ms-nmo-beijing

mguaypaq · 2024-06-11T15:11:49Z

I created the repo and gave @plbenveniste write access:
https://data.neuro.polymtl.ca/datasets/ms-nmo-beijing

plbenveniste · 2024-06-11T17:41:50Z

Currently working on building a bash script and a config file for converting the data from DICOM to BIDS.

To run the script, do the following:

ms-nmo-beijing-bids/code/convert_dcm2bids.sh ms-nmo-beijing-bids/code/dcm2bids_config.json ms-nmo-beijing-bids/ ~/duke/mri/beijing/MS-NMO科研序列纵向数据梳理-仇小路/MS纵向科研序列-影像原始数据/

Because the file conventions are pretty variable, the .json is still under construction and starting to be pretty long.

plbenveniste · 2024-06-13T17:44:37Z

All done with MS subjects.

Only encountered issues with MS52-王军 because it contained a folder called Phoenix Document which couldn't be converted using dcm2bids. The folder was removed and then the following line of code was ran:

dcm2bids -d MS52-王军/  -p MS52  -s 20160404 -c ms-nmo-beijing-bids/code/dcm2bids_config.json -o Archive_MS52/  --bids_validate

Moving on to NMO patietnts

plbenveniste · 2024-06-14T19:34:11Z

When working on the NMO participants, I saw that some sessions didn't work through the script:

21-LIMEI-20150509
49-LULULU-20151027

The error was the same as the one in the comment above:

subprocess.CalledProcessError: Command '['dcm2niix', '-b', 'y', '-ba', 'y', '-z', 'y', '-f', '%3s_%f_%p_%t', '-o', PosixPath('temp_21_limei_20150509/tmp_dcm2bids/sub-NMO21_ses-20150509'), PosixPath('21-LIMEI-20150509')]' returned non-zero exit status 8.

For 21-LIMEI-20150509: The problem came from the folder ep2d_DTI_A1_D30_S2.0_80 which was therefore removed and discarded.
For 49-LULULU-20151027: The problem came from the fact that everything was stored in a subfolder. All good.

These two were done separately and are currently stored under : p119007/create_beijing_gitannex/temp_21_limei_20150509 and create_beijing_gitannex/temp_49_lululu_20151027.

Waiting for the script to end to gather everything and then commit and push to git-annex.

plbenveniste · 2024-06-17T17:20:29Z

All files were pushed to git-annex.

Last thing to do is to:

fill the participants.json and participants.tsv files.

plbenveniste · 2024-06-17T18:49:47Z

The files participants.tsv, participants.json and dataset_description.json were updates using the following command:

python code/fill_participants_data.py --participants-tsv participants.tsv --participants-json participants.json --input-data sourcedata/MS-NMO纵向数据-小路整理.xlsx --dataset-description-json dataset_description.json

I also pushed to raw xlsx data in the sourcedata folder. I want your opinions on this since it contains sensitive information (name, phone number...)

PR was created and ready for review.

plbenveniste · 2024-07-10T00:19:54Z

@jcohenadad You might have missed these questions I asked in the PR 1. Could you answer these 3 questions ?

A few questions:
Should we keep the file in sourcedata which contains sensitive information regarding the participants ?
Also, the participants.tsv contains clinical information in mandarin which currently cannot be translated without using an online API. The informations is stored for now in mandarin since it is not used for now. Is it okay to leave it like this for now?
Also, what is the dataset contact name and email?

jcohenadad · 2024-07-10T02:22:10Z

@jcohenadad You might have missed these questions I asked in the PR 1. Could you answer these 3 questions ?

answered-- thank you for the reminder

plbenveniste · 2024-07-10T16:21:29Z

Modifications done! Ready for review now!

plbenveniste · 2024-07-22T17:18:31Z

Working on updating the files suffixes using the json sidecars. Here are the possible combinations fron jjson sidecar (first 3 columns) and the chosen suffixed (two last columns):

Columns: (SeriesDescription, ScanningSequence, SequenceName) : [suffix-acq, suffix-contrast]

{   
    "('t1_mpr_ss_sag_spine', 'GR\\\\IR', '*tfl3d1')": ["acq-sagSpine", "MPRAGE"],
    "('localizer_haste_auto_T', 'SE', '*h2d1_154')": ["acq-localizer", "T2w"],
    "('t2_tse_rst_tra', 'SE', '*tseR2d1rr23')": ["acq-axTseRst", "T2w"],
    "('t2_tse_cor_fs', 'SE', '*tse2d1_15')": ["acq-corTse", "T2w"],
    "('t2_Double_IR_spc_sag_p2_iso', 'SE\\\\IR', 'spcir3d1_218ns')": ["acq-isoDoubleIR", "T2w"],
    "('localizer_haste_auto_C', 'SE', '*h2d1_154')": ["acq-localizer", "T2w"],
    "('t2_tse_rst_tra_384-P2', 'SE', '*tseR2d1rr27')": ["acq-axTseRst", "T2w"],
    "('t2_tse_rst_sag', 'SE', '*tseR2d1rr31')": ["acq-sagTseRst", "T2w"],
    "('dual echo Axial PD-T2 TSE', 'SE', '*tse2d2_7')": ["acq-axTse", "PDw"],
    "('t1_mpr_ns_sag_iso', 'GR\\\\IR', '*tfl3d1_ns')": ["acq-iso", "MPRAGE"],
    "('t2_tirm_SAG_dark-fluid_p2', 'SE\\\\IR', '*tir2d1_25')": ["acq-sagTirmDF", "T2w"],
    "('t2_tse_rst_sag_512', 'SE', '*tseR2d1rr29')": ["acq-sagTseRst", "T2w"],
    "('localizer_haste', 'SE', '*h2d1_154')": ["acq-localizer", "T2w"],
    "('localizer_haste_T', 'SE', '*h2d1_154')": ["acq-localizer", "T2w"],
    "('localizer_haste_auto', 'SE', '*h2d1_154')": ["acq-localizer", "T2w"],
    "('PosDisp: [26] t2_tse_rst_tra ', 'SE', '*h2d1_154')": ["acq-axTseRstLocalizer", "T2w"],
    "('PosDisp: [26] t2_tse_rst_tra ', 'SE', '*tseR2d1rr31')": ["acq-axTseRstLocalizer", "T2w"],
    "('t1_fl2d_tra_FIL', 'GR', '*fl2d1')": ["acq-ax", "T1w"],
    "('t1_tse_sag', 'SE', '*tse2d1_5')": ["acq-sagTse", "T1w"],
    "('t2_tirm_tra_dark-fluid_p2_FIL', 'SE\\\\IR', '*tir2d1_25')": ["acq-axTirmDF", "T2w"],
    "('t2_tse_rst_sag', 'SE', '*tseR2d1rr21')": ["acq-sagTseRst", "T2w"],
    "('t2_tse_rst_tra', 'SE', '*tseR2d1rs17')": ["acq-axTseRst", "T2w"],
    "('SWI_Images', 'GR', '*swi3d1r')": ["acq-img", "swi"],
    "('Mag_Images', 'GR', '*swi3d1r')": ["acq-mag", "swi"],
    "('t1_tse_tra', 'SE', '*tse2d1_5')": ["acq-axTse", "T1w"],
    "('t2_tse_rst_sag_S4_DIS2D', 'RM', '*tseR2d1rr21')": ["acq-sagTseRst", "T2w"],
    "('t2_tse_rst_sag_S2_DIS2D', 'RM', '*tseR2d1rr21')": ["acq-sagTseRst", "T2w"],
    "('SPINE', 'RM', '*tseR2d1rr21')": ["acq-localizer", "T2w"],
    "('t1_fl2d_sag_320_FIL', 'GR', '*fl2d1')": ["acq-sag", "T1w"],
    "('t2_tse_tra_FIL', 'SE', '*tse2d1_21')": ["acq-axTse", "T2w"],
    "('mIP_Images(SW)', 'GR', '*swi3d1r')": ["acq-mip", "swi"],
    "('Pha_Images', 'GR', '*swi3d1r')": ["acq-pha", "swi"],
    "('localizer_haste', 'SE', '*h2d1_148')": ["acq-localizer", "T2w"],
    "('memprage_wip900', 'GR\\\\IR', 'tfl3d4_ns')": ["", "MPRAGE"],
    "('T1SAG+C', 'SE', '*tse2d1_4')": ["acq-sag", "T1w"],
    "('Inline_Morpho_Results', '', '')": ["", ""],
    "('T1SAG', 'SE', '*tse2d1_4')": ["acq-sag", "T1w"],
    "('T2SAG', 'SE', '*tseR2d1rr21')": ["acq-sag", "T2w"],
    "('t2_tse_sag_384', 'SE', '*tseR2d1rr21')": ["acq-sagTse", "T2w"],
    "('t1_tse_sag_384', 'SE', '*tse2d1_4')": ["acq-sagTse", "T1w"],
    "('memprage_wip900_combEcho_LabelIm', 'GR\\\\IR', 'tfl3d4_ns')": ["acq-combEchoLabelIm", "MPRAGE"],
    "('memprage_wip900_combEcho_SkullStrip', 'GR\\\\IR', 'tfl3d4_ns')": ["acq-combEchoSkullStrip", "MPRAGE"],
    "('memprage_wip900_combEcho', 'GR\\\\IR', 'tfl3d4_ns')": ["acq-combEcho", "MPRAGE"],
    "('t2_tse_rst_sag', 'SE', '*tseR2d1rr33')": ["acq-sagTseRst", "T2w"],
    "('t2_tirm_tra_dark-fluid_p2', 'SE\\\\IR', '*tir2d1_25')": ["acq-axTirmDF", "T2w"],
    "('<MPR Thick Range>', 'GR\\\\IR', '*tfl3d1')": ["acq-thick", "MPRAGE"],
    "('PosDisp: [25] t2_tse_rst_tra ', 'SE', '*tseR2d1rr31')": ["acq-axTseRst", "T2w"],
    "('t1_fl3d_TRA_fs', 'GR', '*fl3d1rr')": ["acq-ax", "T1w"],
    "('t1_tse_sag_384', 'SE', '*tse2d1_3')": ["acq-sagTse", "T1w"],
    "('C-t1_tse_sag_384p2', 'SE', '*tse2d1_3')": ["acq-sagTse", "T1w"],
    "('C-t1_tse_tra', 'SE', '*tse2d1_3')": ["acq-axTse", "T1w"],
    "('t1_tse_sag', 'SE', '*tse2d1_3')": ["acq-sagTse", "T2w"],
    "('t2_tse_rst_tra_384', 'SE', '*tseR2d1rr31')": ["acq-axTseRst", "T2w"],
    "('t2_tse_tra', 'SE', '*tse2d1_21')": ["acq-axTse", "T2w"],
    "('t1_fl2d_sag_ep', 'GR', '*fl2d1')": ["acq-sag", "T1w"],
    "('t1_fl2d_tra', 'GR', '*fl2d1')": ["acq-ax", "T1w"],
    "('<MPR Thick Range>', 'GR\\\\IR', '*tfl3d1_ns')": ["acq-thick", "MPRAGE"],
    "('PosDisp: [18] t1_fl2d_sag_ep ', 'SE', '*h2d1_154')": ["acq-sagLocalizer", "T2w"],
    "('PosDisp: [12] t2_tse_rst_tra ', 'SE', '*h2d1_154')": ["acq-axTseRstLocalizer", "T2w"],
    "('t2_tse_rst_sag', 'SE', '*tseR2d1rr19')": ["acq-sagTseRst", "T2w"],
    "('t2_tse_rst_tra', 'SE', '*tseR2d1rr27')": ["acq-axTseRst", "T2w"],
    "('t2_tse_rst_tra_384', 'SE', '*tseR2d1rr27')": ["acq-axTseRst", "T2w"],
    "('t2_tse_tra', 'SE', '*tse2d1_15')": ["acq-axTse", "T2w"],
    "('t2_tse_rst_sag_FIL', 'SE', '*tseR2d1rr33')": ["acq-sagTseRst", "T2w"],
    "('<MPR Range>', 'GR\\\\IR', '*tfl3d1_ns')": ["", "MPRAGE"],
    "('<MPR Collection>', 'GR\\\\IR', '*tfl3d1_ns')": ["", "MPRAGE"],
    "('localizer', 'GR', '*fl2d1')": ["acq-localizer", "T2star"],
    "('t2_tse_tra_320_p2-19', 'SE', '*tseR2d1rs20')": ["acq-axTse", "T2w"],
    "('PosDisp: [11] t2_tse_rst_tra', 'SE', '*h2d1_154')": ["acq-axTseRst", "T2w"],
    "('PosDisp: [22] t1_fl2d_tra_FIL', 'SE', '*h2d1_148')": ["acq-ax", "T2w"],
    "('t1_tse_tra', 'SE', '*tse2d1_3')": ["acq-axTse", "T1w"],
    "('T-t1_tse_sag_384_P2', 'SE', '*tse2d1_5')": ["acq-sagTse", "T1w"],
    "('t2_tse_rst_tra_384_FIL', 'SE', '*tseR2d1rr31')": ["acq-axTseRst", "T2w"],
    "('t1_tse_tra_FIL', 'SE', '*tse2d1_5')": ["acq-axTse", "T1w"],
    "('PosDisp: [12] t2_tse_rst_tra', 'SE', '*h2d1_154')": ["acq-axTseRst", "T2w"]
    
}

@jcohenadad What is your opinion on this?

Adding this text output file which could be usefull to find files with the above specified combinations:
output.txt

plbenveniste · 2024-07-23T17:50:07Z

Here are the notes from the discussion regarding how to chose the appropriate suffixes for each acquisition combination:

RST = Restore Pulse
FS = Fat suppressed

Here are the ones that should be deleted:

('SPINE', 'RM', '*tseR2d1rr21') : there is only one file: it's one slice of the SC → to be removed
('Inline_Morpho_Results', '', '') : these files are not images → to be removed
('memprage_wip900_combEcho_LabelIm', 'GR\\IR', 'tfl3d4_ns') : it's a segmentation of tge brain → to be removed
('memprage_wip900_combEcho_SkullStrip', 'GR\\IR', 'tfl3d4_ns') : it's a crop of the brain → to be removed

Combinations of which we were sure:

'\<MPR Thick Range\>': there are only two associated files:

→ The suffix chosen are : ["acq-sag", "T1w"]

('<MPR Range>', 'GR\\\\IR', '*tfl3d1_ns') : only one file

→ The suffix chosen are : ["acq-sag", T1w]

('', 'GR\\IR', '*tfl3d1_ns') : only one file (and only one slice inside

→ To be removed

('PosDisp: [11] t2_tse_rst_tra', 'SE', '*h2d1_154') : there are only two files:

→ The suffix chosen is : ["acq-axTseRstLocalizer", "T2w"]

('PosDisp: [22] t1_fl2d_tra_FIL', 'SE', '*h2d1_148') : there are only 3 files

→ Not useful: to be removed

Here is the end json file:

{   
    "('t1_mpr_ss_sag_spine', 'GR\\\\IR', '*tfl3d1')": ["", "T1w"],
    "('localizer_haste_auto_T', 'SE', '*h2d1_154')": ["acq-localizer", "T2w"],
    "('t2_tse_rst_tra', 'SE', '*tseR2d1rr23')": ["acq-axTseRst", "T2w"],
    "('t2_tse_cor_fs', 'SE', '*tse2d1_15')": ["acq-corTse", "T2w"],
    "('t2_Double_IR_spc_sag_p2_iso', 'SE\\\\IR', 'spcir3d1_218ns')": ["acq-isoDir", "T2w"], 
    "('localizer_haste_auto_C', 'SE', '*h2d1_154')": ["acq-localizer", "T2w"],
    "('t2_tse_rst_tra_384-P2', 'SE', '*tseR2d1rr27')": ["acq-axTseRst", "T2w"], 
    "('t2_tse_rst_sag', 'SE', '*tseR2d1rr31')": ["acq-sagTseRst", "T2w"], 
    "('dual echo Axial PD-T2 TSE', 'SE', '*tse2d2_7')": ["acq-axTse", "T2w"], 
    "('t1_mpr_ns_sag_iso', 'GR\\\\IR', '*tfl3d1_ns')": ["", "T1w"],
    "('t2_tirm_SAG_dark-fluid_p2', 'SE\\\\IR', '*tir2d1_25')": ["acq-sagTirmDf", "T2w"], 
    "('t2_tse_rst_sag_512', 'SE', '*tseR2d1rr29')": ["acq-sagTseRst", "T2w"], 
    "('localizer_haste', 'SE', '*h2d1_154')": ["acq-localizer", "T2w"], 
    "('localizer_haste_T', 'SE', '*h2d1_154')": ["acq-localizer", "T2w"],
    "('localizer_haste_auto', 'SE', '*h2d1_154')": ["acq-localizer", "T2w"],
    "('PosDisp: [26] t2_tse_rst_tra ', 'SE', '*h2d1_154')": ["acq-axTseRstLocalizer", "T2w"],
    "('PosDisp: [26] t2_tse_rst_tra ', 'SE', '*tseR2d1rr31')": ["acq-axTseRstLocalizer", "T2w"],
    "('t1_fl2d_tra_FIL', 'GR', '*fl2d1')": ["acq-ax", "T1w"],
    "('t1_tse_sag', 'SE', '*tse2d1_5')": ["acq-sagTse", "T1w"],
    "('t2_tirm_tra_dark-fluid_p2_FIL', 'SE\\\\IR', '*tir2d1_25')": ["acq-axTirmDf", "T2w"],
    "('t2_tse_rst_sag', 'SE', '*tseR2d1rr21')": ["acq-sagTseRst", "T2w"],
    "('t2_tse_rst_tra', 'SE', '*tseR2d1rs17')": ["acq-axTseRst", "T2w"],
    "('SWI_Images', 'GR', '*swi3d1r')": ["", "swi"], 
    "('Mag_Images', 'GR', '*swi3d1r')": ["part-mag", "GRE"],
    "('t1_tse_tra', 'SE', '*tse2d1_5')": ["acq-axTse", "T1w"],
    "('t2_tse_rst_sag_S4_DIS2D', 'RM', '*tseR2d1rr21')": ["acq-sagTseRst", "T2w"],
    "('t2_tse_rst_sag_S2_DIS2D', 'RM', '*tseR2d1rr21')": ["acq-sagTseRst", "T2w"],
    "('SPINE', 'RM', '*tseR2d1rr21')": ["acq-localizer", "T2w"], # TO BE REMOVED
    "('t2_tse_tra_FIL', 'SE', '*tse2d1_21')": ["acq-axTse", "T2w"],
    "('mIP_Images(SW)', 'GR', '*swi3d1r')": ["", "minIP"],
    "('Pha_Images', 'GR', '*swi3d1r')": ["part-phase", "GRE"],
    "('localizer_haste', 'SE', '*h2d1_148')": ["acq-localizer", "T2w"],
    "('memprage_wip900', 'GR\\\\IR', 'tfl3d4_ns')": ["acq-memprage", "T1w"],
    "('T1SAG+C', 'SE', '*tse2d1_4')": ["acq-sag", "T1w"],
    "('Inline_Morpho_Results', '', '')": ["", ""], # TO BE REMOVED
    "('T1SAG', 'SE', '*tse2d1_4')": ["acq-sag", "T1w"], 
    "('T2SAG', 'SE', '*tseR2d1rr21')": ["acq-sag", "T2w"],
    "('t2_tse_sag_384', 'SE', '*tseR2d1rr21')": ["acq-sagTse", "T2w"],
    "('t1_tse_sag_384', 'SE', '*tse2d1_4')": ["acq-sagTse", "T1w"],
    "('memprage_wip900_combEcho_LabelIm', 'GR\\\\IR', 'tfl3d4_ns')": ["acq-combEchoLabelIm", "T1w"], # TO BE REMOVED
    "('memprage_wip900_combEcho_SkullStrip', 'GR\\\\IR', 'tfl3d4_ns')": ["acq-combEchoSkullStrip", "T1w"], # TO BE REMOVED
    "('memprage_wip900_combEcho', 'GR\\\\IR', 'tfl3d4_ns')": ["acq-memprage", "T1w"],
    "('t2_tse_rst_sag', 'SE', '*tseR2d1rr33')": ["acq-sagTseRst", "T2w"],
    "('t2_tirm_tra_dark-fluid_p2', 'SE\\\\IR', '*tir2d1_25')": ["acq-axTirmDf", "T2w"],
    "('<MPR Thick Range>', 'GR\\\\IR', '*tfl3d1')": ["acq-sag", "T1w"], # CHECKED
    "('PosDisp: [25] t2_tse_rst_tra ', 'SE', '*tseR2d1rr31')": ["acq-axTseRstLocalizer", "T2w"], 
    "('t1_fl3d_TRA_fs', 'GR', '*fl3d1rr')": ["acq-ax", "T1w"], 
    "('t1_tse_sag_384', 'SE', '*tse2d1_3')": ["acq-sagTse", "T1w"], 
    "('C-t1_tse_sag_384p2', 'SE', '*tse2d1_3')": ["acq-sagTse", "T1w"], 
    "('C-t1_tse_tra', 'SE', '*tse2d1_3')": ["acq-axTse", "T1w"], 
    "('t1_tse_sag', 'SE', '*tse2d1_3')": ["acq-sagTse", "T2w"],
    "('t2_tse_rst_tra_384', 'SE', '*tseR2d1rr31')": ["acq-axTseRst", "T2w"],
    "('t2_tse_tra', 'SE', '*tse2d1_21')": ["acq-axTse", "T2w"],
    "('t1_fl2d_sag_ep', 'GR', '*fl2d1')": ["acq-sag", "T1w"],
    "('t1_fl2d_tra', 'GR', '*fl2d1')": ["acq-ax", "T1w"],
    "('<MPR Thick Range>', 'GR\\\\IR', '*tfl3d1_ns')": ["acq-sag", "T1w"], # CHECKED
    "('PosDisp: [18] t1_fl2d_sag_ep ', 'SE', '*h2d1_154')": ["acq-sagLocalizer", "T1w"], 
    "('PosDisp: [12] t2_tse_rst_tra ', 'SE', '*h2d1_154')": ["acq-axTseRstLocalizer", "T2w"],
    "('t2_tse_rst_sag', 'SE', '*tseR2d1rr19')": ["acq-sagTseRst", "T2w"],
    "('t2_tse_rst_tra', 'SE', '*tseR2d1rr27')": ["acq-axTseRst", "T2w"],
    "('t2_tse_rst_tra_384', 'SE', '*tseR2d1rr27')": ["acq-axTseRst", "T2w"],
    "('t2_tse_tra', 'SE', '*tse2d1_15')": ["acq-axTse", "T2w"],
    "('t2_tse_rst_sag_FIL', 'SE', '*tseR2d1rr33')": ["acq-sagTseRst", "T2w"], 
    "('<MPR Range>', 'GR\\\\IR', '*tfl3d1_ns')": ["acq-sag", "T1w"], # CHECKED
    "('<MPR Collection>', 'GR\\\\IR', '*tfl3d1_ns')": ["", "T1w"], # TO BE REMOVED
    "('localizer', 'GR', '*fl2d1')": ["acq-localizer", "T1w"], 
    "('t2_tse_tra_320_p2-19', 'SE', '*tseR2d1rs20')": ["acq-axTse", "T2w"], 
    "('PosDisp: [11] t2_tse_rst_tra', 'SE', '*h2d1_154')": ["acq-axTseRstLocalizer", "T2w"], # CHECKED
    "('PosDisp: [22] t1_fl2d_tra_FIL', 'SE', '*h2d1_148')": ["acq-axLocalizer", "T1w"], # TO BE REMOVED
    "('t1_tse_tra', 'SE', '*tse2d1_3')": ["acq-axTse", "T1w"], 
    "('T-t1_tse_sag_384_P2', 'SE', '*tse2d1_5')": ["acq-sagTse", "T1w"],
    "('t2_tse_rst_tra_384_FIL', 'SE', '*tseR2d1rr31')": ["acq-axTseRst", "T2w"],
    "('t1_tse_tra_FIL', 'SE', '*tse2d1_5')": ["acq-axTse", "T1w"],
    "('PosDisp: [12] t2_tse_rst_tra', 'SE', '*h2d1_154')": ["acq-axTseRstLocalizer", "T2w"] 
    
}

@jcohenadad Can you confirm the last remaining suffixes chosen above?

Tagging @Nilser3 and @NathanMolinier which were part of the conversation

jcohenadad · 2024-07-23T18:53:11Z

all good-- thank you so much for this extensive data curation @plbenveniste 🙏

plbenveniste · 2024-07-24T15:45:05Z

Referencing this comment on further discussion on suffix choice: neuropoly/data-management#63 (comment)

plbenveniste · 2024-08-12T19:40:56Z

Here I am modifying the suffixed based on this comment: neuropoly/data-management#63 (comment)

{   
    "('t1_mpr_ss_sag_spine', 'GR\\\\IR', '*tfl3d1')": ["", "T1w"],
    "('localizer_haste_auto_T', 'SE', '*h2d1_154')": ["acq-localizer", "T2w"],
    "('t2_tse_rst_tra', 'SE', '*tseR2d1rr23')": ["acq-axTseRst", "T2w"],
    "('t2_tse_cor_fs', 'SE', '*tse2d1_15')": ["acq-corTse", "T2w"],
    "('t2_Double_IR_spc_sag_p2_iso', 'SE\\\\IR', 'spcir3d1_218ns')": ["acq-iso", "FLAIR"],  # MODIFIED
    "('localizer_haste_auto_C', 'SE', '*h2d1_154')": ["acq-localizer", "T2w"],
    "('t2_tse_rst_tra_384-P2', 'SE', '*tseR2d1rr27')": ["acq-axTseRst", "T2w"], 
    "('t2_tse_rst_sag', 'SE', '*tseR2d1rr31')": ["acq-sagTseRst", "T2w"], 
    "('dual echo Axial PD-T2 TSE', 'SE', '*tse2d2_7')": ["acq-axTse", "T2w"], 
    "('t1_mpr_ns_sag_iso', 'GR\\\\IR', '*tfl3d1_ns')": ["", "T1w"],
    "('t2_tirm_SAG_dark-fluid_p2', 'SE\\\\IR', '*tir2d1_25')": ["acq-sagTirmDf", "T2w"], 
    "('t2_tse_rst_sag_512', 'SE', '*tseR2d1rr29')": ["acq-sagTseRst", "T2w"], 
    "('localizer_haste', 'SE', '*h2d1_154')": ["acq-localizer", "T2w"], 
    "('localizer_haste_T', 'SE', '*h2d1_154')": ["acq-localizer", "T2w"],
    "('localizer_haste_auto', 'SE', '*h2d1_154')": ["acq-localizer", "T2w"],
    "('PosDisp: [26] t2_tse_rst_tra ', 'SE', '*h2d1_154')": ["acq-axTseRstLocalizer", "T2w"],
    "('PosDisp: [26] t2_tse_rst_tra ', 'SE', '*tseR2d1rr31')": ["acq-axTseRstLocalizer", "T2w"],
    "('t1_fl2d_tra_FIL', 'GR', '*fl2d1')": ["acq-ax", "T1w"],
    "('t1_tse_sag', 'SE', '*tse2d1_5')": ["acq-sagTse", "T1w"],
    "('t2_tirm_tra_dark-fluid_p2_FIL', 'SE\\\\IR', '*tir2d1_25')": ["acq-axTirmDf", "T2w"],
    "('t2_tse_rst_sag', 'SE', '*tseR2d1rr21')": ["acq-sagTseRst", "T2w"],
    "('t2_tse_rst_tra', 'SE', '*tseR2d1rs17')": ["acq-axTseRst", "T2w"],
    "('SWI_Images', 'GR', '*swi3d1r')": ["", "swi"], 
    "('Mag_Images', 'GR', '*swi3d1r')": ["part-mag", "GRE"],
    "('t1_tse_tra', 'SE', '*tse2d1_5')": ["acq-axTse", "T1w"],
    "('t2_tse_rst_sag_S4_DIS2D', 'RM', '*tseR2d1rr21')": ["acq-sagTseRst", "T2w"],
    "('t2_tse_rst_sag_S2_DIS2D', 'RM', '*tseR2d1rr21')": ["acq-sagTseRst", "T2w"],
    "('SPINE', 'RM', '*tseR2d1rr21')": ["acq-localizer", "T2w"], # TO BE REMOVED
    "('t2_tse_tra_FIL', 'SE', '*tse2d1_21')": ["acq-axTse", "T2w"],
    "('mIP_Images(SW)', 'GR', '*swi3d1r')": ["", "minIP"],
    "('Pha_Images', 'GR', '*swi3d1r')": ["part-phase", "GRE"],
    "('localizer_haste', 'SE', '*h2d1_148')": ["acq-localizer", "T2w"],
    "('memprage_wip900', 'GR\\\\IR', 'tfl3d4_ns')": ["acq-memprage", "T1w"],
    "('T1SAG+C', 'SE', '*tse2d1_4')": ["acq-sag", "T1w"],
    "('Inline_Morpho_Results', '', '')": ["", ""], # TO BE REMOVED
    "('T1SAG', 'SE', '*tse2d1_4')": ["acq-sag", "T1w"], 
    "('T2SAG', 'SE', '*tseR2d1rr21')": ["acq-sag", "T2w"],
    "('t2_tse_sag_384', 'SE', '*tseR2d1rr21')": ["acq-sagTse", "T2w"],
    "('t1_tse_sag_384', 'SE', '*tse2d1_4')": ["acq-sagTse", "T1w"],
    "('memprage_wip900_combEcho_LabelIm', 'GR\\\\IR', 'tfl3d4_ns')": ["acq-combEchoLabelIm", "T1w"], # TO BE REMOVED
    "('memprage_wip900_combEcho_SkullStrip', 'GR\\\\IR', 'tfl3d4_ns')": ["acq-combEchoSkullStrip", "T1w"], # TO BE REMOVED
    "('memprage_wip900_combEcho', 'GR\\\\IR', 'tfl3d4_ns')": ["acq-memprage", "T1w"],
    "('t2_tse_rst_sag', 'SE', '*tseR2d1rr33')": ["acq-sagTseRst", "T2w"],
    "('t2_tirm_tra_dark-fluid_p2', 'SE\\\\IR', '*tir2d1_25')": ["acq-axTirmDf", "T2w"],
    "('<MPR Thick Range>', 'GR\\\\IR', '*tfl3d1')": ["acq-sag", "T1w"], # CHECKED
    "('PosDisp: [25] t2_tse_rst_tra ', 'SE', '*tseR2d1rr31')": ["acq-axTseRstLocalizer", "T2w"], 
    "('t1_fl3d_TRA_fs', 'GR', '*fl3d1rr')": ["acq-ax", "T1w"], 
    "('t1_tse_sag_384', 'SE', '*tse2d1_3')": ["acq-sagTse", "T1w"], 
    "('C-t1_tse_sag_384p2', 'SE', '*tse2d1_3')": ["acq-sagTse", "T1w"], 
    "('C-t1_tse_tra', 'SE', '*tse2d1_3')": ["acq-axTse", "T1w"], 
    "('t1_tse_sag', 'SE', '*tse2d1_3')": ["acq-sagTse", "T2w"],
    "('t2_tse_rst_tra_384', 'SE', '*tseR2d1rr31')": ["acq-axTseRst", "T2w"],
    "('t2_tse_tra', 'SE', '*tse2d1_21')": ["acq-axTse", "T2w"],
    "('t1_fl2d_sag_ep', 'GR', '*fl2d1')": ["acq-sag", "T1w"],
    "('t1_fl2d_tra', 'GR', '*fl2d1')": ["acq-ax", "T1w"],
    "('<MPR Thick Range>', 'GR\\\\IR', '*tfl3d1_ns')": ["acq-sag", "T1w"], # CHECKED
    "('PosDisp: [18] t1_fl2d_sag_ep ', 'SE', '*h2d1_154')": ["acq-sagLocalizer", "T1w"], 
    "('PosDisp: [12] t2_tse_rst_tra ', 'SE', '*h2d1_154')": ["acq-axTseRstLocalizer", "T2w"],
    "('t2_tse_rst_sag', 'SE', '*tseR2d1rr19')": ["acq-sagTseRst", "T2w"],
    "('t2_tse_rst_tra', 'SE', '*tseR2d1rr27')": ["acq-axTseRst", "T2w"],
    "('t2_tse_rst_tra_384', 'SE', '*tseR2d1rr27')": ["acq-axTseRst", "T2w"],
    "('t2_tse_tra', 'SE', '*tse2d1_15')": ["acq-axTse", "T2w"],
    "('t2_tse_rst_sag_FIL', 'SE', '*tseR2d1rr33')": ["acq-sagTseRst", "T2w"], 
    "('<MPR Range>', 'GR\\\\IR', '*tfl3d1_ns')": ["", "T1w"], # CHECKED
    "('<MPR Collection>', 'GR\\\\IR', '*tfl3d1_ns')": ["", "T1w"], # TO BE REMOVED
    "('localizer', 'GR', '*fl2d1')": ["acq-localizer", "T1w"], 
    "('t2_tse_tra_320_p2-19', 'SE', '*tseR2d1rs20')": ["acq-axTse", "T2w"], 
    "('PosDisp: [11] t2_tse_rst_tra', 'SE', '*h2d1_154')": ["acq-axTseRstLocalizer", "T2w"], # CHECKED
    "('PosDisp: [22] t1_fl2d_tra_FIL', 'SE', '*h2d1_148')": ["acq-axLocalizer", "T1w"], # TO BE REMOVED
    "('t1_tse_tra', 'SE', '*tse2d1_3')": ["acq-axTse", "T1w"], 
    "('T-t1_tse_sag_384_P2', 'SE', '*tse2d1_5')": ["acq-sagTse", "T1w"],
    "('t2_tse_rst_tra_384_FIL', 'SE', '*tseR2d1rr31')": ["acq-axTseRst", "T2w"],
    "('t1_tse_tra_FIL', 'SE', '*tse2d1_5')": ["acq-axTse", "T1w"],
    "('PosDisp: [12] t2_tse_rst_tra', 'SE', '*h2d1_154')": ["acq-axTseRstLocalizer", "T2w"] 
    
}

Only one modification.
I was also having doubts with the tirm images: I was wondering if they were FLAIR images, but it seems that not.

Waiting for @jcohenadad and @NathanMolinier feedback to apply the changes

plbenveniste · 2024-09-10T21:21:58Z

Modifications were done.
Here is the comment I added in the PR

I ran the code python code/update_anat_suffixes.py -i ~/update_gitannex/ms-nmo-beijing/
This code does the following things in order:

remove files which were identified as unwanted in this comment Create a git-annex repo for data from beijing #14 (comment)

remove files which were empty (empty array)

rename files with the appropriate suffix: if the file name already exist, the new file takes run-0X (with X starting at 2 and which is not currently used)

finally I rename files which don't include run-0X but have other similar files that contain run-02, run-03, ... I add run-01 as it was not done in the above

It's now ready for review !

plbenveniste · 2024-10-10T20:55:32Z

Referencing this for the choice of some of the suffixes for swi files: reference link.
That's how we chose the suffix GRE, swi and minIP.

plbenveniste · 2024-11-04T19:12:42Z

The PR was merged.

Here is the final message in the PR:

As discussed in-person: there are no problems with the anat/ folders, but it's still unclear how to completely fix the other folders. So, I'll do a rebase to split this branch in two:
I'll` keep the anat/ changes in this branch, so we can merge this PR.
I'll create a separate branch on top for the other changes, open a new PR with a link to this PR, and we can let it go for now. That way, the work won't be completely lost if we need it in the future.

plbenveniste added the data label May 28, 2024

plbenveniste self-assigned this May 28, 2024

mguaypaq mentioned this issue Jun 11, 2024

Add a first little bit of dataset admin guide neuropoly/intranet.neuro.polymtl.ca#126

Merged

plbenveniste closed this as completed Nov 4, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Create a git-annex repo for data from beijing #14

Create a git-annex repo for data from beijing #14

plbenveniste commented May 28, 2024 •

edited

Loading

jcohenadad commented May 28, 2024

mguaypaq commented Jun 11, 2024

plbenveniste commented Jun 11, 2024

plbenveniste commented Jun 13, 2024

plbenveniste commented Jun 14, 2024

plbenveniste commented Jun 17, 2024 •

edited

Loading

plbenveniste commented Jun 17, 2024

plbenveniste commented Jul 10, 2024

jcohenadad commented Jul 10, 2024

plbenveniste commented Jul 10, 2024

plbenveniste commented Jul 22, 2024 •

edited

Loading

plbenveniste commented Jul 23, 2024 •

edited

Loading

jcohenadad commented Jul 23, 2024

plbenveniste commented Jul 24, 2024

plbenveniste commented Aug 12, 2024

plbenveniste commented Sep 10, 2024

plbenveniste commented Oct 10, 2024

plbenveniste commented Nov 4, 2024

Create a git-annex repo for data from beijing #14

Create a git-annex repo for data from beijing #14

Comments

plbenveniste commented May 28, 2024 • edited Loading

jcohenadad commented May 28, 2024

mguaypaq commented Jun 11, 2024

plbenveniste commented Jun 11, 2024

plbenveniste commented Jun 13, 2024

plbenveniste commented Jun 14, 2024

plbenveniste commented Jun 17, 2024 • edited Loading

plbenveniste commented Jun 17, 2024

plbenveniste commented Jul 10, 2024

jcohenadad commented Jul 10, 2024

plbenveniste commented Jul 10, 2024

plbenveniste commented Jul 22, 2024 • edited Loading

plbenveniste commented Jul 23, 2024 • edited Loading

jcohenadad commented Jul 23, 2024

plbenveniste commented Jul 24, 2024

plbenveniste commented Aug 12, 2024

plbenveniste commented Sep 10, 2024

plbenveniste commented Oct 10, 2024

plbenveniste commented Nov 4, 2024

plbenveniste commented May 28, 2024 •

edited

Loading

plbenveniste commented Jun 17, 2024 •

edited

Loading

plbenveniste commented Jul 22, 2024 •

edited

Loading

plbenveniste commented Jul 23, 2024 •

edited

Loading