Skip to content

Commit

Permalink
Merge pull request #475 from tsalo/ref/json-entity
Browse files Browse the repository at this point in the history 
[INFRA] Convert entity table to yaml
  • Loading branch information
@sappelhoff
sappelhoff authored Aug 11, 2020
2 parents abf6f73 + 177b15a commit a3f92ce
Show file tree
Hide file tree
Showing 21 changed files with 990 additions and 38 deletions.
4 changes: 2 additions & 2 deletions .gitignore
View file
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
site/
.DS_Store
src/.DS_Store
src/04-modality-specific-files/.DS_Store
.idea
venvs
28 changes: 18 additions & 10 deletions .travis.yml
View file
Original file line number Diff line number Diff line change
@@ -1,10 +1,18 @@
language: node_js
node_js:
- "10"
cache:
directories:
- node_modules # NPM packages
before_script:
- npm install `cat npm-requirements.txt`
script:
- remark src/*.md src/*/*.md --frail
matrix:
include:
- language: node_js
node_js:
- "10"
cache:
directories:
- node_modules # NPM packages
before_script:
- npm install `cat npm-requirements.txt`
script:
- remark src/*.md src/*/*.md --frail
- language: python
python: 3.7
install:
- pip install yamllint
script:
- yamllint -f standard src/schema/
53 changes: 52 additions & 1 deletion CONTRIBUTING.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,9 @@ Been here before? Already know what you're looking for in this guide? Jump to th
* [Understanding issues](#understanding-issues)
* [Commenting on a pull request](#commenting-on-a-pull-request)
* [Writing in markdown](#writing-in-markdown)
* [Make a change with a pull request](#making-a-change-with-a-pull-request)
* [Making a change with a pull request](#making-a-change-with-a-pull-request)
* [Example pull request](#example-pull-request)
* [Updating the specification schema](#updating-the-schema)
* [Fixing Remark errors from Travis](#fixing-travis-remark-errors)
* [Recognizing contributions](#recognizing-contributions)

Expand Down Expand Up @@ -224,6 +225,56 @@ GitHub has a [nice introduction](https://help.github.com/articles/github-flow/)
<img align="center" src="https://i.imgur.com/s8yELfK.png" alt="Example-Contribution" width="800"/>


## Updating the schema

Portions of the BIDS specification are defined using YAML files, in order to make the specification machine-readable.
Currently, the only portion of the specification that relies on this schema is the Entity Table, but any changes to the specification should be mirrored in the schema.

### The format of the schema

The schema reflects the files and objects in the specification, as well as associations between these objects.
Here is a list of the files and subfolders of the schema, roughly in order of importance:

- `datatypes/*.yaml`:
Data types supported by the specification.
Each datatype may support many suffixes.
These suffixes are divided into groups based on what extensions and entities are allowed for each.
Data types correspond to subfolders (e.g., `anat`, `func`) in the BIDS structure.
- `auxdatatypes/*.yaml`:
Auxiliary (not directly imaging or data-containing) data types supported by the specification.
Each auxiliary data type is associated with a set of data types, and these auxiliary data types are grouped based on what data types, extensions, and entities are allowed for each.
Examples of auxiliary data types include `channels`, `electrodes`, and `photo`.
- `entities.yaml`:
A list of entities (key/value pairs in folder and filenames) with associated descriptions and formatting rules.
The order of the entities in the file determines the order in which entities must appear in filenames.
- `top_level_files.yaml`:
Modality-agnostic files stored at the top level of a BIDS dataset.
The schema specifies whether these files are required or optional, as well as acceptable extensions for each.
- `modalities.yaml`:
Modalities supported by the specification, along with a list of associated data types.
Modalities are not reflected directly in the BIDS structure, but data types are modality-specific.
- `associated_data.yaml`:
Folders that are commonly contained within the same folder as a BIDS dataset, but which do not follow the BIDS structure internally, such as `code` or `sourcedata`.
The schema specifies which folders are accepted and whether they are required or optional.

### Making a change to the schema

#### 1. Ensure that changes to the specification are matched in the schema

The schema formalizes the rules described in the specification text, so you must ensure that any changes which impact the rules of the specification (including, but not limited to, adding new entities, suffixes, datatypes, modalities, etc.) are reflected in the schema as well.

#### 2. Generate an updated entity table before pushing your changes

Run the Python script `tools/bids_schema.py`:

```bash
python tools/bids_schema.py entity src/schema/ src/99-appendices/04-entity-table.md
```

#### 3. Push your changes

For more information on making general changes with a pull request, please review [Making a change with a pull request](#making-a-change-with-a-pull-request).

## Fixing Travis Remark errors

We use a linter called [Remarkjs](https://github.com/remarkjs/remark-lint) to ensure all of
Expand Down
65 changes: 40 additions & 25 deletions src/99-appendices/04-entity-table.md
View file
Original file line number Diff line number Diff line change
@@ -1,33 +1,48 @@
# Appendix IV: Entity table

This section compiles the entities (key-value pairs) described throughout this
specification, and establishes a common order within a filename. For example, if
a file has an acquisition and reconstruction label, the acquisition entity must
precede the reconstruction entity. REQUIRED and OPTIONAL entities for a given
file type are denoted. Entity formats indicate whether the value is alphanumeric
specification, and establishes a common order within a filename. 
For example, if a file has an acquisition and reconstruction label, the
acquisition entity must precede the reconstruction entity.
REQUIRED and OPTIONAL entities for a given file type are denoted.
Entity formats indicate whether the value is alphanumeric
(`<label>`) or numeric (`<index>`).

A general introduction to entities is given in the section on
[file name structure](../02-common-principles.md#file-name-structure)

| Entity | Subject | Session | Task | Acquisition | Contrast Enhancing Agent | Reconstruction | Phase-Encoding Direction | Run | Corresponding modality | Echo | Recording | Processed (on device) | Space | Split |
| :--------------------------------------------------------------------------------------------- | :------------ | :------------ | :------------- | :------------ | :----------------------- | :------------- | :----------------------- | :------------ | :--------------------- | :------------- | :------------------ | :-------------------- | :---------------| :-------------- |
| Format | `sub-<label>` | `ses-<label>` | `task-<label>` | `acq-<label>` | `ce-<label>` | `rec-<label>` | `dir-<label>` | `run-<index>` | `mod-<label>` | `echo-<index>` | `recording-<label>` | `proc-<label>` | `space-<label>` | `split-<index>` |
| anat<br>(T1w T2w T1rho T1map T2map T2star FLAIR FLASH PD PDmap PDT2 inplaneT1 inplaneT2 angio) | REQUIRED | OPTIONAL | | OPTIONAL | OPTIONAL | OPTIONAL | | | | | | | | |
| anat<br>(defacemask) | REQUIRED | OPTIONAL | | OPTIONAL | OPTIONAL | OPTIONAL | | | OPTIONAL | | | | | |
| func<br>(bold cbv phase sbref events) | REQUIRED | OPTIONAL | REQUIRED | OPTIONAL | OPTIONAL | OPTIONAL | OPTIONAL | OPTIONAL | | OPTIONAL | | | | |
| func<br>(physio stim) | REQUIRED | OPTIONAL | REQUIRED | OPTIONAL | | OPTIONAL | | OPTIONAL | | | OPTIONAL | OPTIONAL | | |
| dwi<br>(dwi bvec bval) | REQUIRED | OPTIONAL | | OPTIONAL | | | OPTIONAL | OPTIONAL | | | | | | |
| fmap<br>(phasediff phase1 phase2 magnitude1 magnitude2 magnitude fieldmap) | REQUIRED | OPTIONAL | | OPTIONAL | | | | OPTIONAL | | | | | | |
| fmap<br>(epi) | REQUIRED | OPTIONAL | | OPTIONAL | OPTIONAL | | REQUIRED | OPTIONAL | | | | | | |
| beh<br>(beh events) | REQUIRED | OPTIONAL | REQUIRED | OPTIONAL | | | | OPTIONAL | | | | | | |
| beh<br>(stim physio) | REQUIRED | OPTIONAL | REQUIRED | OPTIONAL | | | | OPTIONAL | | | OPTIONAL | | | |
| meg<br> | REQUIRED | OPTIONAL | REQUIRED | OPTIONAL | | | | OPTIONAL | | | | OPTIONAL | | OPTIONAL |
| eeg<br> | REQUIRED | OPTIONAL | REQUIRED | OPTIONAL | | | | OPTIONAL | | | | | | |
| ieeg<br> | REQUIRED | OPTIONAL | REQUIRED | OPTIONAL | | | | OPTIONAL | | | | | | |
| channels<br>(meg/eeg/ieeg) | REQUIRED | OPTIONAL | REQUIRED | | | | | OPTIONAL | | | | | | |
| headshape<br>(meg) | REQUIRED | OPTIONAL | | OPTIONAL | | | | | | | | | OPTIONAL | |
| markers<br>(meg) | REQUIRED | OPTIONAL | OPTIONAL | OPTIONAL | | | | | | | | | OPTIONAL | |
| photo<br>(meg/eeg/ieeg) | REQUIRED | OPTIONAL | | OPTIONAL | | | | | | | | | | |
| electrodes<br>(eeg/ieeg) | REQUIRED | OPTIONAL | | OPTIONAL | | | | | | | | | OPTIONAL | |
| events<br>(meg/eeg/ieeg) | REQUIRED | OPTIONAL | REQUIRED | | | | | OPTIONAL | | | | | | |
## Magnetic Resonance Imaging

| Entity | Subject | Session | Task | Acquisition | Contrast Enhancing Agent | Reconstruction | Phase-Encoding Direction | Run | Corresponding Modality | Echo | Recording |
|------------------------------------------------------------------------------------------------|---------------|---------------|----------------|---------------|----------------------------|------------------|----------------------------|---------------|--------------------------|----------------|---------------------|
| Format | `sub-<label>` | `ses-<label>` | `task-<label>` | `acq-<label>` | `ce-<label>` | `rec-<label>` | `dir-<label>` | `run-<index>` | `mod-<label>` | `echo-<index>` | `recording-<label>` |
| anat<br>(T1w T2w T1rho T1map T2map T2star FLAIR FLASH PD PDmap PDT2 inplaneT1 inplaneT2 angio) | REQUIRED | OPTIONAL | | OPTIONAL | OPTIONAL | OPTIONAL | | OPTIONAL | | | |
| anat<br>(defacemask) | REQUIRED | OPTIONAL | | OPTIONAL | OPTIONAL | OPTIONAL | | OPTIONAL | OPTIONAL | | |
| dwi<br>(dwi sbref) | REQUIRED | OPTIONAL | | OPTIONAL | | | OPTIONAL | OPTIONAL | | | |
| fmap<br>(phasediff phase1 phase2 magnitude1 magnitude2 magnitude fieldmap) | REQUIRED | OPTIONAL | | OPTIONAL | | | | OPTIONAL | | | |
| fmap<br>(epi) | REQUIRED | OPTIONAL | | OPTIONAL | OPTIONAL | | REQUIRED | OPTIONAL | | | |
| func<br>(bold cbv phase sbref events) | REQUIRED | OPTIONAL | REQUIRED | OPTIONAL | OPTIONAL | OPTIONAL | OPTIONAL | OPTIONAL | | OPTIONAL | |
| func<br>(physio stim) | REQUIRED | OPTIONAL | REQUIRED | OPTIONAL | | OPTIONAL | | OPTIONAL | | | OPTIONAL |

## Encephalography (EEG, iEEG, and MEG)

| Entity | Subject | Session | Task | Acquisition | Run | Processed (on device) | Space | Split |
|----------------------------|---------------|---------------|----------------|---------------|---------------|-------------------------|-----------------|-----------------|
| Format | `sub-<label>` | `ses-<label>` | `task-<label>` | `acq-<label>` | `run-<index>` | `proc-<label>` | `space-<label>` | `split-<index>` |
| eeg<br>(eeg) | REQUIRED | OPTIONAL | REQUIRED | OPTIONAL | OPTIONAL | | | |
| ieeg<br>(ieeg) | REQUIRED | OPTIONAL | REQUIRED | OPTIONAL | OPTIONAL | | | |
| meg<br>(meg) | REQUIRED | OPTIONAL | REQUIRED | OPTIONAL | OPTIONAL | OPTIONAL | | OPTIONAL |
| meg<br>(headshape) | REQUIRED | OPTIONAL | | OPTIONAL | | | OPTIONAL | |
| meg<br>(markers) | REQUIRED | OPTIONAL | OPTIONAL | OPTIONAL | | | OPTIONAL | |
| channels<br>(meg eeg ieeg) | REQUIRED | OPTIONAL | REQUIRED | | OPTIONAL | | | |
| electrodes<br>(eeg ieeg) | REQUIRED | OPTIONAL | | OPTIONAL | | | OPTIONAL | |
| events<br>(meg eeg ieeg) | REQUIRED | OPTIONAL | REQUIRED | | OPTIONAL | | | |
| photo<br>(meg eeg ieeg) | REQUIRED | OPTIONAL | | OPTIONAL | | | | |

## Behavioral Data

| Entity | Subject | Session | Task | Acquisition | Run | Recording |
|----------------------|---------------|---------------|----------------|---------------|---------------|---------------------|
| Format | `sub-<label>` | `ses-<label>` | `task-<label>` | `acq-<label>` | `run-<index>` | `recording-<label>` |
| beh<br>(stim physio) | REQUIRED | OPTIONAL | REQUIRED | OPTIONAL | OPTIONAL | OPTIONAL |
| beh<br>(events beh) | REQUIRED | OPTIONAL | REQUIRED | OPTIONAL | OPTIONAL | |
9 changes: 9 additions & 0 deletions src/schema/associated_data.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
- code/:
- required: false
- derivatives/:
- required: false
- sourcedata/:
- required: false
- stimuli/:
- required: false
15 changes: 15 additions & 0 deletions src/schema/auxdatatypes/channels.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
---
- datatypes:
- meg
- eeg
- ieeg
suffixes:
- channels
extensions:
- .json
- .tsv
entities:
sub: required
ses: optional
task: required
run: optional
14 changes: 14 additions & 0 deletions src/schema/auxdatatypes/electrodes.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
---
- datatypes:
- eeg
- ieeg
suffixes:
- electrodes
extensions:
- .json
- .tsv
entities:
sub: required
ses: optional
acq: optional
space: optional
Loading

0 comments on commit a3f92ce

Please sign in to comment.