Fix space other use and correct json sidecars #193
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| name: Dataset Validator | |
| on: | |
| push: | |
| branches: [ master ] | |
| pull_request: | |
| branches: [ master ] | |
| jobs: | |
| Validate: | |
| runs-on: ubuntu-latest | |
| steps: | |
| - name: Checkout | |
| uses: actions/checkout@v2 | |
| with: | |
| # make sure to download directly from the PR's repo, whether that is this repo or a fork | |
| # By default github generates a merge commit for each PR in this repo, but only for the one branch under test | |
| # but `git-annex` needs access to *two* branches: the current branch and `git-annex` | |
| # this might be subtly buggy since it is testing the remote version, not the merged version | |
| # | |
| # *if* this is not a pull request, this will fall back to its default behaviour. | |
| repository: ${{github.event.pull_request.head.repo.full_name}} | |
| ref: ${{ github.event.pull_request.head.ref }} | |
| - name: Install dos2unix | |
| run: | | |
| # the easy way to do this: convert everything to unix and ask git if that changed anything | |
| sudo apt-get install -y dos2unix | |
| - name: Check line endings | |
| run: | | |
| # the easy way to do this: convert everything to unix and ask git if that changed anything | |
| find ./ -path ./.git -prune -o -type f -print0 | xargs -0 dos2unix -q | |
| # this version is safer, but more maintenance: | |
| #find ./ -path ./.git -prune -o -type f \( ! -name "*.nii" -a ! -name "*.nii.gz" \) -print0 | xargs -0 dos2unix -q | |
| git diff --stat --exit-code | |
| if [ "$?" -ne 0 ]; then | |
| echo "error: Windows line endings found." | |
| exit 1 | |
| fi | |
| - name: Lint participants.tsv | |
| run: .github/workflows/lint-tsv participants.tsv | |
| #- name: Update software | |
| # run: | | |
| # # do we want to do this? it's helpful to avoid testing against surprise out-of-date software, but also so slow. | |
| # sudo apt-get update && | |
| # sudo DEBIAN_FRONTEND=noninteractive apt-get -y dist-upgrade && | |
| # sudo DEBIAN_FRONTEND=noninteractive apt-get autoremove | |
| - name: Install git-annex | |
| run: | | |
| sudo apt-get install -y git-annex | |
| git config --global annex.thin true | |
| - name: Install bids-validator | |
| run: | | |
| # install proper NodeJS for bids-validator | |
| curl -sL https://deb.nodesource.com/setup_current.x | sudo -E bash - | |
| sudo apt-get install -y nodejs | |
| sudo npm install -g bids-validator | |
| - name: Set up Python | |
| uses: actions/setup-python@v2 | |
| with: | |
| python-version: 3.7 | |
| - name: Install spine-generic for acquisition parameters check | |
| run: pip install spinegeneric@git+https://github.com/spine-generic/spine-generic.git@master | |
| #- name: Increase free space | |
| # run: | | |
| # # this takes about 2 minutes but saves about 30GB, which is space we might need for git-annex. | |
| # # annex.thin saves a lot of space, but if the dataset grows beyond what Github can handle | |
| # # try enabling this. | |
| # # ref: https://github.com/actions/virtual-environments/issues/2606#issuecomment-772683150 | |
| # sudo rm -rf /usr/local/lib/android /usr/share/dotnet | |
| - name: git config | |
| run: | | |
| # this is needed by git-annex so that it can write to the local git-annex branch | |
| # the tracking information about the local copy | |
| git config --global user.name "gh-actions" | |
| git config --global user.email "gh-actions" | |
| # but actually, disable those writes. We don't need them because we're throwing away this copy immediately. | |
| # this might make the previous two lines unnecessary, but it's safer just to leave them both in | |
| git config annex.alwayscommit false | |
| - name: Download dataset | |
| run: | | |
| git fetch --depth=1 origin git-annex:git-annex # actions/checkout@v2 does a shallow checkout, so it is missing this important branch | |
| git annex init | |
| git annex get -J8 | |
| - name: Checking BIDS compliance | |
| run: bids-validator --verbose ./ | |
| - name: Checking acquisition parameters | |
| run: sg_params_checker -path-in ./ | |
| - name: Checking data consistency | |
| run: sg_check_data_consistency -path-in ./ |