Allow for .bids-validator-config.json filename

[yarikoptic]

We have a legitimate configuration for (legacy) bids-validation within dataset in .bids-validator-config.json .
git push was denied with no specific file mentioned but I guess it is because of that file.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 44.96%. Comparing base (5de073c) to head (f54e4b4).

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3141      +/-   ##
==========================================
+ Coverage   44.03%   44.96%   +0.93%     
==========================================
  Files         593      593              
  Lines       37807    37807              
  Branches     1125     1159      +34     
==========================================
+ Hits        16647    16999     +352     
+ Misses      20958    20617     -341     
+ Partials      202      191      -11     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[yarikoptic]

Well, there seems to be quite a bit of logic linked to having .bidsignore file... I wonder if I should also add more on supporting .bids-validator-config.json? (e.g. .gitattributes config for it etc). Or may be it is somehow automatically disabled (e.g. a custom config provided to invocation of bids-validator)?

But in either case -- why to forbid other .dotfiles in general???? If I were on a mission to abuse openneuro I could stash my abusing materials under .datalad/ or smth like that which is ignored already.

[yarikoptic]

yikes, tests fail with

........................................................................ [ 75%]
Fatal Python error: Segmentation fault

unrelated, filed

• test_move_files_nesting results(?) in "Fatal Python error: Segmentation fault" once in a while #3142

[effigies]

All this would do is not fail on the presence of this file. OpenNeuro will continue to pass its own configuration, so these files will not affect the validation of the dataset on OpenNeuro. Is your goal to override OpenNeuro's configuration? What are your use cases? I doubt we'd permit this.

[yarikoptic]

All this would do is not fail on the presence of this file. OpenNeuro will continue to pass its own configuration, so these files will not affect the validation of the dataset on OpenNeuro.

that would be unfortunate but I think less critical -- most of them are warnings so we could ensure we are ignoring what we know is Ok and then would react on any new warning or error if they emerge.

here is the content of that file:

{
  "ignore": [
    "INCONSISTENT_PARAMETERS",
    "INCONSISTENT_SUBJECTS",
    "MISSING_SESSION",
    "STIMULUS_FILE_MISSING",
    "UNUSED_STIMULUS"
  ]
}

I really think that providing targetted configuration for validation is ways better than the current facilitated practice many folks adhere to - ignoring all tricky files via .bidsignore

check out those counts and note even a case of abuse of .bidsignore as a directory

$> wc -l ds*/.bidsignore | sort -n | nl | tail
wc: ds001583/.bidsignore: Is a directory
   245	     33 ds002041/.bidsignore
   246	     34 ds000108/.bidsignore
   247	     39 ds003684/.bidsignore
   248	     64 ds000240/.bidsignore
   249	     82 ds002320/.bidsignore
   250	     87 ds004475/.bidsignore
   251	    216 ds004808/.bidsignore
   252	    553 ds004505/.bidsignore
   253	   2674 ds000221/.bidsignore
   254	   4576 total

and cases like

$> cat ds000108/.bidsignore
sub-01/func
sub-02/func
...

[yarikoptic]

@effigies @nellh @rwblair WDYT about a global solution to allow / use the provided .bids-validator-config.json? e.g.

• where is the one you provide from openneuro? may be check could be added that if there is one provided in dataset, it should include what you require for openneuro?
• validator could be made to consume multiple files and taking them as a union??

[yarikoptic]

ping on this issue. Here is my current list of .dotfiles

(deno) yoh@typhon:/mnt/DATA/data/yoh/1076_spacetop$ ls -ld .*
-rw-r--r-- 1 yoh yoh  31 Oct  2 14:32 .bidsignore
-rw-r--r-- 1 yoh yoh 156 Sep  6 11:18 .bids-validator-config.json
drwxr-xr-x 1 yoh yoh  40 Sep  4 12:49 .datalad
drwxr-xr-x 1 yoh yoh 342 Oct 25 14:51 .git
-rw-r--r-- 1 yoh yoh 479 Jun 24 02:24 .gitattributes
-rw-r--r-- 1 yoh yoh  26 May 29 13:47 .gitignore
-rw-r--r-- 1 yoh yoh 506 Sep  4 12:35 .gitmodules

and IMHO they are all legitimate ;-) Even if my .bids-validator-config.json gets ignored -- I will "survive" while progressing forward while openneuro still complains - I will just finally ignore stuff in .bidsignore : ATM I cannot even push!

[yarikoptic]

me complaining about all kinds of legitimate .dotfiles

ping on this:

(deno) yoh@typhon:/mnt/DATA/data/yoh/1076_spacetop$ git push openneuro-git master:main
Enumerating objects: 8110, done.
Counting objects: 100% (7689/7689), done.
Delta compression using up to 32 threads
Compressing objects: 100% (2549/2549), done.
Writing objects: 100% (7264/7264), 6.80 MiB | 11.58 MiB/s, done.
Total 7264 (delta 5949), reused 5764 (delta 4696), pack-reused 0
remote: Resolving deltas: 100% (5949/5949), completed with 321 local objects.
remote: .bidsignore
remote: .datalad/.gitattributes
remote: .datalad/config
remote: .gitattributes
remote: .gitignore
remote: .gitmodules
remote: 
remote: -------------------------------------------------------------------------
remote: Your push was rejected because it failed validation.
remote: Please test with bids-validator locally to resolve any errors before pushing.
remote: -------------------------------------------------------------------------
To https://openneuro.org/git/0/ds005256
 ! [remote rejected]       master -> main (pre-receive hook declined)
error: failed to push some refs to 'https://openneuro.org/git/0/ds005256'

[effigies]

This is #3247.