Inconsistent behaviour for "empty" CSV column of unrequired file

[awgymer]

A csv can be "quoted" or not.

If there is a field defined like so:

"bai": {
    "type": "string",
    "format": "file-path",
    "exists": true,
    "pattern": "^\\S+\\.bai$",
    "errorMessage": "If provided the BAM index (bai) must exist and have extension '.bai'"
},

then the following csv have different behaviour:

Unquoted "empty" field parses fine as expected.

field1,bai
myfield,

Quoted "empty" field errors out.

field1,bai
myfield,""

ERROR ~ ERROR: Validation of 'input' file failed!

 -- Check '.nextflow.log' file for details
The following errors have been detected:

* -- Entry 1 - bai: the file or directory '""' does not exist.
* -- Entry 1 - bai: If provided the BAM index (bai) must exist and have extension '.bai' ("")

This feels somewhat counterintuitive as an empty string is obviously not a path, it's an empty column.

Not sure if this behaviour is to be expected and I am misunderstanding or whether we can somehow improve this to behave more uniformly

[nvnieuwk]

I don't think this is a bug since the field isn't actually empty, but contains an empty string.

[awgymer]

But in a quoted csv then "" is the same as a truly empty field.

[nvnieuwk]

Have you tried this with the unreleased version? Does it work in the same way there? My guess is that the .splitCsv function of nextflow is the issue here. When a CSV field is truly empty, the function returns null but when it contains an empty string it returns "".

[awgymer]

No I haven't, I guess maybe with the new castToType this would be fixed as we do =="": return null

[nvnieuwk]

Yes that will probably change that behaviour :)

[nvnieuwk]

Hi @awgymer can you try this out with nf-schema?