Automating development of pipeline nf-tests

[nschcolnicov]

I recently had to develop nf-tests in bulk for nf-core/smrnaseq, and developed a series of scripts to achieve this in a semi-automated way. There are many improvements to be done, in terms of optimization and further automation, but the scripts were able to succesfully generate working nf-tests which are now part of the dev branch (nf-core/smrnaseq#381).

There is already a debate on whether if we should validate the process output channels or the actual files published in the outdir, my approach does the latter.

Would love to hear your thoughts about adding a similar feature to the nf-test tools.

https://github.com/nschcolnicov/nf-test-helper

@lukfor @sateeshperi @seppinho @edmundmiller

[maxulysse]

Hi,
I've going down the same road but for nf-core/rnaseq (nf-core/rnaseq#1360).
I went through a similar process but with less python and less automation for now.
I'm also wondering if we should be checking that the files are published which is what I think we're currently doing with the pipeline level tests (plus the occasional md5sum), but I think it would make more sense to emit channels out or the pipeline and tests that similarly to what we do at the module/subworkflow level (but currently happy with what we're doing).
Main struggle I had was hitting the more than 254 item in a collection to snapshot, which let me to split the different files by tools, and I guess that's exactly why @nschcolnicov came up with the same idea.
It's a bit cumbersome to relaunch the test at the pipeline level since you cannot resume, so I'm also wondering if it could be an option to help out for faster prototyping.

[sateeshperi]

hi @lukfor

@maxulysse @pinin4fjords recently came up with these groovy functions:
UTILS.groovy
UTILS_PIPELINE.groovy

to semi-automate the capture of stable and unstable elements in snapshots for pipeline level.

This simplifies a test much further (example)

Would be curious to hear your thoughts and if you see any room for improvement and integration ?

[maxulysse]

Would you say that the nft-ignore idea should be in the plugin?

[lukfor]

My first feeling would be yes. Then the 'pipeline-snapshots' plugin would bring the functionality and the configuration.

[maxulysse]

ok, then I'll try to include all that nicely in https://github.com/nf-core/nft-utils/

[lukfor]

cool! Thanks 🙏 Looking forwards to see you all in Barcelona

[maxulysse]

So as you probably already noticed with
askimed/nf-test-plugins#9
I have a first release of https://github.com/nf-core/nft-utils/
There is obviously room for improvements as I haven't written pure JAVA in such a long time, but at least it's not Python 😆.