pipeline not compatible with nextflow 23 => PosixFileAttributeView

[nick-youngblut]

Description of the bug

Running the latest commit of scrnaseq with nextflow 23.10.1 results in: ERROR ~ PosixFileAttributeView right after the start of the pipeline.

Updating nextflow to 24.04.3 fixes the problem.

So, it appears that the latest version of scrnaseq is not backwards compatible. If this is the case, it would be best if the pipeline checked the nextflow version on start and raised an error if the nextflow version was <24.

Command used and terminal output

nextflow run main.nf \
   -ansi-log false \
   -profile docker,test_full \
   --fasta cellranger/refdata-gex-GRCh38-2024-A/fasta/genome.fa \
   --gtf cellranger/refdata-gex-GRCh38-2024-A/genes/genes.gtf \
   --outdir TEST_OUTPUT

Relevant files

The relevant section of the .nextflow.log:

Jul-11 02:04:39.330 [Actor Thread 126] ERROR nextflow.extension.OperatorImpl - @unknown
java.lang.UnsupportedOperationException: PosixFileAttributeView
	at com.google.cloud.storage.contrib.nio.CloudStorageFileSystemProvider.getFileAttributeView(CloudStorageFileSystemProvider.java:855)
	at java.base/java.nio.file.Files.getFileAttributeView(Files.java:1790)
	at java.base/java.nio.file.CopyMoveHelper.copyToForeignTarget(CopyMoveHelper.java:151)
	at java.base/java.nio.file.CopyMoveHelper.moveToForeignTarget(CopyMoveHelper.java:193)
	at java.base/java.nio.file.Files.move(Files.java:1433)
	at nextflow.file.SimpleFileCollector.saveFile(SimpleFileCollector.groovy:102)
	at nextflow.file.FileCollector.saveTo0(FileCollector.groovy:228)
	at nextflow.file.FileCollector.saveTo(FileCollector.groovy:251)
	at nextflow.file.FileCollector$saveTo.call(Unknown Source)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:139)
	at nextflow.extension.CollectFileOp.emitItems(CollectFileOp.groovy:187)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:107)
	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1254)
	at groovy.lang.MetaClassImpl.invokeMethodClosure(MetaClassImpl.java:1042)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1128)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1030)
	at groovy.lang.Closure.call(Closure.java:427)
	at groovy.lang.Closure.call(Closure.java:416)
	at groovy.lang.Closure$call.call(Unknown Source)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
	at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:53)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:139)
	at nextflow.extension.DataflowHelper$3.afterStop(DataflowHelper.groovy:257)
	at groovyx.gpars.dataflow.operator.DataflowProcessor.fireAfterStop(DataflowProcessor.java:324)
	at groovyx.gpars.dataflow.operator.DataflowProcessorActor.afterStop(DataflowProcessorActor.java:59)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:107)
	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1254)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1030)
	at org.codehaus.groovy.runtime.InvokerHelper.invokePojoMethod(InvokerHelper.java:1024)
	at org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:1015)
	at groovyx.gpars.actor.Actor.callDynamic(Actor.java:369)
	at groovyx.gpars.actor.Actor.handleTermination(Actor.java:320)
	at groovyx.gpars.actor.AbstractLoopingActor.terminate(AbstractLoopingActor.java:204)
	at groovyx.gpars.dataflow.operator.DataflowProcessor.terminate(DataflowProcessor.java:147)
	at groovyx.gpars.dataflow.operator.DataflowProcessor$terminate$1.call(Unknown Source)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:130)
	at nextflow.extension.DataflowHelper$_subscribeImpl_closure2.doCall(DataflowHelper.groovy:290)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:107)
	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323)
	at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:274)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1030)
	at groovy.lang.Closure.call(Closure.java:427)
	at groovyx.gpars.dataflow.operator.DataflowOperatorActor.startTask(DataflowOperatorActor.java:120)
	at groovyx.gpars.dataflow.operator.DataflowOperatorActor.onMessage(DataflowOperatorActor.java:108)
	at groovyx.gpars.actor.impl.SDAClosure$1.call(SDAClosure.java:43)
	at groovyx.gpars.actor.AbstractLoopingActor.runEnhancedWithoutRepliesOnMessages(AbstractLoopingActor.java:293)
	at groovyx.gpars.actor.AbstractLoopingActor.access$400(AbstractLoopingActor.java:30)
	at groovyx.gpars.actor.AbstractLoopingActor$1.handleMessage(AbstractLoopingActor.java:93)
	at groovyx.gpars.util.AsyncMessagingCore.run(AsyncMessagingCore.java:132)
	at java.base/java.util.concurrent.ThreadPerTaskExecutor$TaskRunner.run(ThreadPerTaskExecutor.java:314)
	at java.base/java.lang.Thread.run(Thread.java:1583)
Jul-11 02:04:39.339 [Actor Thread 126] DEBUG nextflow.Session - Session aborted -- Cause: PosixFileAttributeView
Jul-11 02:04:39.365 [Actor Thread 126] DEBUG nextflow.Session - The following nodes are still active:

System information

• Hardware: GCP VM
• Executor: tried both GCP and local
• Container engine: Docker
• OS: Linux
• Pipeline version: latest commit in the master branch

[grst]

I believe this is more related to the cloud setup than the pipeline itself, judging from

CloudStorageFileSystemProvider.java:855

Probably there's a nextflow bug related to GCP file storage in nextflow < 24. The pipeline itself can run on 23.04 (which is the nf-core standard minimal version atm), this is ensured by CI.