Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error executing process > 'NFCORE_RNASEQ:RNASEQ:INPUT_CHECK:SAMPLESHEET_CHECK (samples.csv)' #1056

Closed
animesh opened this issue Jul 4, 2023 · 6 comments
Closed

Error executing process > 'NFCORE_RNASEQ:RNASEQ:INPUT_CHECK:SAMPLESHEET_CHECK (samples.csv)' #1056

animesh opened this issue Jul 4, 2023 · 6 comments
Labels
bug Something isn't working

Comments

@animesh
Copy link

animesh commented Jul 4, 2023 •
edited by ewels
Loading

Description of the bug

Trying to run the pipeline with samples provided as

at samples.csv
samples,fastq_1,fastq_2,strandedness
s5,/cluster/projects/nn9036k/rnaSeqChk/S5-QUALITY_PASSED_R1.fastq,/cluster/projects/nn9036k/rnaSeqChk/S5-QUALITY_PASSED_R2.fastq,auto
s4,/cluster/projects/nn9036k/rnaSeqChk/S40-QUALITY_PASSED_R1.fastq,/cluster/projects/nn9036k/rnaSeqChk/S40-QUALITY_PASSED_R2.fastq,auto
s3,/cluster/projects/nn9036k/rnaSeqChk/S3-QUALITY_PASSED_R1.fastq,/cluster/projects/nn9036k/rnaSeqChk/S3-QUALITY_PASSED_R2.fastq,auto
s2,/cluster/projects/nn9036k/rnaSeqChk/S2-QUALITY_PASSED_R1.fastq,/cluster/projects/nn9036k/rnaSeqChk/S2-QUALITY_PASSED_R2.fastq,auto

but it fails with following error?

Command used and terminal output

$HOME/bin/nextflow run nf-core/rnaseq --outdir output --input samples.csv  --genome GRCh37 -profile singularity
N E X T F L O W  ~  version 23.04.2
Launching `https://github.com/nf-core/rnaseq` [chaotic_ptolemy] DSL2 - revision: 3bec2331ca [master]


------------------------------------------------------
                                        ,--./,-.
        ___     __   __   __   ___     /,-._.--~'
  |\ | |__  __ /  ` /  \ |__) |__         }  {
  | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                        `._,._,'
  nf-core/rnaseq v3.12.0-g3bec233
------------------------------------------------------
Core Nextflow options
  revision       : master
  runName        : chaotic_ptolemy
  containerEngine: singularity
  launchDir      : /cluster/projects/nn9036k/rnaSeqChk
  workDir        : /cluster/projects/nn9036k/rnaSeqChk/work
  projectDir     : /cluster/home/ash022/.nextflow/assets/nf-core/rnaseq
  userName       : ash022
  profile        : singularity
  configFiles    : /cluster/home/ash022/.nextflow/assets/nf-core/rnaseq/nextflow.config

Input/output options
  input          : samples.csv
  outdir         : output

Reference genome options
  genome         : GRCh37
  fasta          : s3://ngi-igenomes/igenomes/Homo_sapiens/Ensembl/GRCh37/Sequence/WholeGenomeFasta/genome.fa
  gtf            : s3://ngi-igenomes/igenomes/Homo_sapiens/Ensembl/GRCh37/Annotation/Genes/genes.gtf
  gene_bed       : s3://ngi-igenomes/igenomes/Homo_sapiens/Ensembl/GRCh37/Annotation/Genes/genes.bed
  star_index     : s3://ngi-igenomes/igenomes/Homo_sapiens/Ensembl/GRCh37/Sequence/STARIndex/

!! Only displaying parameters that differ from the pipeline defaults !!
------------------------------------------------------
If you use nf-core/rnaseq for your analysis please cite:

* The pipeline
  https://doi.org/10.5281/zenodo.1400710

* The nf-core framework
  https://doi.org/10.1038/s41587-020-0439-x

* Software dependencies
  https://github.com/nf-core/rnaseq/blob/master/CITATIONS.md
------------------------------------------------------
executor >  local (1)
[-        ] process > NFCORE_RNASEQ:RNASEQ:PREPARE_GENOME:GTF_GENE_FILTER                                          -
[-        ] process > NFCORE_RNASEQ:RNASEQ:PREPARE_GENOME:MAKE_TRANSCRIPTS_FASTA                                   -
[-        ] process > NFCORE_RNASEQ:RNASEQ:PREPARE_GENOME:CUSTOM_GETCHROMSIZES                                     -
[09/208b9d] process > NFCORE_RNASEQ:RNASEQ:INPUT_CHECK:SAMPLESHEET_CHECK (samples.csv)                             [  0%] 0 of 1
[-        ] process > NFCORE_RNASEQ:RNASEQ:CAT_FASTQ                                                               -
[-        ] process > NFCORE_RNASEQ:RNASEQ:FASTQ_SUBSAMPLE_FQ_SALMON:SALMON_INDEX                                  -
[-        ] process > NFCORE_RNASEQ:RNASEQ:FASTQ_SUBSAMPLE_FQ_SALMON:FQ_SUBSAMPLE                                  -
[-        ] process > NFCORE_RNASEQ:RNASEQ:FASTQ_SUBSAMPLE_FQ_SALMON:SALMON_QUANT                                  -
[-        ] process > NFCORE_RNASEQ:RNASEQ:FASTQ_FASTQC_UMITOOLS_TRIMGALORE:FASTQC                                 -
[-        ] process > NFCORE_RNASEQ:RNASEQ:FASTQ_FASTQC_UMITOOLS_TRIMGALORE:TRIMGALORE                             -
executor >  local (1)
[-        ] process > NFCORE_RNASEQ:RNASEQ:PREPARE_GENOME:GTF_GENE_FILTER                                          -
[-        ] process > NFCORE_RNASEQ:RNASEQ:PREPARE_GENOME:MAKE_TRANSCRIPTS_FASTA                                   -
[-        ] process > NFCORE_RNASEQ:RNASEQ:PREPARE_GENOME:CUSTOM_GETCHROMSIZES                                     -
[09/208b9d] process > NFCORE_RNASEQ:RNASEQ:INPUT_CHECK:SAMPLESHEET_CHECK (samples.csv)                             [100%] 1 of 1, failed: 1 ✘
[-        ] process > NFCORE_RNASEQ:RNASEQ:CAT_FASTQ                                                               -
[-        ] process > NFCORE_RNASEQ:RNASEQ:FASTQ_SUBSAMPLE_FQ_SALMON:SALMON_INDEX                                  -
[-        ] process > NFCORE_RNASEQ:RNASEQ:FASTQ_SUBSAMPLE_FQ_SALMON:FQ_SUBSAMPLE                                  -
[-        ] process > NFCORE_RNASEQ:RNASEQ:FASTQ_SUBSAMPLE_FQ_SALMON:SALMON_QUANT                                  -
[-        ] process > NFCORE_RNASEQ:RNASEQ:FASTQ_FASTQC_UMITOOLS_TRIMGALORE:FASTQC                                 -
[-        ] process > NFCORE_RNASEQ:RNASEQ:FASTQ_FASTQC_UMITOOLS_TRIMGALORE:TRIMGALORE                             -
[-        ] process > NFCORE_RNASEQ:RNASEQ:ALIGN_STAR:STAR_ALIGN_IGENOMES                                          -
[-        ] process > NFCORE_RNASEQ:RNASEQ:ALIGN_STAR:BAM_SORT_STATS_SAMTOOLS:SAMTOOLS_SORT                        -
[-        ] process > NFCORE_RNASEQ:RNASEQ:ALIGN_STAR:BAM_SORT_STATS_SAMTOOLS:SAMTOOLS_INDEX                       -
[-        ] process > NFCORE_RNASEQ:RNASEQ:ALIGN_STAR:BAM_SORT_STATS_SAMTOOLS:BAM_STATS_SAMTOOLS:SAMTOOLS_STATS    -
[-        ] process > NFCORE_RNASEQ:RNASEQ:ALIGN_STAR:BAM_SORT_STATS_SAMTOOLS:BAM_STATS_SAMTOOLS:SAMTOOLS_FLAGSTAT -
executor >  local (1)
[-        ] process > NFCORE_RNASEQ:RNASEQ:PREPARE_GENOME:GTF_GENE_FILTER                                          -
[-        ] process > NFCORE_RNASEQ:RNASEQ:PREPARE_GENOME:MAKE_TRANSCRIPTS_FASTA                                   -
[-        ] process > NFCORE_RNASEQ:RNASEQ:PREPARE_GENOME:CUSTOM_GETCHROMSIZES                                     -
[09/208b9d] process > NFCORE_RNASEQ:RNASEQ:INPUT_CHECK:SAMPLESHEET_CHECK (samples.csv)                             [100%] 1 of 1, failed: 1 ✘
[-        ] process > NFCORE_RNASEQ:RNASEQ:CAT_FASTQ                                                               -
[-        ] process > NFCORE_RNASEQ:RNASEQ:FASTQ_SUBSAMPLE_FQ_SALMON:SALMON_INDEX                                  -
[-        ] process > NFCORE_RNASEQ:RNASEQ:FASTQ_SUBSAMPLE_FQ_SALMON:FQ_SUBSAMPLE                                  -
[-        ] process > NFCORE_RNASEQ:RNASEQ:FASTQ_SUBSAMPLE_FQ_SALMON:SALMON_QUANT                                  -
[-        ] process > NFCORE_RNASEQ:RNASEQ:FASTQ_FASTQC_UMITOOLS_TRIMGALORE:FASTQC                                 -
[-        ] process > NFCORE_RNASEQ:RNASEQ:FASTQ_FASTQC_UMITOOLS_TRIMGALORE:TRIMGALORE                             -
[-        ] process > NFCORE_RNASEQ:RNASEQ:ALIGN_STAR:STAR_ALIGN_IGENOMES                                          -
[-        ] process > NFCORE_RNASEQ:RNASEQ:ALIGN_STAR:BAM_SORT_STATS_SAMTOOLS:SAMTOOLS_SORT                        -
[-        ] process > NFCORE_RNASEQ:RNASEQ:ALIGN_STAR:BAM_SORT_STATS_SAMTOOLS:SAMTOOLS_INDEX                       -
[-        ] process > NFCORE_RNASEQ:RNASEQ:ALIGN_STAR:BAM_SORT_STATS_SAMTOOLS:BAM_STATS_SAMTOOLS:SAMTOOLS_STATS    -
[-        ] process > NFCORE_RNASEQ:RNASEQ:ALIGN_STAR:BAM_SORT_STATS_SAMTOOLS:BAM_STATS_SAMTOOLS:SAMTOOLS_FLAGSTAT -
[-        ] process > NFCORE_RNASEQ:RNASEQ:ALIGN_STAR:BAM_SORT_STATS_SAMTOOLS:BAM_STATS_SAMTOOLS:SAMTOOLS_IDXSTATS -
[-        ] process > NFCORE_RNASEQ:RNASEQ:QUANTIFY_STAR_SALMON:SALMON_QUANT                                       -
[-        ] process > NFCORE_RNASEQ:RNASEQ:QUANTIFY_STAR_SALMON:SALMON_TX2GENE                                     -
[-        ] process > NFCORE_RNASEQ:RNASEQ:QUANTIFY_STAR_SALMON:SALMON_TXIMPORT                                    -
[-        ] process > NFCORE_RNASEQ:RNASEQ:QUANTIFY_STAR_SALMON:SALMON_SE_GENE                                     -
[-        ] process > NFCORE_RNASEQ:RNASEQ:QUANTIFY_STAR_SALMON:SALMON_SE_GENE_LENGTH_SCALED                       -
[-        ] process > NFCORE_RNASEQ:RNASEQ:QUANTIFY_STAR_SALMON:SALMON_SE_GENE_SCALED                              -
[-        ] process > NFCORE_RNASEQ:RNASEQ:QUANTIFY_STAR_SALMON:SALMON_SE_TRANSCRIPT                               -
[-        ] process > NFCORE_RNASEQ:RNASEQ:DESEQ2_QC_STAR_SALMON                                                   -
[-        ] process > NFCORE_RNASEQ:RNASEQ:BAM_MARKDUPLICATES_PICARD:PICARD_MARKDUPLICATES                         -
[-        ] process > NFCORE_RNASEQ:RNASEQ:BAM_MARKDUPLICATES_PICARD:SAMTOOLS_INDEX                                -
[-        ] process > NFCORE_RNASEQ:RNASEQ:BAM_MARKDUPLICATES_PICARD:BAM_STATS_SAMTOOLS:SAMTOOLS_STATS             -
[-        ] process > NFCORE_RNASEQ:RNASEQ:BAM_MARKDUPLICATES_PICARD:BAM_STATS_SAMTOOLS:SAMTOOLS_FLAGSTAT          -
[-        ] process > NFCORE_RNASEQ:RNASEQ:BAM_MARKDUPLICATES_PICARD:BAM_STATS_SAMTOOLS:SAMTOOLS_IDXSTATS          -
[-        ] process > NFCORE_RNASEQ:RNASEQ:STRINGTIE_STRINGTIE                                                     -
[-        ] process > NFCORE_RNASEQ:RNASEQ:SUBREAD_FEATURECOUNTS                                                   -
[-        ] process > NFCORE_RNASEQ:RNASEQ:MULTIQC_CUSTOM_BIOTYPE                                                  -
[-        ] process > NFCORE_RNASEQ:RNASEQ:BEDTOOLS_GENOMECOV                                                      -
[-        ] process > NFCORE_RNASEQ:RNASEQ:BEDGRAPH_BEDCLIP_BEDGRAPHTOBIGWIG_FORWARD:UCSC_BEDCLIP                  -
[-        ] process > NFCORE_RNASEQ:RNASEQ:BEDGRAPH_BEDCLIP_BEDGRAPHTOBIGWIG_FORWARD:UCSC_BEDGRAPHTOBIGWIG         -
[-        ] process > NFCORE_RNASEQ:RNASEQ:BEDGRAPH_BEDCLIP_BEDGRAPHTOBIGWIG_REVERSE:UCSC_BEDCLIP                  -
[-        ] process > NFCORE_RNASEQ:RNASEQ:BEDGRAPH_BEDCLIP_BEDGRAPHTOBIGWIG_REVERSE:UCSC_BEDGRAPHTOBIGWIG         -
[-        ] process > NFCORE_RNASEQ:RNASEQ:QUALIMAP_RNASEQ                                                         -
[-        ] process > NFCORE_RNASEQ:RNASEQ:DUPRADAR                                                                -
[-        ] process > NFCORE_RNASEQ:RNASEQ:BAM_RSEQC:RSEQC_BAMSTAT                                                 -
[-        ] process > NFCORE_RNASEQ:RNASEQ:BAM_RSEQC:RSEQC_INNERDISTANCE                                           -
[-        ] process > NFCORE_RNASEQ:RNASEQ:BAM_RSEQC:RSEQC_INFEREXPERIMENT                                         -
[-        ] process > NFCORE_RNASEQ:RNASEQ:BAM_RSEQC:RSEQC_JUNCTIONANNOTATION                                      -
[-        ] process > NFCORE_RNASEQ:RNASEQ:BAM_RSEQC:RSEQC_JUNCTIONSATURATION                                      -
[-        ] process > NFCORE_RNASEQ:RNASEQ:BAM_RSEQC:RSEQC_READDISTRIBUTION                                        -
[-        ] process > NFCORE_RNASEQ:RNASEQ:BAM_RSEQC:RSEQC_READDUPLICATION                                         -
[-        ] process > NFCORE_RNASEQ:RNASEQ:CUSTOM_DUMPSOFTWAREVERSIONS                                             -
[-        ] process > NFCORE_RNASEQ:RNASEQ:MULTIQC                                                                 -
Execution cancelled -- Finishing pending tasks before exit
ERROR ~ Error executing process > 'NFCORE_RNASEQ:RNASEQ:INPUT_CHECK:SAMPLESHEET_CHECK (samples.csv)'

Caused by:
  Process `NFCORE_RNASEQ:RNASEQ:INPUT_CHECK:SAMPLESHEET_CHECK (samples.csv)` terminated with an error exit status (127)

Command executed:

  check_samplesheet.py \
      samples.csv \
      samplesheet.valid.csv

  cat <<-END_VERSIONS > versions.yml
  "NFCORE_RNASEQ:RNASEQ:INPUT_CHECK:SAMPLESHEET_CHECK":
      python: $(python --version | sed 's/Python //g')
  END_VERSIONS

Command exit status:
  127

Command output:
  (empty)

Command error:
  INFO:    Environment variable SINGULARITYENV_NXF_DEBUG is set, but APPTAINERENV_NXF_DEBUG is preferred
  env: can't execute 'python3
  ': No such file or directory

Work dir:
  /cluster/projects/nn9036k/rnaSeqChk/work/09/208b9d407c556428387b2cf6f2bab3

Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`

 -- Check '.nextflow.log' file for details
ERROR ~ Unexpected error [AbortedException]

 -- Check script '/cluster/home/ash022/.nextflow/assets/nf-core/rnaseq/./workflows/rnaseq.nf' at line: 671 or see '.nextflow.log' file for more details

Relevant files

No response

System information

No response
@animesh animesh added the bug Something isn't working label Jul 4, 2023
@MatthiasZepper
Copy link
Member

What happened when you went in the work.dir and reran bash .command.run ? Did you get a more comprehensive error?

From the truncated error shown, it seems that there might be a problem with the shebang interpretation, since modules/local/samplesheet_check.nf does not explicitly call python check_samplesheet.py, but just launches the script.

Maybe, due to your singularity/apptainer configuration and non-standard bind-paths, #!/usr/bin/env python3 might not evaluate to the path of a python version on the host system instead of the container?

If you like, you could create a small test script testversion.py:

#!/usr/bin/env python3

import sys
print("Version test script")
print (sys.version)

Now use the script and run

singularity exec $NXF_SINGULARITY_CACHEDIR/depot.galaxyproject.org-singularity-python-3.9--1.img "python" "./testversion.py"

as well as

singularity exec $NXF_SINGULARITY_CACHEDIR/depot.galaxyproject.org-singularity-python-3.9--1.img "./testversion.py"

If the latter doesn't work for your output is not

Version test script
3.9.5 | packaged by conda-forge | (default, Jun 19 2021, 00:32:32)

your script is not executed by the python version inside the container.

@animesh
Copy link
Author

animesh commented Jul 20, 2023

Thanks for the suggestions @MatthiasZepper 👍🏽 Moving into the directory and running didn't give any more info so i tried your second suggestion by creating the testVersion.py. I noticed that


echo $NXF_SINGULARITY_CACHEDIR

is empty to i modified the cmd to web address and that seems to work?

singularity exec https://depot.galaxyproject.org/singularity/python:3.9--1 "python" "./testVersion.py"
Version test script
3.9.5 | packaged by conda-forge | (default, Jun 19 2021, 00:32:32)
[GCC 9.3.0]

@MatthiasZepper
Copy link
Member

And what happened when you relied on env?

singularity exec "https://depot.galaxyproject.org/singularity/python:3.9--1" "./testVersion.py"

@animesh
Copy link
Author

animesh commented Aug 1, 2023

That seems to work 👍🏽

(base) [[email protected] ~/cluster/rnaSeqChk]$ singularity exec "https://depot.galaxyproject.org/singularity/python:3.9--1" "./testVersion.py"
Version test script
3.9.5 | packaged by conda-forge | (default, Jun 19 2021, 00:32:32)
[GCC 9.3.0]
(base) [[email protected] ~/cluster/rnaSeqChk]$

@MatthiasZepper
Copy link
Member

Odd. I had hoped that there was an issue with the environment, since it seemed that python3 was not found:

  env: can't execute 'python3
  ': No such file or directory

but if the test script runs as expected, this seems to be fine. I am admittedly out of ideas.

Maybe try the #rnaseq channel on Slack? There has been somebody seeking help with a similar problem a couple of months ago, but also without avail. People suspected an issue with line endings and whitespace of the sample sheet in that case, but I fail to see how that should cause this problem.

@animesh
Copy link
Author

animesh commented Aug 2, 2023

Thanks for that hint, line-ending cos it looks like the issue is probably the UNIX/DOS config of git, for reasons which are beyond my comprehension, but rm -rf .gitconfig from $HOME seems to have pushed the flow to

executor >  local (7)
[b0/915c08] process > NFCORE_RNASEQ:RNASEQ:PREPARE_GENOME:GTF_GENE_FILTER (genome.fa)                              [100%] 1 of 1 ✔
[53/24f5ef] process > NFCORE_RNASEQ:RNASEQ:PREPARE_GENOME:MAKE_TRANSCRIPTS_FASTA (rsem/genome.fa)                  [  0%] 0 of 1
[50/76b94f] process > NFCORE_RNASEQ:RNASEQ:PREPARE_GENOME:CUSTOM_GETCHROMSIZES (genome.fa)                         [100%] 1 of 1 ✔
[70/96c383] process > NFCORE_RNASEQ:RNASEQ:INPUT_CHECK:SAMPLESHEET_CHECK (samples.csv)                             [100%] 1 of 1 ✔
[-        ] process > NFCORE_RNASEQ:RNASEQ:CAT_FASTQ                                                               -
[-        ] process > NFCORE_RNASEQ:RNASEQ:FASTQ_SUBSAMPLE_FQ_SALMON:SALMON_INDEX                                  -
[e5/ca7aa2] process > NFCORE_RNASEQ:RNASEQ:FASTQ_SUBSAMPLE_FQ_SALMON:FQ_SUBSAMPLE (TK10_3)                         [100%] 3 of 3 ✔

keeping fingers crossed!

@animesh animesh closed this as completed Aug 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@animesh @MatthiasZepper