rule polish_clusters

[belgn]

I keep getting an "Error in rule polish_clusters:" using the example data and configuration. Any ideas? The log for the job is shown below. Thanks!

Targets: EGFR_917
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 30
Rules claiming more threads will be scaled down.
Job counts:
count jobs
1 cluster_consensus
1 copy_bed
1 detect_umi_consensus_fasta
2 map_consensus
1 polish_clusters
1 reads
1 reformat_consensus_clusters
1 seqkit_bam_acc_tsv
9

[Mon Nov 16 16:57:00 2020]
rule polish_clusters:
input: example_egfr_single_read_run/clustering/EGFR_917/clusters_fa, example_egfr_single_read_run/clustering/EGFR_917/smolecule_clusters.fa
output: example_egfr_single_read_run/fasta/EGFR_917_consensus_tmp, example_egfr_single_read_run/fasta/EGFR_917_consensus.bam, example_egfr_single_read_run/fasta/EGFR_917_consensus.fasta
jobid: 6
wildcards: name=example_egfr_single_read_run, target=EGFR_917
threads: 30

[Mon Nov 16 16:57:06 2020]
Error in rule polish_clusters:
jobid: 6
output: example_egfr_single_read_run/fasta/EGFR_917_consensus_tmp, example_egfr_single_read_run/fasta/EGFR_917_consensus.bam, example_egfr_single_read_run/fasta/EGFR_917_consensus.fasta
shell:

    rm -rf example_egfr_single_read_run/fasta/EGFR_917_consensus_tmp
    medaka smolecule --threads 30 --length 50 --depth 2 --model r941_min_high_g360 --method spoa example_egfr_single_read_run/clustering/EGFR_917/smolecule_clusters.fa example_egfr_single_read_run/fasta/EGFR_917_consensus_tmp 2> example_egfr_single_read_run/fasta/EGFR_917_consensus.bam_smolecule.log
    cp example_egfr_single_read_run/fasta/EGFR_917_consensus_tmp/consensus.fasta example_egfr_single_read_run/fasta/EGFR_917_consensus.fasta
    cp example_egfr_single_read_run/fasta/EGFR_917_consensus_tmp/subreads_to_spoa.bam example_egfr_single_read_run/fasta/EGFR_917_consensus.bam && cp example_egfr_single_read_run/fasta/EGFR_917_consensus_tmp/subreads_to_spoa.bam.bai example_egfr_single_read_run/fasta/EGFR_917_consensus.bam.bai
    
    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Removing output files of failed job polish_clusters since they might be corrupted:
example_egfr_single_read_run/fasta/EGFR_917_consensus_tmp
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: /home/elegen/pipeline-umi-amplicon/.snakemake/log/2020-11-16T165700.628404.snakemake.log

[cliu32]

I got the same error and would appreciate any help.
Some additional info from the log file is included below. Not sure why /clusters_fa and smolecule_clusters.fa were empty, which might have disrupted the run?

Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 4
Rules claiming more threads will be scaled down.
Job counts:
count jobs
2 cluster
2 cluster_consensus
1 copy_bed
2 detect_umi_consensus_fasta
2 detect_umi_fasta
4 map_consensus
2 polish_clusters
1 reads
2 reformat_consensus_clusters
2 reformat_filter_clusters
2 seqkit_bam_acc_tsv
22
Select jobs to execute...

[Sat Feb 6 22:33:44 2021]
rule detect_umi_fasta:
input: umi_sample/fasta_filtered
output: umi_sample/fasta_umi/DQB105:01:01:01_detected_umis.fasta
jobid: 9
wildcards: name=umi_sample, target=DQB105:01:01:01

[Sat Feb 6 22:33:44 2021]
rule copy_bed:
input: /media/cliu/ExtraDrive1/data/athlon2/pipeline-umi-amplicon/data/cdna052.targets.bed
output: umi_sample/targets.bed
jobid: 1
wildcards: name=umi_sample

[Sat Feb 6 22:33:44 2021]
rule detect_umi_fasta:
input: umi_sample/fasta_filtered
output: umi_sample/fasta_umi/DQA105:05:01:01_detected_umis.fasta
jobid: 19
wildcards: name=umi_sample, target=DQA105:05:01:01

[Sat Feb 6 22:33:44 2021]
Finished job 1.
1 of 22 steps (5%) done
[Sat Feb 6 22:33:45 2021]
Finished job 9.
2 of 22 steps (9%) done
Select jobs to execute...
[Sat Feb 6 22:33:45 2021]
Finished job 19.
3 of 22 steps (14%) done
Failed to solve scheduling problem with ILP solver. Falling back to greedy solver.Run Snakemake with --verbose to see the full solver output for debugging the problem.

[Sat Feb 6 22:33:45 2021]
rule cluster:
input: umi_sample/fasta_umi/DQB105:01:01:01_detected_umis.fasta
output: umi_sample/clustering/DQB105:01:01:01/clusters_centroid.fasta, umi_sample/clustering/DQB105:01:01:01/clusters_consensus.fasta, umi_sample/clustering/DQB105:01:01:01/vsearch_clusters
jobid: 8
wildcards: name=umi_sample, target=DQB1*05:01:01:01
threads: 4

Select jobs to execute...
[Sat Feb 6 22:33:45 2021]
Finished job 8.
4 of 22 steps (18%) done
Select jobs to execute...

[Sat Feb 6 22:33:45 2021]
rule cluster:
input: umi_sample/fasta_umi/DQA105:05:01:01_detected_umis.fasta
output: umi_sample/clustering/DQA105:05:01:01/clusters_centroid.fasta, umi_sample/clustering/DQA105:05:01:01/clusters_consensus.fasta, umi_sample/clustering/DQA105:05:01:01/vsearch_clusters
jobid: 18
wildcards: name=umi_sample, target=DQA1*05:05:01:01
threads: 4

[Sat Feb 6 22:33:45 2021]
Finished job 18.
5 of 22 steps (23%) done
Select jobs to execute...

[Sat Feb 6 22:33:45 2021]
rule reformat_filter_clusters:
input: umi_sample/clustering/DQA105:05:01:01/clusters_consensus.fasta, umi_sample/clustering/DQA105:05:01:01/vsearch_clusters
output: umi_sample/clustering/DQA105:05:01:01/clusters_fa, umi_sample/stats/DQA105:05:01:01_vsearch_cluster_stats.tsv, umi_sample/clustering/DQA105:05:01:01/smolecule_clusters.fa
jobid: 17
wildcards: name=umi_sample, target=DQA105:05:01:01

[Sat Feb 6 22:33:45 2021]
rule reformat_filter_clusters:
input: umi_sample/clustering/DQB105:01:01:01/clusters_consensus.fasta, umi_sample/clustering/DQB105:01:01:01/vsearch_clusters
output: umi_sample/clustering/DQB105:01:01:01/clusters_fa, umi_sample/stats/DQB105:01:01:01_vsearch_cluster_stats.tsv, umi_sample/clustering/DQB105:01:01:01/smolecule_clusters.fa
jobid: 7
wildcards: name=umi_sample, target=DQB105:01:01:01

[Sat Feb 6 22:33:45 2021]
Finished job 7.
6 of 22 steps (27%) done
[Sat Feb 6 22:33:45 2021]
Finished job 17.
7 of 22 steps (32%) done
Select jobs to execute...

[Sat Feb 6 22:33:45 2021]
rule polish_clusters:
input: umi_sample/clustering/DQA105:05:01:01/clusters_fa, umi_sample/clustering/DQA105:05:01:01/smolecule_clusters.fa
output: umi_sample/fasta/DQA105:05:01:01_consensus_tmp, umi_sample/fasta/DQA105:05:01:01_consensus.bam, umi_sample/fasta/DQA105:05:01:01_consensus.fasta
jobid: 16
wildcards: name=umi_sample, target=DQA105:05:01:01
threads: 4

Select jobs to execute...
[Sat Feb 6 22:33:46 2021]
Error in rule polish_clusters:
jobid: 16
output: umi_sample/fasta/DQA105:05:01:01_consensus_tmp, umi_sample/fasta/DQA105:05:01:01_consensus.bam, umi_sample/fasta/DQA1*05:05:01:01_consensus.fasta
shell:

    rm -rf umi_sample/fasta/DQA1*05:05:01:01_consensus_tmp
    medaka smolecule --threads 4 --length 50 --depth 2 --model r941_min_high_g360 --method spoa umi_sample/clustering/DQA1*05:05:01:01/smolecule_clusters.fa umi_sample/fasta/DQA1*05:05:01:01_consensus_tmp 2> umi_sample/fasta/DQA1*05:05:01:01_consensus.bam_smolecule.log
    cp umi_sample/fasta/DQA1*05:05:01:01_consensus_tmp/consensus.fasta umi_sample/fasta/DQA1*05:05:01:01_consensus.fasta
    cp umi_sample/fasta/DQA1*05:05:01:01_consensus_tmp/subreads_to_spoa.bam umi_sample/fasta/DQA1*05:05:01:01_consensus.bam && cp umi_sample/fasta/DQA1*05:05:01:01_consensus_tmp/subreads_to_spoa.bam.bai umi_sample/fasta/DQA1*05:05:01:01_consensus.bam.bai
    
    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Removing output files of failed job polish_clusters since they might be corrupted:
umi_sample/fasta/DQA1*05:05:01:01_consensus_tmp
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: /media/cliu/ExtraDrive1/data/athlon2/pipeline-umi-amplicon/.snakemake/log/2021-02-06T223344.848636.snakemake.log