VEP error

[mriduljohari]

Hi,

I got a strange error while running the load_data.sh. The VEP is not able to index the bgzipped fasta file. The fasta file is present in the required directory and has the read and write permissions for the docker.

• I downloaded the reference data again but still faced the same error.
• I then manually re-downloaded the fasta file and gunzipped and then bgzipped it but got the same error.
• I also indexed the Homo_sapiens.GRCh38.dna.primary_assembly.fa.gz file with bgzip and samtools faidx but still the same error.
• I pulled the latest version of the containers but still have the same error.

Below is the output:

ERROR: [pid 64] Worker Worker(salt=481523203, workers=1, host=f56c78818bd1, username=root, pid=64) failed    SeqrVCFToMTTask(sourc
e_paths=/input_vcfs/GRCh38/09-001_GRCh38_AS.leftaligned_trimmed.vcf.gz, dest_path=/input_vcfs/GRCh38/09-001_GRCh38_AS.mt, genome_v
ersion=38, vep_runner=VEP, reference_ht_path=/seqr-reference-data/GRCh38/combined_reference_data_grch38.ht, clinvar_ht_path=/seqr-
reference-data/GRCh38/clinvar.GRCh38.ht, hgmd_ht_path=None, sample_type=WES, dont_validate=False, dataset_type=VARIANTS, remap_pat
h=, subset_path=, vep_config_json_path=/vep_configs/vep-GRCh38-loftee.json, grch38_to_grch37_ref_chain=gs://hail-common/references
/grch38_to_grch37.over.chain.gz)
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/luigi/worker.py", line 199, in run
    new_deps = self._run_get_new_deps()
  File "/usr/local/lib/python3.7/site-packages/luigi/worker.py", line 141, in _run_get_new_deps
    task_gen = self.task.run()
  File "/hail-elasticsearch-pipelines/luigi_pipeline/seqr_loading.py", line 86, in run
    self.read_vcf_write_mt()
  File "/hail-elasticsearch-pipelines/luigi_pipeline/seqr_loading.py", line 118, in read_vcf_write_mt
    mt.write(self.output().path, stage_locally=True, overwrite=True)
  File "<decorator-gen-1154>", line 2, in write 
  File "/usr/local/lib/python3.7/site-packages/hail/typecheck/check.py", line 577, in wrapper
  return __original_func(*args_, **kwargs_)
  File "/usr/local/lib/python3.7/site-packages/hail/matrixtable.py", line 2556, in write
    Env.backend().execute(ir.MatrixWrite(self._mir, writer))
  File "/usr/local/lib/python3.7/site-packages/hail/backend/py4j_backend.py", line 104, in execute
    self._handle_fatal_error_from_backend(e, ir)
  File "/usr/local/lib/python3.7/site-packages/hail/backend/backend.py", line 181, in _handle_fatal_error_from_backend
    raise err
  File "/usr/local/lib/python3.7/site-packages/hail/backend/py4j_backend.py", line 98, in execute
    result_tuple = self._jbackend.executeEncode(jir, stream_codec)
  File "/usr/local/lib/python3.7/site-packages/py4j/java_gateway.py", line 1305, in __call__
    answer, self.gateway_client, self.target_id, self.name)
  File "/usr/local/lib/python3.7/site-packages/hail/backend/py4j_backend.py", line 31, in deco
    raise fatal_error_from_java_error_triplet(deepest, full, error_id) from None
hail.utils.java.FatalError: HailException: VEP command '/vep --format vcf --json --everything --allele_number --no_stats --cache --offline --minimal --assembly GRCh38 --dir_cache /vep_data --fasta /vep_data/homo_sapiens/Homo_sapiens.GRCh38.dna.primary_assembly.fa.gz --plugin LoF,gerp_bigwig:/vep_data/loftee_data/GRCh38/gerp_conservation_scores.homo_sapiens.GRCh38.bw,human_ancestor_fa:/vep_data/loftee_data/GRCh38/human_ancestor.fa.gz,conservation_file:/vep_data/loftee_data/GRCh38/loftee.sql -o STDOUT' failed with non-zero exit status 2
  VEP Error output:

-------------------- EXCEPTION --------------------
MSG: ERROR: Cannot index bgzipped FASTA file with Bio::DB::Fasta

STACK Bio::EnsEMBL::Variation::Utils::FastaSequence::setup_fasta /ensembl-vep-release-99/Bio/EnsEMBL/Variation/Utils/FastaSequence.pm:210
STACK Bio::EnsEMBL::VEP::BaseVEP::fasta_db /ensembl-vep-release-99/modules/Bio/EnsEMBL/VEP/BaseVEP.pm:477
STACK Bio::EnsEMBL::VEP::Runner::init /ensembl-vep-release-99/modules/Bio/EnsEMBL/VEP/Runner.pm:129
STACK Bio::EnsEMBL::VEP::Runner::run /ensembl-vep-release-99/modules/Bio/EnsEMBL/VEP/Runner.pm:194
STACK toplevel /vep:224
Date (localtime)    = Fri Jul 22 08:36:18 2022
Ensembl API version = 99

[hanars]

Can you please include the full command you are running to get this error? Also, is this the same error you got every time you ran the pipeline, or is this only the error you got after the latest time where you bgzipped/indexed the fasta file?

[hanars]

Can you please also provide the error output when you run the command?

[anjalijain22]

Sure! Its the same as @mriduljohari's

MSG: ERROR: Cannot index bgzipped FASTA file with Bio::DB::Fasta

STACK Bio::EnsEMBL::Variation::Utils::FastaSequence::setup_fasta /ensembl-vep-release-99/Bio/EnsEMBL/Variation/Utils/FastaSequence.pm:210
STACK Bio::EnsEMBL::VEP::BaseVEP::fasta_db /ensembl-vep-release-99/modules/Bio/EnsEMBL/VEP/BaseVEP.pm:477
STACK Bio::EnsEMBL::VEP::Runner::init /ensembl-vep-release-99/modules/Bio/EnsEMBL/VEP/Runner.pm:129
STACK Bio::EnsEMBL::VEP::Runner::run /ensembl-vep-release-99/modules/Bio/EnsEMBL/VEP/Runner.pm:194
STACK toplevel /vep:224
Date (localtime)    = Thu Aug 11 20:57:48 2022
Ensembl API version = 99

[hanars]

Thanks. Can you try updating the config in the /vep_configs to reference the unzipped fasta file instead of the zipped fasta file:
Change

"--fasta", "/vep_data/homo_sapiens/Homo_sapiens.GRCh38.dna.primary_assembly.fa.gz"

to

"--fasta", "/vep_data/homo_sapiens/Homo_sapiens.GRCh38.dna.primary_assembly.fa"

[mriduljohari]

Thank you @hanars this change worked well and the files are annotated now. Off to loading them on the platform, hopefully, no big issues in the next step :)

[anjalijain22]

This fix works for me as well! Thank you for your help with this @hanars :)

[hanars]

I have updated the configuration in the latest released docker image so hopefully future users will not run into this issue