feat: preparation for somatic cnv

[ericblanc20]

Adds several (fairly simple & simple-minded) steps required for proper CNV calling:

• guess_sex: simple inference of sex for autosome & sex chromosome coverage
• germline_snvs: simple identification of well-supported germline SNPs. The variant_calling step unfortunately cannot be used for this task, as it is designed for trios.
• somatic_variants_for_cnv: creates input for cnv tools using B-allele fractions to improve/verify CNV calls based on coverage alone. The somatic_variant_calling step cannot be used, as the somatic variants from mutect2 differ greatly when germline variants are included or not.

The current code is OK, but can certainly be improved:

• Pipes could be extended, but it needs more clever choices in germline_snvs/__init__.py
• The problems with model serialization & validation should be understood & fixed: at the moment, automatic alias generation in bcftools models seems to work in isolation, when only the step model is considered. But validation fails during registration of sub-steps.
• Better design of the snappy_wrapper is probably possible. Also, the derived BcftoolsWrapper is a first attempt at streamlining UNIX-like tools (such as bcftools, bedtools, bedops, samtools, rnaqc, ...). Its design should be critically reviewed, before similar wrappers are built.
• The treatment of ignored_chroms should also be seen as a first attempt to be critically reviewed. The code in genome_windows is exercised in the ignored_chroms wrapper, called from the germline_snvs & somatic_variants_for_cnv snakefiles.

[github-actions]

• Please format your Python code with ruff: make fmt
• Please check your Python code with ruff: make check
• Please format your Snakemake code with snakefmt: make snakefmt

You can trigger all lints locally by running make lint

[coveralls]

coverage: 80.765% (-5.0%) from 85.765%
when pulling 2330461 on 587-preparation-for-somatic-cnv
into f569530 on main.

[tedil]

Phew, this is a lot! Thank you for all the work and effort that went into it:

Here are some remarks, questions & feedback, in no particular order:

• make sure VCF_TAG_PATTERN and ANNOTATION_VCF_TAG_PATTERN cover the definitions of the VCF specification
• try to use the model and its attributes where applicable instead of dict access (i.e. wf.config.property.… instead of config["step_config"][…])
• use functions for getting resources and params/args, so we can more easily add dynamic resource estimation later on
• I think "last" is an unfortunate name for an action, I'd prefer something akin to "finalize" or "gather_final_result" or …)
• make use of dictify and listify consistently
• should we produce a little illustration for the documentation on how the steps introduced here interact/intertwine/support one another?
• avoid code duplication:
  • I think _collapsed_arg_value and collapse_args appear multiple times with basically the same code; in this case, we may also find a cleaner way to do it, but it's fine for now!
  • get_args is often the same 2 lines I think, is that not part of Base/AbstractStepXYZ already?
• hardcoded extra_args seem weird to me, can they not be part of the default config instead?
• instead of using the do_md5 argument, we could add an automatic check for this
• TODO: check regular expressions for correctness
• use named groups in regexes
• for even more comments, see the respective line comments ;)