Medaka not detecting installed tools

[BMTTMC]

Describe the bug
When attempting to run medaka_consensus within a script, it is failing due to a failure to detect installed tools (i.e. samtools, bcftools, etc.). minimap2 is found, but others are not (see the logging section).
Interestingly, samtools is used on its own from within the script without any issue.
This issue is focusing on samtools, but the issue involves the other tools listed as well. Any help would be appreciated.

Logging
When trying to run medaka_consensus (note that this behavior is consistent whether or not I specify a model):

$ medaka_consensus  -i ../testing_pipeline/output/IMP9/BRCA1/BRCA1_exon16_41242915_244_110_189/d2e6f6e1-bf1d-47cf-b1db-a4887ee99025.fastq -d ../testing_pipeline/output/IMP9/BRCA1/BRCA1_exon16_41242915_244_110_189/BRCA1_exon16_41242915_244_110_189_reference.fasta -o ../testing_pipeline/output/IMP9/BRCA1/BRCA1_exon16_41242915_244_110_189/d2e6f6e1-bf1d-47cf-b1db-a4887ee99025 -q
Attempting to automatically select model version.
WARNING: Failed to detect a model version, will use default: 'r1041_e82_400bps_sup_v5.0.0'
Checking program versions
This is medaka 2.0.0
Program    Version    Required   Pass     
bcftools   Not found  1.11       False    
bgzip      Not found  1.11       False    
minimap2   2.24       2.11       True     
samtools   Not found  1.11       False    
tabix      Not found  1.11       False   

To confirm that samtools (for example) is installed:

$ samtools --version
samtools 1.13

$ which samtools
/usr/bin/samtools

I've also confirmed that the PATH includes samtools:

$ echo $PATH
/home/ubuntu/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/ubuntu/.local/bin:/home/ubuntu/.local/bin:/usr/bin/samtools

Environment:

• Installation method: pypi
• OS: Ubuntu 22.04.5
• medaka version: 2.0.0
• AWS EC2 m3.2xlarge instance

[cjw85]

The code that performs these checks is here. You'll see it doesn't do much other than run the commands. I'm afraid I can't suggest a reason why these checks are failing.

[BMTTMC]

That's true, but it's called in report_binaries(), which will throw an error if anything is missing, preventing the script from running.

[BMTTMC]

Update: By installing all of the tools using apt (as opposed to manually installing them), they are now all detected except for samtools. I have uninstalled and reinstalled samtools, but to no avail.

[vineeth-s]

@BMTTMC if you have not solved this yet, this is an encoding issue
I stepped through the code and the decoding of what subprocessing returns is the issue for samtools

(samtools version 1.13
the issue seems to be in the Samtools compilation details section where the CFLAGS line is
CFLAGS: -g -O2 -ffile-prefix-map=�BUILDPATH�=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security
The culprits are the 2 �)

So in my medaka installation, I had to edit lib/python3.10/site-packages/medaka/__init__.py
and add decoding parameters to the line
You have to change
first_line = proc.stdout.decode().split("\n", 1)[0]
to
first_line = proc.stdout.decode('utf-8',errors='replace').split("\n", 1)[0]

and then I am able to run medaka just fine

[zwets]

Confirming the fix by @vineeth-s for distro-provided samtools on Ubuntu 24.04. proc.stdout.decode() chokes on the tofu in the output of samtools --version.